A system and method for determining clinical outcomes using signal-based feature analysis.

Abstract

The present disclosure provides a method for receiving individual electrical signals generated based on a body part, generating a plurality of extracted features based on the individual electrical signals, and identifying clinically relevant features from the plurality of extracted features, the clinically relevant features meeting a threshold determined based on a clinical outcome.

Description

Technical Field

[0001] The embodiments disclosed herein are directed to systems and methods for profiling features derived from signals (e.g., signals based on a subject's biometric cues using biometric devices including, but not limited to, wearable devices) for use in clinical outcomes. Clinical outcomes can include early detection and / or treatment of potential diseases or disorders experienced by a patient. Exemplary aspects of wearable biometric devices are also disclosed. This application claims priority to U.S. Provisional Application No. 63 / 129,357, filed Dec. 22, 2020, which is incorporated herein by reference in its entirety.

Background Art

[0002] Identifying statistical data for providing clinical outcomes (e.g., for clinical trials, for disease or disorder identification, for treatment planning, etc.) is difficult due to the type, amount, and / or depth of data available.

[0003] For example, various neuromuscular diseases affect the nerves that transmit electrical signals controlling voluntary muscles. This impairment can damage and gradually weaken the nerves, leading to muscle atrophy and death over time. One example of a neuromuscular disease is myasthenia gravis (MG), which causes facial effects such as ptosis (drooping eyelids), diplopia (double vision), and / or difficulty making facial expressions. Myasthenia gravis (MG) can also cause difficulty speaking, breathing, chewing, and / or swallowing. Traditionally, healthcare professionals screen patients for neuromuscular diseases through observation and self-assessment (e.g., patient-reported outcomes). For example, assessment may involve a clinician administering a questionnaire in which the patient ranks the observed physical effects (e.g., ptosis and gazing) and their ability to perform certain activities (e.g., closing eyes, speaking, and chewing). Such methods are inaccurate because observations are subjective and patients may adapt their behavior over time to compensate for problematic symptoms. In clinical visits, patients often underreport their chewing and swallowing symptoms and severity, especially if they have had symptoms for a long time, as they may have adapted to softer or liquid diets. As a result, current assessment methods used in clinics can lead to inaccurate and missed diagnoses and treatments. Therefore, there is an unmet medical need for objective, quantitative, and accurate assessment of symptoms and severity in patients.

[0004] The use of statistical data to generate diagnoses and treatments can provide objective, data-driven results. However, the use of statistical data is challenging due to the type, quantity, and / or depth of data available for a given trial. For example, data applicable to identifying one clinical outcome may not be applicable to identifying another. Also, given data parameters, signal acquisition mechanisms, and / or actions during signal acquisition may be optimal for the first clinical output but not for a second.

[0005] Therefore, improved techniques are needed to perform assessments, determine diagnoses, and assign treatments to patients with neuromuscular diseases. [Overview of the Initiative]

[0006] Aspects of this disclosure relate to signal-based feature analysis. In one aspect, the disclosure relates to a method comprising receiving individual electrical signals generated based on a body part, generating a plurality of extracted features based on the individual electrical signals, and identifying clinically relevant features from the plurality of extracted features, wherein the clinically relevant features satisfy a threshold determined based on a clinical outcome.

[0007] The method may also include determining a clinical outcome by applying the clinically relevant features, which may be a diagnosis or a treatment plan. The individual electrical signals may be generated based on bodily electrical signals generated by the body part. The individual electrical signals may be generated based on the movement of the body part. The individual electrical signals may be generated based on the characteristics of the body part. The multiple extracted features may be based on one or more of the following: amplitude features, zero crossing rate, standard deviation, variance, root mean square, kurtosis, frequency, bandwidth power, or skew. The individual electrical signals may be generated by a wearable device equipped with sensors, in which case the wearable device may be configured to output a mixed signal, and / or a signal separation module extracts the extracted features from the mixed signal.

[0008] For example, the signal separation module can extract features from a mixed signal by applying one or more of the following: blind signal separation, blind source separation, discrete transform, Fourier transform, integral transform, two-sided Laplace transform, Mellin transform, Hartley transform, short-time Fourier transform (or short-term Fourier transform) (STFT), rectangular mask short-time Fourier transform, charplet transform, fractional Fourier transform (FRFT), Hankel transform, Fourier-Bross-Iagornitzer transform, or linear canonical transform. A random forest algorithm may be used to score the extracted features. The threshold may be a random forest threshold, and extracted features having a random forest score equal to or greater than the random forest threshold may be identified as clinically relevant features. The threshold may be a reliability threshold, and extracted features having a reliability score equal to or greater than the reliability threshold may be identified as clinically relevant features. The reliability score may be based on one or more of the following: Spearman correlation, intraclass correlation (ICC), covariance (CV), area under the curve (AUC), clustering, or Z score.

[0009] In another embodiment, the disclosure relates to a system comprising a wearable device including a plurality of sensors, a processor, and a computer-readable data storage device storing instructions. When the processor executes the instructions, the system performs the following actions: obtain electrical activity information of a subject detected by the plurality of sensors from the wearable device, and identify clinically relevant features based on the electrical activity information.

[0010] The system may further be configured to classify the clinically relevant features as one or more diseases, determine the subject's disease based on the one or more diseases, determine the extent of the disease, and / or determine a treatment plan based on the extent of the disease. The plurality of sensors may include electroencephalography (EEG) sensors, electrooculography (EOG) sensors, electromyography (EMG) sensors, image sensors, and / or eye-tracking sensors. The clinically relevant features may be identified using machine learning algorithms. [Brief explanation of the drawing]

[0011] [Figure 1A] Figure 1A is a diagram illustrating the relationship between feature extraction and selection according to the embodiments of this disclosure. [Figure 1B] Figure 1B is a system block diagram showing an exemplary headgear according to an aspect of the present disclosure. [Figure 2] Figure 2 is a system block diagram showing an example of an environment for implementing a system and process according to an aspect of this disclosure. [Figure 3] Figure 3 is a block diagram showing an example of a controller according to an aspect of this disclosure. [Figure 4A] Figure 4A is a flowchart showing an example of a method performed by the system according to an aspect of this disclosure. [Figure 4B] Figure 4B is a flowchart showing an example of a method performed by the system according to an aspect of this disclosure. [Figure 4C] Figure 4C is a flowchart illustrating the identification and application of clinically relevant features according to aspects of this disclosure. [Figure 5] Figure 5 illustrates the limitations of existing clinical outcome scales due to their partial influence by recall bias. [Figure 6] Figure 6 shows an exemplary output measurement value according to one embodiment of the present disclosure. [Figure 7] Figure 7 shows an additional exemplary output measurement according to one aspect of the present disclosure. [Figure 8] Figure 8 is a heatmap showing the correlation between four tasks and Z-scores according to one embodiment of the present disclosure. [Figure 8-1] Figure 8-1 is a heatmap showing the correlation between four tasks and Z-scores according to one embodiment of the present disclosure. [Figure 8-2] Figure 8-2 is a heatmap showing the correlation between four tasks and Z-scores according to one embodiment of the present disclosure. [Figure 9]Figure 9 shows heatmaps of variables and activities illustrating qualitative differences between subjects and activities according to embodiments of the present disclosure. [Figure 9-1] Figure 9-1 shows heatmaps of variables and activities illustrating qualitative differences between subjects and between activities according to embodiments of the present disclosure. [Figure 9-2] Figure 9-2 shows heatmaps of variables and activities illustrating qualitative differences between subjects and between activities according to embodiments of the present disclosure. [Figure 9-3] Figure 9-3 shows heatmaps of variables and activities illustrating qualitative differences between subjects and between activities according to embodiments of the present disclosure. [Figure 9-4] Figure 9-4 shows heatmaps of variables and activities illustrating qualitative differences between subjects and between activities according to embodiments of the present disclosure. [Figure 9-5] Figure 9-5 shows heatmaps of variables and activities illustrating qualitative differences between subjects and between activities according to embodiments of the present disclosure. [Figure 9-6] Figure 9-6 shows heatmaps of variables and activities illustrating qualitative differences between subjects and between activities according to embodiments of the present disclosure. [Figure 9-7] Figure 9-7 shows heatmaps of variables and activities illustrating qualitative differences between subjects and between activities according to embodiments of the present disclosure. [Figure 9-8] Figure 9-8 shows heatmaps of variables and activities illustrating qualitative differences between subjects and between activities according to embodiments of the present disclosure. [Figure 10] Figure 10 is a Spearman plot of the data from Figure 9 according to an embodiment of the present disclosure, demonstrating that the highly correlated parameters are likely to measure similar aspects of facial biology. [Figure 10-1] Figure 10-1 is a Spearman plot of the data from Figure 9 according to an embodiment of the present disclosure, demonstrating that the highly correlated parameters are likely to measure similar aspects of facial biology. [Figure 10-2]Figure 10-2 is a diagram showing the Spearman plot of the data in Figure 9 according to an embodiment of the present disclosure, and is a diagram proving that highly correlated parameters are likely to measure similar aspects of facial biology. [Figure 10-3] Figure 10-3 is a diagram showing the Spearman plot of the data in Figure 9 according to an embodiment of the present disclosure, and is a diagram proving that highly correlated parameters are likely to measure similar aspects of facial biology. [Figure 10-4] Figure 10-4 is a diagram showing the Spearman plot of the data in Figure 9 according to an embodiment of the present disclosure, and is a diagram proving that highly correlated parameters are likely to measure similar aspects of facial biology. [Figure 10-5] Figure 10-5 is a diagram showing the Spearman plot of the data in Figure 9 according to an embodiment of the present disclosure, and is a diagram proving that highly correlated parameters are likely to measure similar aspects of facial biology. [Figure 11] Figure 11 is a diagram showing the intraclass correlation (ICC) measurement for testing the retest reliability of parameters and inferring clinical significance according to an embodiment of the present disclosure. [Figure 12] Figure 12 is a diagram showing the ICC measurement useful for testing the retest reliability of parameters and inferring clinical significance according to an embodiment of the present disclosure. [Figure 12-1] Figure 12-1 is a diagram showing the ICC measurement useful for testing the retest reliability of parameters and inferring clinical significance according to an embodiment of the present disclosure. [Figure 12-2] Figure 12-2 is a diagram showing the ICC measurement useful for testing the retest reliability of parameters and inferring clinical significance according to an embodiment of the present disclosure. [Figure 13] Figure 13 is a diagram showing the schema of the random forest approach according to an embodiment of the present disclosure. [Figure 14] Figure 14 is a diagram showing the F1 score for measuring how well a model classifies a specific activity (e.g., swallowing) according to an embodiment of the present disclosure. [Figure 15]Figure 15 is a diagram showing an improved F1 score using parameters from two rounds of feature operations according to an embodiment of the present disclosure, which shows that an improved F1 score is obtained for several activities. [Figure 16] Figure 16 shows an improved F1 score presented in histogram format according to one embodiment of the present disclosure. [Figure 17] Figure 17 shows the morning and evening F1 scores according to one embodiment of the present disclosure. [Figure 18] Figure 18 shows a chart of algorithmic options for selection according to one embodiment of the present disclosure. [Figure 19] Figure 19 shows swallowing values ​​for different bandwidths according to one embodiment of the present disclosure. [Figure 20] Figure 20 shows a cluster and corresponding channels according to an embodiment of the present disclosure. [Figure 21] Figure 21 shows the amplitude, frequency, and bandwidth power channels with other factors according to one embodiment of the present disclosure. [Figure 22] Figure 22 shows the covariance (CV) of a mixed-effects model based on morning measurements according to one embodiment of the present disclosure. [Figure 23] Figure 23 shows the measurement results of various components collected during multiple measurement times according to one embodiment of the present disclosure. [Figure 24] Figure 24 shows Z-scores for various combinations of individuals, time, and tasks according to embodiments of the present disclosure. [Figure 24-1] Figure 24-1 shows Z-scores for various combinations of individuals, time, and tasks according to embodiments of the present disclosure. [Figure 25] Figure 25 shows a task plotted on a homogeneous manifold approximation and projection (UMAP) chart according to an embodiment of the present disclosure. [Figure 26]Figure 26 shows individual data plotted on a homogeneous manifold approximation and projection (UMAP) chart according to one embodiment of the present disclosure. [Figure 27] Figure 27 shows time data plotted on a homogeneous manifold approximation and projection (UMAP) chart according to one embodiment of the present disclosure. [Figure 28] Figure 28 shows swallowing values ​​across multiple channels according to one embodiment of the present disclosure. [Figure 28-1] Figure 28-1 shows swallowing values ​​across multiple channels according to one embodiment of the present disclosure. [Figure 28-2] Figure 28-2 shows swallowing values ​​across multiple channels according to one embodiment of the present disclosure. [Figure 28-3] Figure 28-3 shows swallowing values ​​across multiple channels according to one embodiment of the present disclosure. [Figure 29] Figure 29 shows the random forest importance values ​​across multiple channels according to one embodiment of the present disclosure. [Figure 30] Figure 30 shows cluster statistics according to an embodiment of the present disclosure. [Figure 31] Figure 31 shows the Z-scores between tasks being collected in the morning, according to one embodiment of the present disclosure. [Figure 32] Figure 32 shows the Z-scores between tasks being collected over the course of a night, according to an embodiment of the present disclosure. [Figure 33] Figure 33 shows the Z-score for one-miss cross-validation (LOOCV) for a task according to an embodiment of the present disclosure. [Figure 34] Figure 34 shows the Z-score of the standard deviation (SD) during morning data collection according to one embodiment of the present disclosure. [Figure 35] Figure 35 shows a diagram illustrating the Z-scores between tasks being collected over the course of a night, according to one embodiment of the present disclosure. [Figure 36] Figure 36 shows the coefficient of variation (CV) of a mixed-effects model during evening data collection according to one embodiment of the present disclosure. [Figure 37]Figure 37 shows a bandwidth power measurement spread for smile collection according to an embodiment of the present disclosure. [Figure 37-1] Figure 37-1 shows a bandwidth power measurement spread for smile collection according to an embodiment of the present disclosure. [Figure 38] Figure 38 shows a clustered ICC heatmap for evening measurements according to an embodiment of the present disclosure. [Figure 38-1] Figure 38-1 shows a clustered ICC heatmap for evening measurements according to an embodiment of the present disclosure. [Figure 39] Figure 39 shows a clustered ICC heatmap of a mixed-effects model using evening measurements according to one embodiment of the present disclosure. [Figure 40] Figure 40 shows a t-distribution type stochastic neighbor embedding (t-SNE) chart for individuals, tasks, and time according to an embodiment of the present disclosure. [Figure 41] Figure 41 shows various charts of collection classes according to embodiments of the present disclosure. [Figure 42] Figure 42 shows a UMAP chart relating to individuals, tasks, and time, according to one embodiment of the present disclosure. [Figure 43] Figure 43 shows the verification results for different tasks across various channels according to one embodiment of the present disclosure. [Figure 44] Figure 44 shows the verification results for different tasks across various channels according to one embodiment of the present disclosure. [Figure 45] Figure 45 is a flowchart of the feature manipulation process according to an embodiment of the present disclosure. [Figure 46] Figure 46 shows a feature representation in the frequency and time domain according to one embodiment of the present disclosure. [Figure 47] Figure 47 shows the qualitative differences observed from representative signals according to an embodiment of the present disclosure. [Figure 47-1]Figure 47-1 shows the qualitative differences observed from representative signals according to an embodiment of the present disclosure. [Figure 48] Figure 48 shows the Spearman correlation of features according to an embodiment of the present disclosure. [Figure 49] Figure 49 shows the qualitative differences between 16 simulated PerfO activities according to an embodiment of the present disclosure. [Figure 50] Figure 50 is a diagram showing a heat map of features according to one embodiment of the present disclosure. [Figure 50-1] Figure 50-1 is a diagram showing a heat map of features according to one embodiment of the present disclosure. [Figure 50-2] Figure 50-2 is a diagram showing a heat map of features according to one embodiment of the present disclosure. [Figure 51] Figure 51 shows a CNN model constructed using raw biosignal data from a biometric sensor device according to an embodiment of the present disclosure. [Figure 52] Figure 52 shows an activity chart for activities with level classification F1 scores for the characteristics of a biometric sensor device according to an embodiment of the present disclosure. [Figure 53] Figure 53 shows a heat map of feature attribute analysis according to one embodiment of the present disclosure. [Figure 54] Figure 54 is a diagram of a data flow for training a machine learning model according to one or more embodiments. [Figure 55] Figure 55 is an illustrative diagram of a computing device according to one or more embodiments. [Modes for carrying out the invention]

[0012] The accompanying drawings incorporated herein and constituting part of this specification illustrate various examples and, together with the descriptions, serve to illustrate the principles of the disclosed examples and embodiments. Aspects of this disclosure may be implemented in relation to embodiments shown in the accompanying drawings. These drawings illustrate different aspects of this disclosure, and where appropriate, reference numbers indicating similar structures, components, materials, and / or elements in different drawings are similarly labeled. Various combinations of structures, components, and / or elements other than those specifically shown are also intended and included within the scope of this disclosure.

[0013] Furthermore, many embodiments are described and illustrated herein. This disclosure is not limited to any single aspect or embodiment thereof, nor is it limited to any combination and / or substitution of such aspects and / or embodiments. Each aspect and / or embodiment of this disclosure may be used alone or in combination with one or more other aspects and / or embodiments of this disclosure. For the sake of brevity, certain permutations and combinations are not described and / or illustrated separately herein. Notwithstanding that embodiments or implementations described herein as “exemplary” should not be construed as, for example, preferred or advantageous over other embodiments or implementations, but rather are intended to reflect or indicate that one or more embodiments are “examples” of one or more embodiments.

[0014] As used herein, the terms “encompassing,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus containing a list of elements may not contain only those elements, but may also contain other elements not explicitly listed, or other elements specific to such process, method, article, or apparatus. The term “exemplary” is used in the sense of “example” and not “ideal.” Furthermore, terms such as “first,” “second,” etc., as used herein are not intended to indicate order, quantity, or importance, but are used to distinguish one element or structure from another. In addition, the term “one” as used herein is not intended to indicate a limit on quantity, but rather to indicate the presence of one or more of the items being referenced.

[0015] In particular, specific aspects of the figures illustrate the general structure and / or construction methods of various embodiments for the sake of simplicity and clarity of explanation. Descriptions and details of well-known features and techniques may be omitted to avoid unnecessarily obscuring other features. Elements in the figures are not necessarily drawn to a constant scale, and the dimensions of some features may be exaggerated relative to others to improve the understanding of the exemplary embodiments. For example, a person skilled in the art will understand that the side view is not drawn to scale and should not be seen as representing proportional relationships between different components. The side view is provided to help show the various components of the illustrated assembly and to indicate their relative positions to one another.

[0016] The embodiments of this disclosure shown in the attached drawings will be referenced in detail below. Wherever possible, the same reference numerals are used throughout the drawings to refer to the same or similar parts. The term “distal” refers to the part of the device furthest from the user when the device is introduced into the object. In contrast, the term “proximal” refers to the part of the device closest to the user when the device is placed into the object. In the following description, relative terms such as “about,” “substantially,” and “approximately” are used to indicate a possible variation of ±10% in the stated figures.

[0017] Aspects of the subject matter of this disclosure generally concern receiving signals generated based on an individual's bodily components. These signals may be, or may be generated based on, electrical activity, physical activity, biometric data, motor data, or any attribute of the individual's body, actions associated with the individual's body, or responses of the individual's body. The signals may be generated by a signal-capturing device capable of capturing signals using one or more sensors. For example, aspects of the subject matter of this disclosure concern methods for profiling a subject's biometric cues using a wearable biometric device. Computer-mediated profiling for the early detection of potential diseases or disorders in patients, so that early diagnosis and treatment can be provided, is also described. Exemplary aspects of wearable biometric devices are also disclosed.

[0018] Implementations of the subject matter of this disclosure include wearable systems for identifying biometric cues of a subject. The systems and methods disclosed herein may be used to address unacceptable detection and treatment gaps in patients presenting with neurological disorders or impairments. In particular, non-invasive wearable biometric devices (e.g., ear-hook devices) are disclosed for detecting patient movements, especially facial movements such as speech, chewing, swallowing, neck movements, and / or eye movements.

[0019] Implementations of the subject matter of this disclosure provide methods for uploading large amounts of data for analysis. For example, the analysis may be performed using advanced statistical analysis and machine-based learning (or artificial intelligence), thereby ensuring reliable results that can be retested and understood. The systems and methods disclosed herein enable patient comfort and compliance, large arrays of input / output channels for large-scale data collection, highly reliable machine-assisted statistical analysis, early detection of disorders or diseases, early intervention therefor, and improved clinical outcomes.

[0020] Forms of the subject matter of this disclosure may be used to detect diseases and / or disorders based on the collection of objective statistical data. For example, implementations of the subject matter of this disclosure may be suitable for neurological disorders, such as myasthenia gravis (MG), in which latent cues may become undetectable.

[0021] Improving pipelines for the development and analysis of wearable sensor data, as well as frameworks for how to use this data in clinical settings, is crucial for improving accurate patient diagnosis and monitoring therapeutic responses at all stages of clinical drug development. Challenges exist in both the development of wearable devices and how wearable data is processed and analyzed in clinical settings. The methods disclosed herein address these challenges and include an exemplary proof of concept, as shown in Figure 1B, for evaluating facial and / or eye muscle movements in a healthy control group study using a signal acquisition device 10 (e.g., a sleep aid / biometric sensor wearable measuring electromyography (EMG), electroencephalography (EEG), and electrooculography (EOG)). The usefulness of an unbiased feature manipulation approach for classifying activities intended to represent true performance outcome assessments (PerfOs) is disclosed. An approach for analyzing and ranking the usefulness of features generated from biometric sensor devices, as well as how this data may be used to classify activities performed in this study setting, is disclosed. The limitations of time-series analysis on biosignal data collected over short periods, compared to feature-based analytical approaches, are also disclosed.

[0022] Data generated by biometric sensor devices can be used to classify an individual's physical information (e.g., certain types of cranial muscle and eye movements). The data disclosed herein suggests that biometric wearable devices can be used to objectively monitor certain physical information (e.g., cranial movements such as blink rate). For example, such physical information may include movements that increase in certain neuromuscular diseases such as ocular myasthenia gravis and decrease in Parkinsonian syndromes. Furthermore, given the demonstrated usefulness of these waveforms for measuring diseases in a clinical setting, there is an advantage to simultaneously measuring multiple types of waveforms from a single device.

[0023] As disclosed herein, feature importance analysis may show that when analyzing which features are most important when classifying activity using an RF model, EOG is associated with a significant contribution to gaze or eye movement activity (up, left, and right). EOG, EMG, and other signals play important roles in such symptoms. The presence of signal artifacts can complicate waveform contribution analysis. For example, when performing chewing activity, if residual EMG activity overlapping with typical EEG frequencies persists in the EEG signal after signal separation, it can lead to an overestimation of the EEG waveform contribution. As described herein, there are numerous neuromuscular and / or neurodegenerative conditions that can benefit from the improved use of wearable sensor technology.

[0024] The methods disclosed herein include several feature manipulation and evaluation considerations. The classification accuracy (F1 score) of models constructed from both processed sensor data is compared to that of models constructed from raw biosignal data. Regardless of data augmentation, regularization, and other methods used to combat overfitting, it has been observed that the training datasets used in the examples provided herein are too small to train a generalizable convolutional neural network (CNN) model. However, the level or amount of data collected in the embodiments disclosed herein may represent data collected in a clinical laboratory environment. Therefore, it is important to understand the most appropriate analytical methods (e.g., clinically relevant features) for a particular clinical outcome, as further disclosed herein. The analytical methods (e.g., clinically relevant features) used for a given clinical outcome may differ from those used for another clinical outcome.

[0025] The term “algorithm” typically refers to a defined sequence of computer-implementable instructions for solving a class of problems or performing computations. Figures 1–3, 4A, and 4B provide components for implementing algorithms and / or implementing examples of algorithms.

[0026] The term "AUC," as understood in this technical field, refers to the Area Under the Curve, which is related to statistical analysis. The term "BCI" refers to a brain-computer interface system that measures the activity of the central nervous system (CNS) and converts that activity into artificial and / or digital outputs for analysis.

[0027] The term "BMI" refers to a Body Mass Index (BMI) value derived from a person's weight and height. BMI is recognized as an indicator for broadly classifying people into underweight, normal weight, and overweight categories. BMI is frequently measured as a factor for entering clinical trials.

[0028] The term "CNN" refers to a convolutional neural network algorithm that can take input and assign importance (learnable weights and biases) to various aspects / objects within that input, as well as distinguish between different aspects / objects. CNNs can perform both generative and descriptive tasks using deep learning and are often used with machine vision, including image and video recognition, as well as recommendation systems and natural language processing (NLP). A CNN's layers may include an input layer, an output layer, and hidden layers containing multiple convolutional layers, pooling layers, fully connected layers, and normalization layers. By removing constraints and improving processing efficiency, much more efficient and easier-to-train systems can be obtained.

[0029] The term "CV" refers to covariance in statistics. A positive number is output when the measured variables have a positive relationship, and a negative number is output when the measured variables have a negative relationship. A high covariance may indicate a strong relationship between the variables. A low value may indicate a weak relationship between the variables.

[0030] The term "EEG" refers to electroencephalography, a biometric assessment of brain activity. EEG can detect abnormalities in the brain's waves, or abnormalities in the brain's electrical activity. EEG can be collected by attaching electrodes, which have small metal discs with thin wires, to the scalp. The electrodes can detect the electrical charges resulting from the activity of brain cells. As disclosed herein, EEG data can be detected using non-invasive devices (e.g., behind-the-ear devices).

[0031] The term "EMG" refers to electromyography, a biometric assessment of facial muscle weakness. EMG data can be collected by recording or receiving the electrical activity of muscle tissue, or its representation as a visual or audible signal, using electrodes attached to the skin or inserted into the muscle. As disclosed herein, EMG data can be detected using non-invasive devices (e.g., behind-the-ear devices).

[0032] The term "EOG" refers to electrooculography (EOG) recording, which is an assessment of eye movement activity. EOG data can be collected by measuring potentials between points close to the eye, particularly used to investigate eye movements in physiological studies. As disclosed herein, EOG data can be detected using non-invasive devices (e.g., behind-the-ear devices).

[0033] The term "F1 score" refers to a measure of a model's accuracy on a dataset as a binary classification, where a score of 0 is poor and a score of 1 is best. The F1 score can be calculated from the precision and recall of a test. Here, precision is the number of true positive results divided by the total number of positive results, including those that were not correctly identified, and recall is the number of true positive results divided by the total number of samples that should have been identified as positive. Precision can be the positive predictor. Recall can be the sensitivity in a diagnostic binary classification. The F1 score can be the harmonic mean of precision and recall.

[0034] The term "false positive" refers to a result where a model incorrectly predicts a positive class. The term "false negative" refers to a result where a model incorrectly predicts a negative class. The term "ICC" refers to the intraclass correlation coefficient, which can be used when quantitative measurements are taken on units organized into groups. It can be used to assess the consistency or reproducibility of quantitative measurements taken by different observers measuring the same quantity.

[0035] The term "ISO" refers to an isometric measure that relates to or indicates muscle activity in which tension is generated without muscle contraction. The term "LOOCV" refers to Leave One Out Cross Validation, a procedure used to estimate the performance of machine learning algorithms. In LOOCV, the number of folds can be equal to the number of instances in the dataset. Therefore, a learning algorithm can be applied once per instance, using all other instances as the training set and a selected instance as the single-item check set.

[0036] The term "MG" refers to myasthenia gravis, a neurodegenerative disease that can be assessed by the application of the subject matter of this disclosure. The term "PSG" refers to polysomnography, a type of sleep study that uses multi-parameter testing as a diagnostic tool in sleep medicine. During a PSG analysis, brain waves, blood oxygen levels, heart rate, respiration, and eye and leg movements may be recorded and / or analyzed.

[0037] The term "random forest" refers to combining many decision trees into a single model. Individually, the predictions made by decision trees (or humans) may not be accurate, but the combination of such predictions can increase their overall accuracy. A random forest, or random decision forest, is an ensemble learning method for classification, regression, and other tasks that works by building a large number of decision trees during training. In the case of a classification task, the output of a random forest may be the class selected by most of the trees.

[0038] The term "RMS" stands for Root Mean Square, and it refers to a statistical measure of the magnitude of a variable. RMS can be calculated for a set of discrete values ​​or for a continuously changing function.

[0039] The term "SD" refers to standard deviation. A low standard deviation indicates that the values ​​tend to be close to the mean (also called the expected value) of the set, while a high standard deviation indicates that the values ​​are dispersed over a wider range.

[0040] The term "spectrogram" refers to a visual representation of the spectrum of a signal's frequencies as they change over time. The term "true positive" refers to a result in which the model correctly predicts the positive class.

[0041] The term "true negative" refers to the result in which the model correctly predicts the negative class. The term "Z-score" refers to the standard deviation value that indicates how far a given data point is from the mean. A Z-score equal to 0 means the data point is at the mean. A positive Z-score indicates that the raw score is above the mean. A negative Z-score indicates that the raw score is below the mean.

[0042] The following are incorporated by reference and relate to EMG and / or EOG: A review of classification techniques of EMG signals during isotonic and isometric contractions; Sensors. 2016 Aug 17;16(8): 1304, Real-Time Surface EMG Pattern Recognition for Hand Gestures Based on an Artificial Neural Network; Sensors. 2019 Jul; 19(14): 3170, and Techniques of EMG signal analysis: detection, processing, classification, and applications; Biol. Proceedings. Online. 2006; 8: 11-35.

[0043] The time and frequency domain analysis of EEG signals for seizure detection (a review) (Harpale, VK and Vinayak K. Bairagi. "Time and frequency domain analysis of EEG signals for seizure detection: A review" 2016 International Conference on Microelectronics, Computing and Communications (MicroCom) (2016): 23-25 ​​Jan. 2016: 1-6) is incorporated by reference and is relevant to the algorithms described herein.

[0044] The challenge in detecting physiological changes using wearable sensor data (Challenges in Detecting Physiological Changes Using Wearable Sensor Data (SciPy 2019)) is incorporated by reference and relates to the interpretation of time-series data.

[0045] International Publication No. 2016 / 110804A1 describes an exemplary mobile wearable monitoring system, including a headgear device, used in connection with the principles of this disclosure, and is incorporated herein by reference.

[0046] Depending on the implementation of the subject matter of this disclosure, as shown in Figure 1A, the signal acquisition device 10 may be used to acquire signals associated with an individual's body. The signals may be based on the body's electrical activity, physical movement, biometric information, temperature information, actions, reactions, or any attribute that can be acquired as a signal (e.g., as an electrical signal). The signal acquisition device 10 may include one or more sensors, electrodes, cameras, or other components used to acquire signals based on an individual's body. The signals acquired by the signal acquisition device 10 may include one or more individual signals 30 (e.g., individual signal A32, individual signal B34, and individual signal C36).

[0047] Alternatively, the signals captured by the signal capture device 10 may be processed via the signal manipulation module 20. The signal manipulation module 20 can analyze individual signals 30 from the raw data received from the signal capture device 10 by applying signal filtering techniques. For example, the signal manipulation module 20 can identify individual signals 30 from the signals provided by the signal capture device 10 by applying one or more of the following: blind signal separation, blind source separation, discrete transform, Fourier transform, integral transform, two-sided Laplace transform, Mellin transform, Hartley transform, Short-time Fourier transform (or short-term Fourier transform) (STFT), rectangular mask short-time Fourier transform, Chirplet transform, Fractional Fourier transform (FRFT), Hankel transform, Fourier-Bros-Iagolnitzer transform, linear canonical transform, etc.

[0048] Individual signals 30 can be used to generate multiple extracted features 40 (e.g., extracted feature A42, extracted feature B44, ..., extracted feature N46). The extracted features 40 can be generated individually or in combination with each other based on the properties of the extracted features 40. For example, the extracted features 40 may be based on one or more of the following: signal frequency band power (e.g., for each of the individual signals 30 across multiple frequencies), spectral entropy, peak frequency collapse, peak frequency, mean amplitude, absolute amplitude percentage, standard deviation, absolute amplitude standard deviation, root mean square, initial deflection maximum amplitude, initial deflection polarity, trend-deselected fluctuating Hurst parameter, petrosian fractal dimension, approximate entropy, zero crossing rate, amplitude kurtosis, amplitude skew, perceptible onset time, amplitude variance, and / or any applicable signal-based attributes.

[0049] An implementation of the present disclosure may be used to identify clinically relevant features 50 (e.g., clinically relevant features 52 and / or clinically relevant features 54) for a given clinical output based on extracted features 40. For example, a first set of extracted features may be optimal for identifying a first clinical output (e.g., diagnosis or treatment of a disease or disorder), while a second set of extracted features may not be optimal for identifying the first clinical output but may be optimal for identifying the second clinical output. Thus, the method of the present disclosure may be used to identify clinically relevant features 50 for a given clinical output based on one or more of the extracted features 40, individual signals 30, signal manipulation module 20, signal capture device 10, clinical output, and individuals. For example, the signal capture device 10 may have sensitivity quality to generate a first feature well, but may not be sensitive enough to generate a second feature. Therefore, when using the signal capture device 10, the first feature may be identified as a clinically relevant feature, while the second feature may not be identified as a clinically relevant feature.

[0050] Similarly, a given clinical outcome (e.g., a diagnosis of Parkinson's disease) can be identified using a first feature that has a lower standard deviation in individuals with Parkinson's disease, compared to a second feature that has a higher standard deviation in individuals with Parkinson's disease. Therefore, the first feature can be identified as a clinically relevant feature because, compared to the second feature, it may be more consistent in predicting the presence of Parkinson's disease, for example.

[0051] Clinically relevant features 50 can be identified based on signal data collected and analyzed for a test user or a group of test users. A group of test users may be any group of test users, or a group of test users having attributes (e.g., demographic factors) that overlap with one or more individuals. For example, one or more signal capture devices 10 may be used to generate individual signals 30 for one or more test users. Extracted features 40 may be generated from these individual signals 30. Clinically relevant features 50 can be identified for each individual and for a given clinical output (e.g., detection of a disorder). As disclosed herein, these clinically relevant features 50 may meet or exceed one or more confidence thresholds so that the clinically relevant features 50 can be relied upon to generate a clinical output with a degree of confidence. Clinically relevant features 50 identified based on data from one of the test users or a group of test users may be permitted for use in clinical trials based on their clinical output confidence level. When one or more individuals participate in such a clinical trial, data corresponding to clinically relevant characteristics 50 for those one or more individuals can be compared to reference data (e.g., data from one or a group of test users).

[0052] Additionally or alternatively, data obtained for a given individual and / or combined data for multiple individuals may be used as endpoints in a given trial. For example, the identification of clinically relevant elements 50 may be an endpoint in a clinical trial. The endpoint may provide an indicator of the quality and / or capability of the signal-capturing device 10 being tested in the clinical trial. Alternatively, the endpoint may provide an indicator of clinically relevant elements 50 that reliably provide clinical output considering the patient population, the type of disease or disability, and / or the signal data.

[0053] Additionally or alternatively, data from an individual (e.g., signal data, discrete signals 30, extracted features 40, and / or clinically relevant features 50) may be compared with corresponding data from one or more other individuals. For example, such data may be collected from each of several users in a clinical trial. In this example, data from one or more individuals receiving treatment (e.g., drugs, therapies, etc.) may be compared with data from one or more individuals receiving alternative treatment (e.g., different doses, durations, or types of drugs or therapies), data from one or more individuals not receiving treatment (e.g., a placebo group), and / or a reference dataset (e.g., control data). According to one implementation, data associated with a given individual at a first time may be compared with data from that individual at a second time.

[0054] Clinically relevant elements can be identified by using data from different individuals (e.g., those receiving different treatments, placebos, or control groups) (e.g., signal data, discrete signals 30, extracted features 40, and / or clinically relevant features 50), or by using time-series comparisons of data from the same individual. Clinically relevant elements may include, for example, the effect of a given treatment, the effect of the dosage or duration of a treatment, differences in the presentation of a given disease or disorder (e.g., for an optimal treatment plan) in different individuals, or the identification of clusters or groupings.

[0055] According to exemplary implementations disclosed herein, electrical information of an individual's brain, nervous system, and / or muscles may be acquired. Additionally or alternatively, sensory information (e.g., visual) (e.g., video, images, infrared images, thermal images, vibrations, etc.) of an individual's body (e.g., the individual's face or a part thereof) may be acquired. According to several implementations, a headgear or wearable device (e.g., signal acquisition device 10) may be used to collect signals from an individual's body. The headgear or wearable device may include one or more sensors for acquiring electrical and / or sensory information. Although the term headgear and / or wearable device is used herein, any device configured to acquire electrical or sensory information on a body part (e.g., an individual's face or a part thereof) may be used in accordance with the methods disclosed herein. The headgear and / or wearable device may include any device configured to be stationary and / or installed on or around an individual's head or body part. The headgear and / or wearable device may or may not be fixed to a part of an individual's body.

[0056] In exemplary implementations, the headgear and / or wearable device may include sensors for capturing electrical signals. Such electrical signals may include electroencephalogram (EEG) data, electrooculogram (EOG) data, and / or electromyogram (EMG) data. The exemplary headgear and / or wearable device may also include sensory information sensors (e.g., image sensors, video sensors, infrared sensors, thermal sensors, vibration sensors, etc.) for capturing individual input data such as facial data (e.g., facial recognition data), eye-tracking data, motion data, and environmental data (e.g., thermal data). Furthermore, a controller may receive signal data (e.g., EEG, EOG, EMG, and individual input data).

[0057] The controller may be configured to classify signal data (e.g., EEG, EOG, EMG, and individual input data), and the signal data may be used to identify clinical outcomes. For example, the signal data may be used to identify whether an individual has one or more diseases. Alternatively or additionally, the signal data may be used to determine the characteristics of an individual's disease and provide an individual treatment plan. According to the implementations disclosed herein, the classification may be performed using machine learning techniques. In some implementations, the system and method may further combine the signal data (e.g., EEG, EOG, EMG data, and individual input data) into the classification of potential conditions.

[0058] The technologies disclosed herein may be used to determine the scope of a condition and / or a treatment plan corresponding to that scope. Depending on the implementation of the subject matter of this disclosure, wearable biometric devices (e.g., headgear and / or wearable devices) may be used to detect and / or treat a disease or disorder in an individual. The subject matter of this disclosure may be used for the early identification of a disease or disorder that may be pre-symptomatic, asymptomatic, and / or undiagnosed. The technologies disclosed herein have a wide range of applications for quantifying a certain range of neurological and muscular diseases and disorders.

[0059] As introduced in this disclosure, technologies in the fields of patient intake, statistical analysis, the use of wearable electronic devices, artificial intelligence (AI), algorithms, machine-based learning, statistical analysis, and wearable devices and / or electronic modes for measuring subjects may be implemented to ensure unbiased, objective metrics relevant to individuals and / or diseases or disorders. By using objective, data-driven analysis, reliable metrics for proactively identifying a person's disease or disorder can be achieved by filtering out inaccurate patient self-reports and potential statistical noise. While such information may be supplemented by subjective questions to determine intake and / or baselines, objective analysis may be used to more accurately identify diseases or disorders and / or provide treatment plans.

[0060] According to the implementations of the subject matter of this disclosure, highly reliable measurement of an individual's biometric cues may be used to detect and / or treat a disease or disorder. One or more sensors (e.g., sensors placed on or around a headgear) may be used to collect an individual's biometric measurements at a given point in time and / or over a range of time. Figure 1B shows a block diagram illustrating an example of an environment 100 for implementing a system and method according to an aspect of this disclosure. In some implementations, the environment 100 may include an individual 101, a headgear 103, and a controller 105. The headgear 103 may correspond to the signal acquisition device 10 in Figure 1A.

[0061] Individual 101 can be any person. In some implementations, Individual 101 may be a baseline individual for the collection of control data. In some implementations, Individual 101 may be a medical patient. For example, Individual 101 may be a patient who has a neuromuscular disease such as myasthenia gravis. While “headgear” and “wearable device” are generally referred to herein, devices for collecting electrical data and / or personal input data may be positioned on, below, around, partially around, or near the head or other body part of the individual at any applicable position. For example, headgear 103 may refer to eyeglasses that can be placed on the individual’s ears. As another example, headgear 103 may refer to earpieces inserted into or around the individual’s ears.

[0062] The headgear 103 may be a device that includes one or more sensors that capture information of an individual 101 representing voluntary muscle movements. The sensors may collect electrical information and / or personal input information (e.g., facial data (e.g., facial recognition data), eye-tracking data, motion data, environmental data (e.g., thermal data), etc.). In some implementations, the headgear 103 may include an electrical sensor 111, a sensory information sensor 113, and / or a device controller 115. The electrical sensor 111 may be configured to capture EEG data, EOG data, and EMG data. The sensory information sensor 113 may include a facial recognition sensor, an eye-tracking sensor, an image sensor, a video sensor, an infrared sensor, a thermal sensor, and a vibration sensor.

[0063] The device controller 115 may be a computing device connected to the controller 105 via one or more wired or wireless communication channels 121. The communication channels 121 may use various serial, parallel, and / or transmission (e.g., video transmission) protocols. The device controller 115 may include hardware, software, firmware, or a combination thereof for performing the operations described herein. The operations may include receiving personal input data such as EEG, EOG, EMG, face data (e.g., face recognition data), eye-tracking data, motion data, and environmental data (e.g., thermal data) from electrical sensors 111 and / or sensory information sensors 113, and transmitting the data to the controller 105 using the communication channels 121 with one or more transmission protocols.

[0064] The controller 105 may include hardware, software, or a combination thereof for performing the operations described herein. The operations performed by the controller 105 may include receiving, filtering, and normalizing data transmitted by the device controller 115 of the headgear 103. The operations may also include classifying the EEG, EOG, EMG, and / or personal input data individually by determining one or more descriptive categories and the severity of each. The categories may be the type or genus of symptoms or disorders. In some implementations, the classification may be performed using machine learning techniques. For example, an elaborate sequence of highly statistical data may be interpreted using a random forest schema, as further described herein.

[0065] Implementations of the subject matter of this disclosure include classifying combinations of EEG, EOG, EMG, and personal input information by determining one or more descriptive categories or disorders. In some implementations, the classification may be performed using machine learning techniques to classify symptoms or disorders determined using the individual classifications of the data. In some implementations, the operation further includes determining treatment plans corresponding to one or more disorders and their respective severity levels.

[0066] For example, Figure 1B shows a system diagram illustrating an example of a headgear 103 according to an aspect of the present disclosure. The headgear 103 may be identical or similar to the headgear and / or wearable devices described above. In some implementations, the headgear 103 may include a device controller 115, an electrical sensor 111, and a sensory information sensor 113. These may be identical or similar to those described above.

[0067] Figure 2 is a system block diagram showing an embodiment of an environment for implementing the systems and processes disclosed herein. The electrical sensor 111 may include any applicable sensors, such as an EEG sensor 205, an EOG sensor 207, and an EMG sensor 209. These sensors generate EEG data, EOG data, and EMG data, which may be identical or similar to those described above. Although the EEG sensor 205, EOG sensor 207, and EMG sensor 209 are shown as separate sensors, one or more of the EEG, EOG, and EMG sensors may also be combined.

[0068] The sensory information sensor 113 may include an image sensor 211 and an eye-tracking sensor 213 that generate face recognition data and eye-tracking data. This face recognition data and eye-tracking data may be identical or similar to those described above. Although the image sensor 211 and the eye-tracking sensor 213 are shown as separate sensors, they may be combined.

[0069] The device controller 201 may be the device controller 115 in Figure 1B or a part thereof, or it may be one or more devices that process data generated by the EEG sensor 205, EOG sensor 207, EMG sensor 209, image sensor 211, and eye-tracking sensor 213. In some implementations, the device controller 201 may include a processor 225, a memory device 227, a storage device 229, a communication interface 231, an input / output (I / O) processor 233, and a data bus 235. The device controller 201 may include a signal manipulation module 20. Alternatively, the signal manipulation module 20 may be independent of the device controller 201 and / or the signal acquisition device 10. For example, the signal manipulation module 20 may be located away from the signal acquisition device 10 or may be part of a separate analysis component.

[0070] In some implementations, the processor 225 may include one or more microprocessors, microchips, or application-specific integrated circuits. The memory device 227 may include one or more types of random access memory (RAM), read-only memory (ROM), and cache memory used during the execution of program instructions. The processor 225 can communicate with the memory device 227, the storage device 229, the communication interface 231, the image processor, and / or the spatial sensor using the data bus 235. The storage device 229 may include a computer-readable non-volatile hardware storage device that stores information and program instructions.

[0071] For example, the storage device 229 may be one or more of a flash drive and / or a hard disk drive. The transmitter / receiver may be one or more devices that can be used to communicate signals and to encode / decode data into radio signals such as ranging signals.

[0072] The processor 225 can execute program instructions (e.g., operating system and / or application programs) that can be stored in the memory device 227 and / or the storage device 229. The processor can also execute program instructions for the sensor module 251. The sensor module 251 may include program instructions for processing data generated by the EEG sensor 205, EOG sensor 207, EMG sensor 209, image sensor 211, and eye-tracking sensor 213. Processing may include, for example, filtering, amplifying, and normalizing the data to remove noise and other artifacts. Note that the device controller 201 is merely representative of various possible equivalent computing devices that can perform the processing and functions described herein. In this regard, in some implementations, the functions provided by the device controller 201 may be any combination of general-purpose and / or purpose-specific hardware and / or program instructions. In each implementation, the program instructions and hardware can be created using standard programming and engineering techniques.

[0073] Figure 3 shows a functional block diagram illustrating a controller 105 according to an aspect of this disclosure. The controller 105 may be identical or similar to those described herein. The controller 105 may include a computing device 306 comprising a processor 305, a memory device 307, a storage device 309, an I / O processor 325, and a data bus 331. The controller 105 may also include input connections (e.g., image input connections) and / or output connections (e.g., image output connections) for receiving and / or transmitting image signals from the image processor. Furthermore, the controller 105 may include input / output connections for receiving / transmitting data signals from the I / O processor 325.

[0074] In multiple implementation configurations, the controller 105 may include one or more microprocessors, microchips, or application-specific integrated circuits. The memory device 307 may include one or more types of random access memory (RAM), read-only memory (ROM), and cache memory used during the execution of program instructions. The controller 105 may also include one or more data buses 331 that communicate with the memory device 307, the storage device 309, and the I / O processor 325.

[0075] The storage device 309 may include a computer-readable non-volatile hardware storage device that stores information and program instructions. For example, the storage device 309 may be one or more flash drives and / or hard disk drives. The storage device 309 may include reference data 310 for access via the communication data bus 331.

[0076] The I / O processor 325 may be connected to the processor 305. The I / O processor 325 may include any device that enables an individual to interact with the processor 305 (e.g., a user interface), and / or any device that enables the processor 305 to communicate with one or more other computing devices using any kind of communication link. The I / O processor 325 may, for example, generate and receive digital and analog inputs / outputs (e.g., electronic signals) according to various data transmission protocols.

[0077] The processor 305 executes program instructions (e.g., operating system and / or application programs) that can be stored in the memory device 307 and / or the storage device 309. The processor 305 can also execute program instructions for module 351.

[0078] The controller 105 may include a signal manipulation module 20. Alternatively, the signal manipulation module 20 may be independent of the controller 105. For example, the signal manipulation module 20 may be located away from the signal acquisition device 10 or may be part of a separate analysis component. The controller 105 may include a fault classification module 355 and / or a sensor classification module 359. The fault classification module 355 and / or the sensor classification module 359 may apply the statistical analysis techniques disclosed herein to identify and / or apply clinically relevant features for the classification of faults and / or sensor signals.

[0079] Controller 105 may include a communication interface 311 to facilitate inter-controller communication or intra-controller communication (e.g., via a data bus 331). Controller 105 may also include one or more I / O devices 333 for communicating with an I / O processor 325. According to one implementation, the I / O devices may include a device controller 201 and / or a device controller 115.

[0080] Note that the controller 105 is merely representative of various possible equivalent computing devices capable of performing the processing and functions described herein. In this regard, in some implementations, the functions provided by the controller 105 may be any combination of general-purpose and / or purpose-specific hardware and / or program instructions. In each implementation, the program instructions and hardware can be created using standard programming and engineering techniques.

[0081] Figures 4A and 4B include a flowchart 400 illustrating an example of an implementation of the subject matter disclosed herein. As described herein, an implementation of the subject matter of this disclosure may be used to identify clinically relevant features for a given clinical output based on features extracted from signal data. Figures 4A and 4B illustrate an example relating to the use of a wearable device to generate signal data based on sensors on the wearable device. Figures 4A and 4B illustrate an example of the function and operation of possible implementations of the system, method, and computer program product by various implementations consistent with this disclosure.

[0082] Each block in the flowchart of Figure 4A or Figure 4B may represent a module, segment, or portion of a program instruction, which contains one or more computer-executable instructions to perform the illustrated function and operation. In some alternative implementations, the functions and / or operations shown in a particular block of the flowchart may occur in a different order than that shown in Figure 4A or Figure 4B.

[0083] For example, two consecutively presented blocks may be executed substantially simultaneously, or multiple blocks may be executed in reverse order depending on the function they contain. Furthermore, each block in a flowchart, and any combination of blocks within it, may also be implemented by a dedicated hardware-based system that performs a specified function or operation, or by a combination of dedicated hardware and computer instructions.

[0084] In flowchart 400, at 401, the subject may perform facial movements. Facial movements may be performed on request or based on the subject's natural state, etc. While facial movements are specifically disclosed herein, it should be understood that any bodily components, actions, or characteristics may be observed as part of the subject matter of this disclosure. In 405, EEG information may be obtained from one or more EEG sensors of a wearable device. EEG information may be collected based on contact or non-contact reception of signals via electrodes. In 409, EOG information may be obtained from one or more EOG sensors of a wearable device. EOG information may be collected based on contact or non-contact reception of signals via electrodes.

[0085] In 413, facial information may be acquired from a sensory information sensor such as an image sensor. Facial information may include motion, attribute information (e.g., temperature, length, elasticity, angle, etc.). Facial information may be acquired using a sensory information sensor based on a trigger (e.g., a request for facial information, a facial action, or a change), or it may be acquired on a continuous basis. In 418, eye position information may be acquired from an eye-tracking sensor. Eye position information may include motion, attribute information (e.g., degree of motion, direction of motion, extension, angle, etc.). Eye position information may be captured using a sensory information sensor based on a trigger (e.g., a request for eye position information, an action, or a change), or it may be captured continuously.

[0086] In 421, one or more of the EEG information, EOG information, facial information, and / or eye position information may be filtered and / or normalized. These information may be filtered and / or normalized based on any applicable techniques disclosed herein. These information may be filtered and / or normalized to remove noise, extract features, etc. In 425, the EEG information may be classified. In 429, the EOG information may be classified. In 433, the facial information may be classified. In 437, the eye position information may be classified.

[0087] In 441 (shown in Figure 4B), EOG information, facial information, and eye-tracking information may be combined. In 445, EEG information and facial information may be combined. It can be understood that the information may be combined based on clinical outcomes. For example, 441 and 445 in Figure 4B may differ for different clinical outcomes, as different information may be combined than the information presented in Figures 4A and 4B. It can be understood that any signal information relevant to an individual may be statistically evaluated based on the flows shown in Figures 1A, 4A, and 4B, for example.

[0088] In 449, combined EOG information, facial information, eye-tracking information, and / or other combined information can be compared with reference information. As shown in Figure 1A, this comparison may be based on extracted features from EOG, EEG, facial information, eye position information, or any other applicable information. This comparison may be based on extracted features determined to be clinically relevant, as shown in Figure 1A.

[0089] In 453, clinically relevant features based on sensory information may be used to determine the subject's state. For example, the state may be determined based on combined EOG information, facial information, and eye-tracking information, combined EEG information and facial information, and reference information. In 457, a determination may be made as to whether a state has been determined. If a state has not been determined, the steps described herein, beginning in 401, may be repeated (for example, as indicated by "B" in Figures 4A and 4B). If a state has been determined in 457, in 461, for example, a determination of the range of the state may be made based on second reference information. In 465, a treatment plan may be determined based on third reference information and / or second reference information (for example, further based on the range of the state). Figure 4C shows a flowchart 470 for the identification and application of clinically relevant features. In 472, multiple extracted features may be received. As described herein, multiple extracted features may be based on signals collected from or about an individual's body. This signal may be collected when an action is being taken or when a specific body part is being observed (e.g., impression of anger, chewing, eye movements, eye iso, medial iso, jaw, left gaze-left, left gaze-right, right gaze-left, right gaze-right, lateral iso, impression of sadness, smile iso, impression of surprise, swallowing, conversation, upward gaze, wrinkle iso, etc., as further described in Table 7). Thus, a given extracted feature may be based on a given type of signal (e.g., ECG, EEG, etc.), the action while the signal is being collected, the time while the signal is being collected, the individual, etc. Multiple available extracted features may be received in 472.

[0090] In 473, statistical filtering techniques for application to multiple extracted features may be identified. The statistical filtering technique may be a single technique or multiple techniques applied simultaneously. The statistical filtering technique may be specified to output clinically relevant features that can be used to best determine clinical outcomes. As described herein, clinical outcomes may be the identification of a disease or disorder (e.g., 453 in Figure 4B) and / or a treatment plan for the identified disease or disorder (e.g., 465 in Figure 4B). Clinical outcomes may be based on objective data (e.g., based on sensor-acquired signals).

[0091] In 474, one or more statistical filtering techniques identified in 473 may be applied to multiple extracted features received in 472. These statistical filtering techniques may include, but are not limited to, Spearman correlation 474A, ICC 474B, random forest algorithm 474C, CV 474D, AUC 474E, clustering 474F, Z-score 474G, or combinations thereof. These statistical filtering techniques are further described herein.

[0092] In 476, clinically relevant features may be identified based on the statistical filtering techniques applied in 474. When identifying clinically relevant features from multiple features received in 472, a clinical trial or other program may be run using only clinically relevant features based on a given clinical outcome, for example. The clinically relevant features to be identified may be features that can be used to examine the clinical outcome in a reliable manner, so that the identified clinically relevant features provide reliable data on a given clinical outcome. Reliability may satisfy a given reliability threshold for a given clinical outcome. A given reliability threshold may be a numerical value determined based on one or more of the following associated with the data: Spearman correlation, ICC, random forest results, CV, AUC, clustering, and / or Z-score. For example, a given reliability threshold may be based on the minimum or maximum value associated with one or more of the following for the raw data or a given set of features: Spearman correlation, ICC, random forest results, CV, AUC, clustering, and / or Z-score. The raw data may correspond to the features received in 427. Therefore, the reliability threshold may be a single value (e.g., binary score, ratio, percentage, etc.) or a set of values ​​(e.g., one for each of Spearman correlation, ICC, random forest results, CV, AUC, clustering, and / or Z-score) indicating that a given set of clinically relevant features provides relevant data about a given clinical outcome.478 In this context, clinically relevant features may be applied to determine clinically relevant outcomes.

[0093] Figure 4C presents a set of exemplary statistical filtering techniques (Spearman correlation, ICC, random forest results, CV, AUC, clustering, and / or Z-scores), but one or more additional techniques may be applied, and the set of statistical filtering techniques is not limited to those disclosed herein. Any statistical filtering technique demonstrating the reliability of one or more features may be used to identify clinically relevant features.

[0094] In a form of implementation of the subject matter of this disclosure, the identification of clinically relevant features in 476 of Figure 4C may be used to determine the limitations and / or quality of a sensor-based device (e.g., signal acquisition device 10 in Figure 1A) used to collect one or more signals (e.g., individual signals 30 or signals provided to signal manipulation module 20) used to determine the extracted features received in 472. By applying the techniques (e.g., procedures) disclosed herein, in 477, clinically relevant elements for a given clinical outcome (e.g., prediction of a particular disorder and / or treatment of a particular disorder) may indicate the limitations and / or quality of the sensor-based device. For example, clinically relevant features based on a wearable device and a given clinical outcome may be identified. Based on clinical features (e.g., one or more frequencies, one or more types of electrical signal channels, one or more spectral entropies, one or more peak frequencies, one or more amplitudes, one or more standard deviations, one or more root mean squares, one or more deflections, one or more variability, one or more fractal dimensions, one or more cross-ratios, one or more amplitude kurtosis, one or more skew, one or more onset times, one or more variances, etc.), it may be determined whether the corresponding sensor-based device can reliably output clinical outcomes (e.g., whether the threshold number extraction features meet the reliability threshold, whether the clinical relevance threshold exceeds the reliability threshold, etc.).

[0095] [Examples] Implementations of the subject matter of this disclosure are disclosed herein with reference to several examples. It should be understood that the implementations disclosed herein are not limited to the data, sequences, or specifications disclosed in the examples.

[0096] [Example 1] Figure 5 includes Chart 500, which illustrates the limitations of existing clinical outcome scales due to their partial influence by recall bias. For example, Chart 500 may be used by clinical professionals to receive subjective physician scores 502 based on condition 504. As an example, healthcare providers may examine ptosis (ease of looking upward) by observing the number of seconds that a gaze direction is maintained. Subjective physical scores may be determined by healthcare providers. However, such physical scores and corresponding assessments are susceptible to errors, recall bias, and missing information, which can lead to misdiagnosis.

[0097] Research was conducted to complete reliable biometric data acquisition activities. Objective data was collected, and the research was planned with the following objectives: 1) Understand the data quality and missing values ​​(missing values) of biometric devices. 2) Understand the test-retest reliability of biometric devices. 3) Understand the capabilities of biometric devices to quantify and differentiate various facial muscle activities.

[0098] As disclosed herein, the above objectives were determined, for example, based on the statistical analysis disclosed in Figure 4C. A total of 16 facial and eye movement tasks were selected for the study (swallowing, chewing, talking, facial expressions, eye closing, and looking in different directions). A control group of N=10 participated in the study. Data were received via a biometric device with 31 variables during the study. Initial data exploration was performed on the measurements.

[0099] According to the implementation of the subject matter of this disclosure, test-retest capability between variables (e.g., 31 variables tested) is recorded using ICC and CV analysis. Using ICC and / or CV, the predictability of all variables based on the task (activity) was evaluated using a multinomial logistic regression model. For example, a clinical focus on the analysis of the “swallowing” task (favorable activity) was addressed.

[0100] Additional measurements were received from the EOG sensor. The sensor was placed on a biometric device in close proximity to the individual. A total of 60 summary variables and ICC analysis included several additional variables with high test-retest capability. Model selection was performed (e.g., in 473 in Figure 4C). The results of model selection identified the best model (e.g., based on signals and / or biometric devices) for predicting activity and objects as a random forest model.

[0101] Figure 6 shows exemplary output readings collected using one or more sensors of a biometric device. As shown in Chart 602, alpha waves (0.3–35 Hz) were collected from various sensors while the eyes-closed task was performed. As shown in Chart 604, vertical EOG data (0.3–10 Hz) were collected from multiple sensors during the blinking task. As shown in Chart 606, horizontal EOG data (0.3–10 Hz) were collected during the left-gaze and right-gaze tasks. As shown in Figure 6, facial EMG (10–100 Hz) was collected using multiple sensors while the teeth-grinding task was performed. As shown in Chart 610, electrodermal activity (EDA) (galvanic skin response) (0.1–1.5 Hz) was collected while the excitation task was performed.

[0102] Figure 7 shows exemplary output readings collected using one or more sensors of a biometric device. The output readings shown in Figure 7 are collected using a device (e.g., signal acquisition device 10) that can reliably measure various aspects of facial biology using waveform signals for brain activity, eye movements, facial muscles, etc., validated against polysomnography (PSG). Amplitude (μV) collected during the smiling task is collected in 702, amplitude collected while puffing out the cheeks is collected in 704, amplitude collected while tightly closing the eyes is collected in 706, and amplitude collected while biting an apple is collected in 708. The amplitudes shown in charts 702, 704, 706, and 708 are shown over elapsed time (seconds).

[0103] The signals shown in Figure 6 and / or Figure 7 may be the individual signals 30 in Figure 1A and / or may be provided to the signal manipulation module 20 to generate the individual signals 30. Predictability based on the task and individual subjects was analyzed using a random forest model with F1 scores. Z scores were also obtained. The Z scores are presented herein as heatmaps. Heatmaps of variables and activities (tasks) show qualitative differences between subjects and activities, as shown in Figures 8 and 9.

[0104] Figure 8 (and Figures 8-1, 8-2) are heatmaps of four tasks and Z-score correlations. Figure 8 includes Chart 802, which provides exemplary feature descriptions and comments. Feature descriptions (e.g., fractal dimension, sample entropy, peak frequency collapse, spectral entropy, and band power) can be extracted from signals (e.g., extracted features 40) shown in Figures 6 and 7, for example. Comments related to features in Chart 802 provide explanations for each figure. Chart 804 shows the Z-scores 804B for each of several features 804C, calculated based on normalized raw data separated by task (e.g., smiling, breathing, closing eyes, chewing), individual (person), and time (e.g., morning / evening). The Z-scores are distributed using values ​​in the range of -3 to +3, with each value assigned a color according to Legend 804A.

[0105] Similarly, Figure 9 (and Figures 9-1 to 9-8) show heatmaps of variables and activities illustrating qualitative differences between subjects and between activities. Chart 904 shows Z-scores 904B for each of several features 904C calculated based on normalized raw data segmented by task (e.g., anger, chewing, eyes, eye iso, medial iso, jaw, left gaze-left (L gaze L), left gaze-right (L gaze R), lateral iso, sadness, smile iso, surprise, swallowing, conversation, upward gaze, wrinkle iso), individual (person), and time (e.g., morning / evening). The Z-scores are distributed using values ​​ranging from -3 to +3, with each value assigned a color according to Legend 904A.

[0106] Figure 10 (and Figures 10-1 to 10-5) shows the Spearman correlation chart 1000 for each feature 1000B, illustrating the correlation of feature 1000B to feature 1000B itself. The Spearman correlation chart 1000 can be used to identify relationships between features. Highly correlated features are likely to measure similar aspects of the facial biological and / or other signals (e.g., electrical activity) collected by the signal acquisition device 10 (e.g., similar aspects of individual signals 30). The Spearman correlation chart 1000 can be used to identify clusters 1000A, and as a result, similar features within the same cluster can be omitted or reduced to reduce redundancy in the analysis. For example, cluster-based reduction can be applied to identify clinically relevant features 50.

[0107] Figure 11 shows the ICC measurement 1100 for feature 1200A, which examines the test-retest reliability of parameters and infers clinical significance, and Figure 12 (and Figures 12-1, 12-2) shows the ICC measurement 1200. The ICC measurement 1100 is shown as a heatmap, which corresponds to morning measurement-based feature 1100A with fixed age, BMI, and sex. The ICC measurement can be measured as low clinical significance (e.g., 0) to high clinical significance (e.g., 1), as shown in legends 1100B and 1200B, respectively.

[0108] ICC measurements can be per subject (individual). This allows high clinical significance (e.g., 1) to indicate that similar data should be expected when a given measurement is repeated for a subject. Conversely, low clinical significance (e.g., 0) can indicate that dissimilar data should be expected when a given measurement is repeated for a subject. For a given clinical outcome, higher clinical significance (e.g., above the clinical significance threshold) may result in a given number being designated as clinically relevant (e.g., as clinically relevant feature 50). Higher clinical significance (e.g., in the range of 0.6 to 1) may be required for use in clinical trials. ICC measurements can be used to cluster features, as shown in 1100C and 1200C, respectively. Clusters of features with higher clinical significance may be designated as clinically relevant (e.g., as clinically relevant feature 50).

[0109] Figure 13 shows an exemplary predictive model 1300 (e.g., a random forest model) in an implementation of the subject matter of this disclosure. As shown in Figure 13, bootstrap sampling may be used to construct a model (e.g., a decision tree or a network). In the case of bootstrap sampling, r (percentage) examples may be selected (e.g., 0.63 in a typical implementation) and divided into random subsamples. As shown, the source sample 1300A may be divided into subsamples 1300B. For each subsample 1300B, a decision tree may be constructed in 1300C based on a random set of m features (covariances), and the results may be classified into leaves. In 1300D, bootstrap aggregation may be performed using the results from all the collected and averaged constructed trees. In 1300E, the final prediction may be derived from each of the predictions in 1300D.

[0110] Several machine learning approaches were attempted to optimize the F1 score, and random forests outperformed the other models. The data was split into 80% for training and 20% for testing to fit the model. Random forests were used to construct numerous decision trees to predict individual activities / objects using the training dataset, and weighted sum predictions for each activity / object were output.

[0111] The F1 score was used to quantitatively evaluate the predictive accuracy from the random forest. The F1 score measures how well the model classifies a particular activity, such as swallowing, as shown by the results and calculations in Figures 14 and 15. Using parameters from the feature manipulation in both rounds, improvements in the F1 score were observed for several activities, as shown in Figures 15 and 16. As shown in Figure 14, via calculation and result 1400, a recall of 0.987 and a precision of 0.975 were output based on true positive, false negative, and false positive test results 1400A. An F1 score of 0.98 was output in 1400B. 1400C shows the model used to output the criteria for the F1 score.

[0112] Chart 1500 in Figure 15 shows the F1 score results divided by morning, evening, and overall. As shown, based on the signal acquisition device 10 used, swallowing has the highest overall F1 score at 0.98, and ocular iso has the lowest overall F1 score at 0.39. Figure 16 shows Chart 1600 with results from random forest models using a CNN model, a first variance model, and a second variance model. As shown, certain features (parameters) show improvement compared to other features based on the model used. Figure 17 shows Chart 1700 with results from studies using all variables from a specific task (activity) to predict each subject, or using all given activities to predict the subject, divided by morning and evening scores. As shown, chewing has the highest morning-evening combined score (0.79 and 0.90, respectively), and sadness has the lowest morning-evening combined score (0.55 and 0.75, respectively). Chewing, wrinkle iso, and speech activities were among the top activities in predicting individual subjects (F1 score > 0.85). Tracking sadness, eye, and anger activities was less reliable in predicting individuals. Slight differences in predicting individuals from morning to evening were recorded. Generally measured facial movements varied throughout the day, morning and evening. As a result of higher F1 score-based reliability, we considered the application of devices used in clinical settings, focusing on measuring chewing, speech, and swallowing.

[0113] Figure 18 shows Chart 1800 of algorithmic options for selection (e.g., for application to random forests). One or more algorithms (rational discrete short-time Fourier transform (DSTFT), Fourier transform, discrete wavelet transform with linear classifier, Gabor wavelet transform, Hjorth parameter, Hilbert-huang transform (HHT), smoothed Wigner-Ville distribution (SWVD)) can be applied to data collected by a signal acquisition device. Chart 1800 provides the methods, applications, advantages, and limitations of these exemplary algorithms.

[0114] Figure 19 shows Chart 1900 of bandwidth power characteristics for 10 individuals regarding swallowing activity. Four different bandwidth powers are shown in each of the four charts. As illustrated, for each individual, multiple amplitudes are collected during both morning and evening examination times. According to one implementation, each of the four different bandwidth powers in each of the four charts could be an evaluation item in a clinical trial. For example, the data represented in each of the four charts could be clinical outputs sought as a result of a clinical trial.

[0115] Figure 20 shows the clusters and corresponding channels in Table 2000. Each feature extracted from the individual signals 30 may be placed in a cluster based on, for example, CV clustering, ICC clustering, random forest clustering, etc., as disclosed herein. Features having similar results may be grouped together, as shown in the examples provided in Table 2000. Clusters may be used to trim the total number of features to those that are most definitively relevant to the clinical output.

[0116] Figure 21 shows graph 2100 with features clustered based on type (e.g., amplitude, frequency, bandwidth, power channel, and / or other factors). Clustering can be used to trim the total number of features to those that are most critically relevant to clinical output.

[0117] Figure 22 shows a heatmap of CV 2200 based on a mixed-effects model using morning measurements with the effects of age, BMI, and sex fixed. The CV heatmap 2200 shows the CV values ​​of feature 2200A for task 2200C based on legend 2200B, ranging from 0 to 1.2. The results of the CV heatmap 2200 can be used to identify which features are reliable (e.g., low variance) in order to determine clinical output (e.g., disease designation). For example, a lower CV for a given feature and activity may indicate that the given feature can be reliably repeated across multiple tests (e.g., meets a CV confidence threshold). Clinical trials may require that the features used for the trial meet such a CV confidence threshold.

[0118] Figure 23 shows Chart 2300, which illustrates how reliably a given task can be used to classify individuals. Bandwidth power measurements of AUC for various tasks are shown in Table 2300A. The results of such measurements for smile Iso are shown in Chart 2300B, and the results of such measurements for sad emotion are shown in Chart 2300C. Higher bandwidth power AUC measurements may indicate that a given task (e.g., smile Iso) meets the AUC threshold for classifying individuals (e.g., distinguishing one individual from the next), while lower bandwidth power AUC measurements may indicate that a given task (e.g., sad emotion) does not meet the AUC threshold. Therefore, the signal capture device 10 used to generate the measurements shown in Figure 23 may be more reliable in distinguishing individuals when the smiling activity is performed compared to when sad emotion is experienced.

[0119] Figure 24 (and Figure 24-1) shows heatmap 2400 (including heatmaps 2400A and 2400B) for parameter 2400C, similar to chart 904 in Figure 9. Heatmap 2400 shows heatmaps of variables and activities that indicate qualitative differences between subjects and between activities. Heatmap 2400 shows Z-scores for each of several features, calculated based on normalized raw data separated by task (e.g., anger, chewing, eyes, eye iso, medial iso, jaw, left gaze-left (L gaze L), left gaze-right (L gaze R), lateral iso, sadness, smile iso, surprise, swallowing, conversation, upward gaze, wrinkle iso), individual (person), and time (e.g., morning / evening). The Z-scores are distributed using values ​​in the range of -3 to +3, with each value assigned a color as shown in the legend.

[0120] Figure 25 shows task 2500A plotted on a UMAP chart 2500. The UMAP chart 2500 can be generated based on a visual representation produced by reducing each of several parameters (e.g., parameter 2400C from Figure 24) to two values. Thus, each trial with multiple iterations (e.g., rows) is reduced to two iterations (e.g., rows), and the results are plotted on the UMAP. The UMAP chart 2500 shows the resulting data separated by task 2500A. For example, the UMAP chart 2500 can be used to cluster based on each of task 2500A. Similarly, Figure 26 is a UMAP chart 2600 generated using the same data used to generate the UMAP chart 2500. The UMAP chart 2600 shows the resulting data separated by individual 2600A. For example, the UMAP chart 2600 can be used to cluster based on each of individual 2600A. Similarly, Figure 27 shows UMAP chart 2700, generated using the same data used to generate UMAP charts 2500 and 2600. UMAP chart 2700 shows the resulting data separated by collection time 2700A. For example, UMAP chart 2700 can be used to cluster based on each of the collection times 2700A.

[0121] Figure 28 (and Figures 28-1 to 28-3) is similar to Figure 19 and shows Chart 2800 of various characteristics for 10 individuals regarding swallowing activity. 31 different data plots are shown in each chart. As illustrated, for each individual, multiple amplitudes are collected for each individual during both morning and evening examination times. The data represented in Chart 2800 can be used, for example, to generate a Z-score-based heatmap shown in Figure 24. According to one implementation, the 31 different data plots, individually or combined, could be evaluation items in a clinical trial as a dataset. For example, the data or combined representations of data represented in the individual data plots could be clinical outputs sought as a result of a clinical trial.

[0122] Figure 29 shows the random forest importance values ​​across multiple features 2900B in Chart 2900, based on the final decision shown in Legend 2900A. Chart 2900 shows the distribution of each feature 2900B for a given task. Chart 2900 can be generated using the Boruta feature selection algorithm for feature selection from a dataset. The applied Boruta algorithm can act as a wrapper algorithm around the random forest. The Boruta algorithm adds randomness to a given parameter-based dataset by creating shuffled copies of all features (e.g., shadow features). A random forest classifier can then be trained on the augmented dataset. A feature importance measure (e.g., mean decreasing precision) may be applied to assess the importance of each feature, where higher is more important. In each iteration, a check may be made to see whether the actual feature has a higher importance than the best of its shadow features (e.g., whether the feature has a higher Z-score than the maximum Z-score of its shadow features). Features deemed not very important (e.g., exceeding the Volta threshold) can be discarded to identify clinical output. Clinical trials may require the use of features that meet or exceed the Volta threshold. The algorithm can complete a cycle when all features have been determined or eliminated, or when a specified limit for random forest execution has been reached.

[0123] The random forest importance values ​​across multiple features shown in Figure 29 can be used to determine how important each feature is to classify a particular activity. Features with high importance (e.g., above the Volta threshold) may be designated as clinically relevant features. As an example, the results shown in Figure 29 can be used to reduce the number of features by comparing the actual value of a given feature with a shuffled value (generated based on replication). A given feature that adds more value to the classification (e.g., determined by removing a feature and determining how much information is lost) can receive a higher score. Machine learning can be used for permutation-based score identification. A given feature may be determined to be more clinically relevant to a given task than another feature. Additionally or alternatively, endpoints in clinical trials may be determined when the removal of any remaining features reduces their ability to classify information by a given threshold. Random forest scores can be obtained based on the results of the Volta model, which are given higher scores. Extracted features with random forest scores above the random forest threshold are identified as clinically relevant features.

[0124] Figure 30 shows Chart 3000 including gap statistics. Gap statistics (e.g., k-values) can be used to determine the optimal number of clusters for a given dataset. Chart 3000 may relate, for example, to the Spearman correlation plot in Figure 10. This Spearman correlation can be used to identify the number of clusters based on available signal-based data. As illustrated, the optimal number of clusters 3000A based on the data presented in Chart 3000 is 4, and each given data point having 4 clusters provides a threshold balance between the differences between features within a cluster and the number of clusters.

[0125] Figure 31 shows the Z-scores for tasks being collected in the morning, using heatmap 3100. The Z-scores are for each feature 3100A for each task 3100B based on legend 3100C, with colors assigned within a range of -3 to +3. Figure 32 shows the Z-scores for tasks being collected in the evening, using heatmap 3200. The Z-scores are for each feature 3200A for each task 3200B based on legend 3200C, with colors assigned within a range of -3 to +3. As disclosed herein, the Z-scores shown on the heatmaps (e.g., heatmaps 3100 and / or 3200) are normalized to their respective ranges such that higher values ​​within the range of possible values ​​correspond to higher Z-scores, and lower values ​​within the range of possible values ​​correspond to lower Z-scores.

[0126] Figure 33 shows the LOOCV Z-scores across tasks on heatmap 3300. Heatmap 3300 in Figure 33 is generated by a procedure used to estimate the performance of a machine learning algorithm. The data is provided to individuals 3300A and tasks 3300B, based on a legend 3300C assigned colors, with values ​​ranging from -3 to +3. In LOOCV, the number of folds can be the same as the number of instances in the dataset. Thus, a learning algorithm can be applied once per instance, using all other instances as the training set and selected instances as a single-item check set. For example, to generate heatmap 3300, individual data can be removed from the training set of a predictive machine learning algorithm. The remaining data can be used to train the algorithm. The removed individual data can then be predicted using that algorithm. The Z-scores shown in heatmap 3300 can be generated based on how the predictions compare to the actual data and / or how well the predictions identify a given task for an individual.

[0127] Figure 34 shows the Z-scores in heatmap 3400 for the standard deviation (SD) during morning data collection. Heatmap 3400 shows the Z-scores for the standard deviation of features 3400A in task 3400B, based on legend 3400C, with colors assigned to values ​​ranging from -3 to +3. Figure 35 shows the Z-scores in heatmap 3500 for the SD during evening data collection. Heatmap 3500 shows the Z-scores for the standard deviation of features 3500A in task 3500B, based on legend 3500C, with colors assigned to values ​​ranging from -3 to +3. Standard deviation indicates the amount of variability in the feature data, with a higher standard deviation potentially indicating lower confidence, while a lower standard deviation may indicate higher confidence.

[0128] Figure 36 shows the CV by heatmap 3600 for a mixed-effects model in evening collection. Figure 36 is similar to Figure 22. Figure 36 shows the CV based on a mixed-effects model using morning measurements with the effects of age, BMI, and sex fixed. The CV heatmap 3600 shows the CV values ​​for features 3600A for task 3600C based on legend 3600B, with colors assigned to values ​​ranging from 0 to 1.2. The results of the CV heatmap 3600 can be used to identify which features are reliable (e.g., low variance) in order to determine clinical output (e.g., disease designation). For example, a lower CV for a given feature and activity may indicate that the given feature can be reliably repeated across multiple tests (e.g., meets a CV confidence threshold). Clinical trials may require that the features used for the trial meet such a CV confidence threshold.

[0129] Figure 37 (and Figure 37-1) shows the bandwidth power measurement spread 3700 for smile collection. Spread 3700 is similar to that calculated for swallowing in Figure 28. Spread 3700 shows various features of smile activity for 10 individuals. Ten different data plots are shown for each spread. As shown, for each individual, multiple amplitudes were collected for each individual during both morning and evening examination times. The data represented in spread 3700 can be used, for example, to generate a Z-score based heatmap shown in Figure 24.

[0130] Figure 38 (and Figure 38-1) shows a clustered ICC heatmap 3800 for evening measurements with the effects of age, BMI, and sex fixed. Heatmap 3800 is based on feature 3800A of Task 3800C based on legend 3800B, with colors assigned in the range of values ​​from 0 to 1. The ICC measurements shown on heatmap 3800 indicate how similar the results are for a given person when the same measurement is calculated for that person across multiple collections. The results identify correlations with data for the same individual. Thus, heatmap 3800 indicates test-and-retest reliability. Higher ICC values ​​(e.g., 1) indicate low variability across multiple tests for a given individual, and therefore the data for that individual correlates with itself. Lower ICC values ​​(e.g., 0) indicate high variability across multiple tests for a given individual. Furthermore, heatmap 3800 shows which features meet the ICC threshold, allowing features associated with higher ICC values ​​(e.g., 1) across multiple subjects to be used in clinical trials as reliable sources of data about individuals. As shown in 3800D, various ICC correlation values ​​can be clustered (e.g., into six clusters in this example).

[0131] Figure 39 shows another clustered ICC heatmap 3900 for evening measurements with the effects of age, BMI, and sex fixed. Heatmap 3900 is based on feature 3900A of Task 3900C based on legend 3900B, with colors assigned in the range of values ​​from 0 to 1. The ICC measurements shown on heatmap 3900 indicate how similar the results are for a given person when the same measurement is calculated for that person across multiple collections. The results identify correlations with data for the same individual. Thus, heatmap 3900 indicates test-and-retest reliability. Higher ICC values ​​(e.g., 1) indicate low variability across multiple tests for a given individual, and therefore the data for that individual correlates with itself. Lower ICC values ​​(e.g., 0) indicate high variability across multiple tests for a given individual. Furthermore, heatmap 3900 shows which features meet the ICC threshold, allowing features associated with higher ICC values ​​(e.g., 1) across multiple subjects to be used in clinical trials as reliable sources of data about individuals. As shown in 3900D, various ICC correlation values ​​can be clustered (e.g., into four clusters in this example).

[0132] Figure 40 shows t-distribution stochastic neighbor embedding (t-SNE) charts 4000A, 4000B, and 4000C for individuals, tasks, and time. Chart 4000A corresponds to reduced data based on individual 4000D, chart 4000B corresponds to reduced data based on task 4000E, and chart 4000C corresponds to reduced collection time 4000F. Charts 4000A, 4000B, and 4000C can be generated based on algorithms that reduce the number of datasets to two dimensions. Charts 4000A, 4000B, and 4000C can be used to compare how similar or dissimilar each dataset is (for example, how variable the data associated with each individual in 4000D is, as shown in 4000A). Comparing charts 4000A, 4000B, and 4000C may reveal contributions from variance based on given parameters (e.g., over individuals 4000D, over tasks 4000E, and / or over collection time 4000F).

[0133] Figure 41 shows various charts 4100 for visualizing data reduced to two dimensions based on task 4100A. Chart 4100B is based on principal component analysis (PCA), where the principal components of the set of points in real coordinate space are a sequence of p unit vectors, and the i-th vector is the direction of the line that best fits the data while being orthogonal to the first i-1 vectors. Chart 4100C is based on both PCA and t-SNE reductions. Chart 4100D is based on t-SNE reduction. Chart 4100E is based on UMAP reduction.

[0134] Figure 42 shows UMAP charts 4200 for individuals in chart 4200A, tasks in chart 4200B, and time in chart 4200C. Charts 4200A, 4200B, and 4200C may be similar to those provided in Figures 25-27. Chart 4200A may show tasks plotted using UMAP reduction. Chart 4200A shows a visual representation generated by reducing each of several parameters (e.g., parameter 2400C from Figure 24) to two values. Thus, each test with multiple iterations (e.g., rows) is reduced to two iterations (e.g., rows), and the results are plotted on the UMAP chart. Chart 4200A shows the resulting data separated by task. For example, chart 4200A may be used to cluster based on each of the tasks. Similarly, chart 4200B is generated using the same data used to generate chart 4200A. Chart 4200B shows the resulting data separated by individual. For example, Chart 4200B may be used to cluster based on each individual. Similarly, Chart 4200C is generated using the same data used to generate Chart 4200A or Chart 4200B. Chart 4200C shows the obtained data separated by collection time. For example, Chart 4200C may be used to cluster based on each time.

[0135] Figures 43 and 44 show the confirmation results in charts 4300 and 4400 for different tasks across various channels. The data provided in charts 4300 and 4400 can be generated based on the Volta analysis (e.g., feature importance) described in Figure 29. Charts 4300 and 4400 show the confirmation results 4300B and 4400B for tasks 4300A and 4400A for parameters 4300C and 4400C. For example, Figure 29 shows the analysis results for a single task, while Figures 43 and 44 show the analysis results for multiple tasks. Charts 4300 and 4400 indicate whether given data (e.g., results 4300B and 4400B for tasks 4300A and 4400A for parameters 4300C and 4400C) are clinically relevant in identifying clinical outcomes. Relevant data are shown as confirmed, and irrelevant data are excluded. Data that do not meet the relevant or irrelevant thresholds are designated as provisional. Clinical trials may require that parameters used in a trial meet the relevant threshold for confirmation in order to determine clinical outcomes.

[0136] [Example 2] This example facilitates the determination of clinically relevant features based on the capabilities of the signal-capturing device 10 and clinical outcomes by providing a procedure used to evaluate the capabilities of the signal-capturing device 10. While this example provides a specific application of the method disclosed herein, it should be understood that additional applications of the method may be implemented.

[0137] As a first step towards developing a digital assessment of neuromuscular diseases, a study was conducted to determine whether biometric sensor devices could be used to objectively measure facial muscle and eye movements intended to represent PerfO using a task designed to model clinical performance outcome assessment (PerfO). This is referred to as simulated PerfO activity. The specific objectives of this study were to determine whether the raw EMG, EOG, and EEG signals from the biometric sensor devices could be processed to extract features that describe these waveforms; to determine the quality, test-retest reliability, and statistical properties of the biometric sensor device feature data; to determine whether the features derived from the biometric sensor devices could be used to determine differences between various facial muscle and eye movement activities; and to determine which features and types of features are important for a classification of simulated PerfO activity levels.

[0138] It can be understood that the clinical outcome being tested in this example is to identify, based on a biometric sensor device, which features and types of features are important for classifying simulated PerfO activity levels.

[0139] The biometric sensor device used in this embodiment is an ear-worn wearable originally developed to measure cognitive function. Since the biometric sensor device measures, for example, electroencephalography (EEG), electromyography (EMG), and electrooculography (EOG) data, it may also have the potential to objectively quantify the activity of facial muscles and eye movements relevant to the evaluation of neuromuscular disorders.

[0140] A total of N=10 healthy volunteers participated in the study. Each participant performed 16 simulated PerfO activities, including talking, chewing, swallowing, closing their eyes, looking in different directions, puffing out their cheeks, biting an apple, and making various facial expressions. Each activity was repeated four times in the morning and four times in the evening. A total of 161 summary features were extracted from EEG, EMG, and EOG biosensor data. Feature vectors were used as input to a machine learning model to classify the simulated PerfO activities, and the model performance was evaluated on a presented test set. In addition, a convolutional neural network (CNN) was used to classify low-level representations of the raw biosensor data for each task, thereby evaluating model performance and directly comparing it to feature classification performance.

[0141] The predictive accuracy of the model for the classification capabilities of biometric sensor devices was quantitatively evaluated. The results showed that the examined biometric sensor devices could potentially quantify different aspects of facial and eye movements and could be used to distinguish simulated PerfO activity. In particular, the identified clinically relevant features indicated that the examined biometric sensor devices distinguished conversation, chewing, and swallowing tasks from other tasks where the observed F1 score was above 0.9. EMG features contributed to classification accuracy for all tasks, while EOG features were important for classifying gaze tasks. Analysis using summary features proved to perform better than CNNs for activity classification.

[0142] As further described herein, the biometric sensor device was determined to meet clinical thresholds so that it can be used to measure cranial muscle activity related to neuromuscular disease assessment. The classification performance of simulated PerfO activity using summary features enables strategies for detecting disease-specific signals relative to a control group, as well as monitoring intra-subject treatment responses.

[0143] Dysfunction of facial / cranial and ocular movements is a key feature of several neurological disorders affecting multiple levels of the brainstem. Examples include complete facial weakness, bifacial dysphagia, ptosis resulting from facial nerve palsy or stroke, and dysphagia caused by myocardial disorders, such as myasthenia gravis, dystonia, complex extraocular dysphagia, asthenia, and dysphagia caused by Parkinson's (and other neurodegenerative) conditions.

[0144] As discussed herein, the clinical assessment of these symptoms remains a challenge in medicine and clinical research. Existing clinical assessments, such as clinician-reported outcomes (ClinRO) or patient-reported outcomes (PRO), may require patients to visit facilities frequently, rely primarily on subjective scales, and may not always reflect the patient's condition in the real world. Importantly, patients' symptoms are intermittent and can change throughout the day, making reliable assessment difficult. Finally, they can vary from patient to patient in response to adaptations to increased muscle weakness. For example, Chart 500 in Figure 5 highlights the subjective nature of observation-based physician scores 502. Such subjective scoring can lead to patient-to-patient and provider-to-provider variability.

[0145] While tools exist for the quantitative analysis of cranial muscle function, these tools have significant limitations. For example, facial movements can be measured using video-based techniques that employ either still images or video capture. Surface EMG, which records the electrical movements of facial muscles, can also be used alone or in combination with video-based methods. Small-scale studies have suggested that EOG, which measures electrical potentials from the anterior to posterior part of the eye, can detect differences between Parkinson's disease patients and control groups. Screen-based trackers and wearable glasses have been used to monitor extraocular movements and supracranial activity (e.g., blinking). In their current applications, these approaches are cumbersome, difficult to implement, and, most importantly, can only capture short-duration facial movements in an artificial setting.

[0146] Therefore, the methods disclosed herein are advantageous in providing an opportunity to identify and / or develop novel non-invasive approaches for measuring individual attributes (e.g., cranial manifestations of neuromuscular and neurodegenerative diseases) to address problems in important patient populations. The technologies disclosed herein play a role in supporting clinicians' diagnosis and assessment of disease progression, as well as outcome assessment in clinical research. If such approaches can leverage wearable sensing technologies, they may be able to address the challenges of existing clinical sensors, which are limited to use in highly controlled environments as opposed to more natural environments (e.g., home).

[0147] The biometric sensor device being tested is an over-ear device developed to measure neural and physiological processes. Electrophysiological signals are acquired at 250 Hz via four reusable electrodes made from conductive silicon material. The device's electrodes are positioned on the scalp above the left and right mastoid processes, directly above the left and right ears, yielding raw biosignal data similar to that which can be acquired at electroencephalogram (EEG) reference positions T3, T4, M1, and M2 with 10 to 20 electrode placements. This EEG is a measurement of surface electroencephalogram function. This electrode configuration also allows for high-fidelity acquisition of EMG activity from the activation of the temporalis muscle and surrounding muscle groups, as well as EOG signals resulting from eye deflection.

[0148] Traditional clinical assessments using biophysiological data can be invasive, expensive, and time-consuming, while the biometric sensor devices examined aim to provide high-fidelity data acquisition and processing to the general population. EMG, EEG, and EOG signals monitored by the examined biometric sensor devices were used for the detection and evaluation of a wide variety of physiological phenomena, including sleep monitoring, drowsiness detection, and acute postoperative pain quantification. Based on the identification of facial muscle-related features 50 using a biometric sensor device (signal capture device 10), it was determined that this device has the potential to support outcome assessment of neuromuscular diseases by objectively quantifying clinical and eye movement tasks through the capture and analysis of biosignal data. Objective quantification quality was evaluated by extracted features 40 generated from individual signals 30 collected via the biometric sensor device. Individual signals 30 were generated using a signal manipulation module 20 that received signals from the biometric sensor device. The techniques disclosed herein were applied to identify clinically relevant features 50 from the extracted features 40. The clinically relevant feature 50 met the threshold for clinical outcomes, including the diagnosis and / or treatment of neuromuscular disease.

[0149] A key challenge addressed by the methods disclosed herein is that raw biosignal data is inherently noisy due to several factors, such as participant movement during clinical evaluation, potential perturbations in electrode-skin contact, and / or artifacts from cardiac activity. In addition, similar factors naturally induce artifacts in the acquired signal data, and the overlap of EEG, EMG, and EOG signals in typical frequency ranges makes direct separation and analysis of waveform data difficult.

[0150] Therefore, in order to develop digital assessments of neuromuscular diseases, this exemplary study was conducted to determine whether biometric sensor devices can measure facial muscle and eye movements. Some specific objectives of this study are as follows: to determine how biometric sensor device EMG / EOG / EEG signals can be processed to extract features; to determine the quality, test-retest reliability, and statistical properties of biometric sensor device feature data; to determine whether features derived from biometric sensor devices can quantify various facial and eye activities; and to determine which features are important for activity level classification (e.g., clinically relevant features) compared with classification methods for raw biosignal data.

[0151] In this study, 16 simulated performance outcome assessments (simulated PerfO) were designed to evaluate facial and eye movements using biometric sensor devices in N=10 control volunteer participants. A feature manipulation pipeline tailored to the purpose was then implemented. This implementation involved deriving features from EMG, EOG, and EEG waveforms, comparing and evaluating feature relationships with each other, and qualitatively assessing how the features differentiated from the simulated PerfO activities. The steps of conducting this exemplary study support the usefulness of the framework for developing digital assessments and the analytical validation steps. In summary, the results from this exemplary study highlight the usefulness of biometric sensor devices as potential measurement tools in clinical trial settings for evaluating facial and eye movement tasks, enabling further clinical development using these and similar devices. The usefulness of biometric sensor devices is determined by the identification of sufficient clinically relevant features, as described herein.

[0152] To evaluate how well a biometric sensor device can classify (e.g., distinguish between) facial muscle tasks and eye movement tasks, a study was conducted with N=10 participants who performed 16 facial muscle task movements (simulated PerfO) four times in the morning and four times in the evening. Table 1 shows the demographic characteristics of the study participants.

[0153] [Table 1] Raw biosignal data from a biometric sensor device is processed into 161 summary features, the majority of which represent EMG, EOG, and EEG waveforms. As shown in Figure 1A, the biometric sensor device (signal capture device 10) provided the raw signal to the signal manipulation module 20. The signal manipulation module 20 generated individual signals 30 (e.g., EMG, EOG, and EEG waveforms) from the raw signal. The 161 summary features were generated based on the characteristics of the individual signals 30.

[0154] The process for summarizing raw biometric sensor device biosignal data into features is described in detail herein. Briefly, features are calculated from EMG, EOG, and EEG waveform components, which are separated from the raw mixed-waveform biosignal via special signal coupling and filtering mechanisms (e.g., by the signal manipulation module 20). A high-level overview of the feature manipulation process 4500 is also summarized in Figure 45.

[0155] As shown in Figure 45, a mixed signal waveform 4502 is received from a biometric sensor device. The signal separation module 4504 can extract individual virtual EEG signals 4506A, virtual EOG signals 4506B, and virtual EMG signals 4506C from the mixed signal waveform 4502. An event-based segmentation algorithm can be applied in the event-based segmentation algorithm 4508, and feature computation in 4510 can result in feature extraction via a feature vector representation 4512. As shown, the signal separation module 4504 is applied to the mixed signal derived from the biometric sensor device to separate the EEG, EMG, and EOG waveforms into their component parts. These signals are then applied to the event-based segmentation algorithm 4508 to extract features.

[0156] As shown in Figure 46, features can represent both the frequency and time domains of biometric device signals through amplitudes at time 4600A and frequency 4600B collected when a subject drinks water. Figure 46 shows the time and frequency representation of EMG activity resulting from a participant drinking water. Plot 4600 shows EMG data for approximately 6.5 seconds in both the time domain 4600A and the frequency domain 4600B.

[0157] Representative mixed signal waveforms 4502 were collected for each of the 16 simulated PerfO activities. For example, Figure 47 shows the qualitative differences observed from representative signals 4700 from each of the 16 simulated PerfOs. As shown, each activity has a qualitatively different waveform. Representative signal 4700 shows the EMG activity visualized in the time domain across the 16 activities.

[0158] As an example, several features extracted from representative signals 4700 from each of the 16 simulated PerfOs can be used to generate a Z-score heatmap, as further described herein and also shown in Figures 8 and 9.

[0159] Tables 2-1 to 2-3 show a list of extracted features in this example, where features are described in the categories described. Amplitude features, zero crossing rate, standard deviation, variance, root mean square, kurtosis, frequency, bandwidth power, skew, and other standard waveform features were processed from the biometric sensor data. Features were selected according to the standard feature processing pipeline. Amplitude features describe the amplitude or maximum distance from the baseline of each wave in the relevant component space. Bandwidth power features describe the average power of the waveform in a specific frequency range (where there are multiple frequency ranges specific to each signal type for the biometric sensor device). Other specified features mathematically describe the shape, variance, or complexity of the EMG, EOG, or EEG waveform.

[0160] [Table 2-1]

[0161] [Table 2-2]

[0162] [Table 2-3] The biometric sensor device-based features shown in Tables 2-1 to 2-3 above can measure intrinsic aspects of facial and eye movements. The relationships between parameters of all 16 simulated PerfO activities were analyzed by performing Spearman correlations of all parameters with respect to each other (e.g., by the method described with respect to Figures 4C and 10). To determine the optimal number of parameter sets that describe the overall signal variation, k-mean clustering of Spearman correlations of all features was performed on all activities with respect to each other, as shown in Figure 48. Six intrinsic clusters of parameters were determined from Spearman correlations (based on k-mean clustering). k-mean clustering is a vector quantization method that divides n observations (e.g., features based on Spearman correlations) into k clusters, where each observation belongs to the cluster with the nearest mean and serves as a prototype for the cluster.

[0163] As described herein, the Spearman correlation chart 4800 is used to identify relationships between features. Highly correlated features are likely to measure similar aspects of the biological and / or other signals (e.g., electrical activity) of the face collected by the biometric sensor device (e.g., similar aspects of individual signals 30). The Spearman correlation chart 4800 is used to identify six clusters so that similar features within the same cluster are omitted or otherwise reduced in order to reduce overlap in the analysis. For example, cluster-based reduction may be applied to identify clinically relevant features 50. Figure 47 shows the Spearman correlations of all 161 Earable features to each other, represented as a heatmap. k=6 clusters (optimal number) from k-mean clustering are shown. All 16 simulated PerfOs are pooled for the correlation analysis shown in Figure 47.

[0164] In this example, the amplitude and bandwidth power parameters tended to cluster together in two of the six clusters, while other parameters, such as those from the frequency domain, clustered separately.

[0165] UMAP dimensionality reduction was performed to investigate the qualitative differences between 16 simulated PerfO activities (across each participant and time point, as well as all 161 biometric sensor device parameters). The qualitative differences between the 16 simulated PerfO activities 4900A are shown in Chart 4900 in Figure 49. There was some overlap among some of the activities 4900A, but activities such as swallowing were clearly separated from the rest using this approach. The UMAP dimensionality reduction shown in Chart 4900 can be used to identify clusters of data and / or to separate data by dimension (e.g., by task). The UMAP dimensionality reduction for all 161 biometric sensor device features is shown in Figure 49. Each individual activity iteration is a point on Chart 4900. The visual features of a given point represent the activity performed during that activity.

[0166] Figure 50 includes Chart 5000, which provides heatmaps of feature Z scores across data showing the differences between tasks 5004 for different classes 5002 of features (amplitude, bandwidth power, frequency, kurtosis, other, skew, time, variance): 5000A (EEG-based features), 5000B (EMG-based features), 5000C (EOG-based features), and 5000D (other features). Data for each individual 5006 is collected for time 5008 and represented as a Z score based on Legend 5010, with colors assigned in the range of -3 to +3. Chart 5000 may be generated and may contain information in a similar manner to Figures 8, 9, and 24 described herein. In summary, the results shown in Chart 5000 demonstrate the usefulness of biometric sensor devices for generating parameters that can describe unique simulated PerfO activity.

[0167] Figure 50 contains a heatmap of all 161 biometric sensor device features (rows) for all activity repetitions (columns). The columns are first sorted by the 16 activities in the study, then by the participant within each activity, and finally by the time the activity was performed, as described above.

[0168] To assess the test-retest reliability of biometric sensor device features, linear mixed-effects modeling is used with participants as random effects to evaluate feature properties. Table 3 shows the variance component analysis of the biometric sensor device features. First, for each of the 161 biometric sensor device features, the ICC for participants to assess the variance associated with each person for each feature is determined based on the flow in Table 4. As described herein, ICC is a measure of how similar, and therefore how reliable, identical data from the same participant are for the same activity, ranging from 0 to 1 (for example, an ICC less than 0.5 can be interpreted as low reliability, an ICC between 0.5 and 0.7 as moderate reliability, and an ICC greater than 0.7 as a reliable metric). The observed ICC values ​​ranged from 0 to 0.92, and the average ICC value for all parameters across the 16 activities was 0.31. Next, the CV for each parameter within the participants across each time point (morning and evening) is calculated according to the method disclosed herein. The variance of each feature of each activity related to the time of activity (morning or evening), the individual participant themselves, and each individual test repeat, as well as unexplained variance, is calculated. The ICC calculation, CV, and / or variability are used to identify clinically relevant features 50 from the extracted features 40, for example, as shown in Figure 1A. In this example, the results provide a metric that supports the reliable measurement of within-subject variability for many biometric sensor device features and allows for ranking of candidate features for further downstream analysis. Thus, such features may be designated as clinically relevant features 50.

[0169] [Table 3]

[0170] [Table 4] The biometric sensor device being tested is determined to be capable of accurately classifying the motor activity of several facial muscles. To investigate whether the biometric sensor device data can classify one of 16 simulated PerfO activities, a random forest classification model described herein is constructed to detect each activity (one-to-all classifications) from the other 15 activities (as described with reference to, for example, Figures 29, 43, and 44). The activity detection F1 score is used as a primary metric to evaluate the model performance, as further described herein.

[0171] Following development evaluation on a test dataset for all 161 features, a second model is constructed for activity level classification. This second model uses an optimized set of biometric sensor device features, aiming to eliminate noisy features that do not contribute to overall classification performance. Feature reduction using the Boruta package is performed to determine the optimized set of biometric sensor device features. Such feature reduction is described in this disclosure with reference to Figures 29, 43, and 44. For example, features from all 161 in Figures 29, 43, and 44 are copied (e.g., designated as shadow features), and their classification labels are randomly shuffled. Each shadow feature is compared to its actual value over 1,000 iterations of classification, and only features that perform better than a given threshold (e.g., 50%) are designated as definitive. This analysis shows the 101 definitive sets of features from Figures 29, 43, and 44 used for the second classification model.

[0172] To evaluate how well biometric sensor device features perform with the use of low-level representations of waveform data from biometric sensor devices, a CNN model is constructed using raw biosignal data from biometric sensor devices to classify 16 simulated PerfO activities, as shown in Figure 51. Figure 51 shows Model 5100, which implements the final CNN architecture diagram implemented for activity classification. A single-channel spectrogram computed from the segmented waveform is input to the model during classification. A probability distribution across each of the 16 activities is output. The activity associated with the highest output likelihood estimate is inferred. In this modeling, a fixed-size spectrogram quantifying how the power at a given frequency changes as a function of time is computed from the simulated PerfO signal segments and used as model input.

[0173] Figure 52 shows the activity chart 5200 for activity 5200A, which has a level classification F1 score for all biometric sensor device features (161 features) 5200B, Boruta-selected biometric sensor device features (101 features) 5200C, and raw waveform data (CNN) 5200C. The F1 score ranges from 0 to 1, with 1 indicating a perfect classification. The biometric sensor device features (101 features) 5200C selected by Boruta may be clinically relevant features 50 extracted from the biometric sensor device features (161 features) 5200B (as shown, for example, in Figures 29, 43, and 44). The raw waveform data (CNN) 5200C may be generated using the model 5100 in Figure 51. Figure 53 includes a heatmap 5300 showing a feature attribute analysis using SHapley Additive exPlanations (SHAP) values ​​for each feature (row) 5300A for each activity (column) 5300B determined on the model from the full set of 161 features. The SHAP values ​​are Z-scored across all activities and assigned colors ranging from -3 to +3, as shown in the legend 5300C.

[0174] As shown in Figure 52, the F1 scores for classification accuracy are compared between the complete set of 161 features 5200B, the optimized set of 101 features 5200C, and predictions from CNN 5200D. Feature attribute analysis by SHAP is applied in Figure 54 to determine the most important underlying features for the complete RF model with 161 features (e.g., 50 clinically relevant features). For a particular activity prediction, the SHAP value of a biometric sensor device feature is calculated as the change in the expected value of the model output when this feature is observed, compared to when it is absent with respect to the test set prediction. The impact of each feature when added to the model is summed and averaged across all 161 biometric sensor device features used. In Figure 53, the features are represented as average SHAP values ​​and shown in the heatmap (log10) 5300. The contribution percentage of each waveform group of features to each activity is determined and shown in Table 5. The normalized sum of absolute SHAP values ​​for each activity is compared to the sum within the EMG, EEG, and EOG features and normalized by the number of features in that group in order to calculate the percentage contribution of each waveform to classification accuracy.

[0175] [Table 5] Table 5 includes 16 simulated PerfO activities and shows how EMG, EEG, and EOG feature sets contribute to classification accuracy. Table 5 shows the relative percentage contributions of EMG, EEG, and EOG to classification importance, along with the normalized sum of absolute SHAP values ​​from the RF model. Feature importance is normalized based on the total number of features in each EMG, EEG, or EOG group compared to the total number of features in all three categories. Features unrelated to any waveform are excluded from this analysis.

[0176] As disclosed herein, a total of 10 healthy volunteers (5 men and 5 women) contributed to this exemplary study. All participants completed two 45-minute sessions. During each session, each participant was asked to complete a set of tasks listed in Table 6 below. These tasks were selected to represent tasks that MG patients might find difficult to complete. Participants were asked to take a one-minute break between each task.

[0177] [Table 6] Each research participant attended two research sessions (one in the morning and one in the evening). The training sessions were conducted one-on-one by a research moderator. In the morning session, the research moderator completed an Informed Consent Form (ICF) with the participant to confirm that they understood the format and agreed to participate. Participants were given time to ask questions before signing the ICF.

[0178] The research moderator read a research script that provided an overview of the study and descriptions of various research activities. The research moderator then collected baseline (background) information from the participants.

[0179] Next, the research moderator had participants perform the following tasks in each research session: Laugh broadly and show as many teeth as possible. 1-minute break Wrinkle your forehead as tightly as possible. 1-minute break Close your eyes as tightly as you can.

[0180] 1-minute break Puff out your cheeks as much as possible. 1-minute break Suck your cheeks in as much as possible. 1-minute break Chew for 30 seconds. 1-minute break swallow 1-minute break Close your eyes normally for 5 seconds. 1-minute break Have a 30-second conversation 1-minute break Stare upwards for 45 seconds. 1-minute break Stare to your left for 45 seconds. 1-minute break Stare to your right for 45 seconds. 1-minute break Open and close your jaw as much as possible. 1-minute break He made a surprised face. 1-minute break Make a sad face 1-minute break He made an angry face. The label annotations disclosed herein correspond to the following tasks, as shown in Table 7.

[0181] [Table 7] Raw biometric sensor device data was collected sequentially during each activity in this exemplary study. To ensure reliable ground truth data annotation, data from each activity was manually labeled by a skilled technician. For each activity, the manifestation and disappearance metrics for each activity performed were annotated as appropriate. Time-synchronized video recordings of participants were used as a reference source in this annotation procedure. These activity annotations were then used to segment the signals according to the annotated manifestation and disappearance timestamps. Raw data collection may also be performed automatically by using sensors that transmit the raw data to one or more receivers or controllers (e.g., as shown in Figures 1-3) according to the methods disclosed herein.

[0182] After the activity was completed, the signals obtained from each channel were scaled to counteract the effects of amplification performed in the device hardware for noise suppression purposes, and filtered offline using a second-order infinite impulse response (IIR) notch filter to remove 60 Hz power line noise. Each signal contained a mixture of EEG, EMG, and EOG data (e.g., mixed signal waveform 4502). A signal separation algorithm was applied to better separate each component (e.g., by signal separation module 4504), resulting in a total of six channels in this example (two each for EEG, EMG, and EEG).

[0183] Following signal scaling, filtering, and separation, the signal from each of the six separated channels was segmented based on the presence or absence of facial motion activity, as shown in Figure 45. For further downstream analysis (e.g., 4510 in Figure 45), a comprehensive approach to feature extraction was taken. General features of each waveform were summarized, apart from a subset of features specific to EMG, EOG, or EEG activity. Features that clearly identify simulated PerfO activity performed within the data acquisition process but do not generalize to the performance of activities outside the laboratory context were omitted (e.g., the duration of the activity each participant was instructed to perform over a specified period).

[0184] Event-based segmentation algorithms 4508 and feature computations 4510 were performed. Statistical measures were computed from each isolated signal segment to summarize the signal behavior in the time domain (see, e.g., Figure 46). Such measurements allow for the depiction of information such as time-varying amplitude behavior, amplitude distribution, and signal trends observable in raw form. Since the frequency and time-frequency domains also contain a vast amount of information within the biosignal data, digital signal processing (DSP) analysis was performed to decompose each isolated signal segment into frequency components and evaluate patterns within this alternative representation, as shown in Figure 45. To better represent such activity in the summarized feature vectors, features related to theoretical EMG, EOG, and EEG behavior during specific simulated PerfO activities were computed.

[0185] As described herein, the steps described above (also shown in Figure 45) yielded a 161-dimensional feature vector representation for each simulated PerfO activity performed, as shown in Tables 2-1 to 2-3. These features correspond to the extracted feature 40 in Figure 1A. Feature reduction using the Volta algorithm was implemented to remove features potentially irrelevant to the activity-based classification for a given clinical outcome studied. As shown in Figure 52, the total of 161 features were trimmed to obtain a lower-dimensional feature vector representation for each simulated PerfO activity. As illustrated, 60 features estimated to be "unimportant" were removed from each feature vector, resulting in a 101-dimensional feature vector. A Python implementation of the Volta package (BorutaPy, version 0.3) was used to perform the feature reduction.

[0186] Correlations were observed between biometric sensor device parameters and the parameter differences between activities. Spearman correlations between all parameters and all activities were calculated, as shown in Figure 48. Silhouette technique was used to determine the optimal number of clusters in the factoextra package in R with the function fviz_nbclust, which has 100 bootstrap samples. For each of the 16 activities, the number of tasks analyzed (n), minimum (min), maximum (max), median, mean, mean standard deviation (sd), and mean standard error (se) are shown in Table 4 for all 161 calculated b-parameters.

[0187] The relationship between biometric sensor device features and activities, or between biometric sensor device features and demographic information, was determined. For data from exemplary studies, the ICC for participants as a group was calculated using linear mixed-effects modeling with the lmer package in R (per participant) in the following way: The ICC was calculated separately for each of the 161 biometric sensor device features and for each of the 16 activities, according to Table 4. The coefficient of variation was also calculated by comparing within each activity, according to Table 4.

[0188] Intra-trial and inter-trial variability attributable to repeated measures, time, and participant was calculated. Variance not explained by these three factors was also calculated according to Table 4. A nested linear mixed-effects model was used to derive the variability explained by each component: approximately 1 + (1 time) + (1 participant) + (1 repeat / time), where the time component represents the time (morning or evening), the participant component represents the subject, and the repeat component represents the nested repetitions of the same activity within the same time period.

[0189] As shown in Figures 25, 26, 27, 40, 41, 42, and 49, dimensionality reduction of biometric device features was performed in Python using umap-learn or one or more applicable dimensionality reduction methods. The reduction was performed using the minimum effective distance between embedding points and default parameters. For example, as shown in Figure 49, UMAP coordinates were plotted in R using ggplot2. As shown in Figure 50 (and Figures 50-1, 50-2), heatmaps of biometric sensor device features are displayed with individual activities as columns and biometric sensor device features as rows. Heatmaps of biometric sensor device data (e.g., Figures 8–12, 22, 24, 31–36, 38, 39, 48, 50, and 53) show Z-scored feature rows calculated across all activities. The heatmaps were constructed using the ComplexHeatmap package in R.

[0190] Research activity and participant level predictions were quantified. A multi-class classification model was implemented using the Python sklearn module to determine how biometric sensor device features could be used to classify each of the 16 activities. A random forest classifier with 500 decision trees (e.g., using the classification of sklearnRandomForestClassifier) ​​was implemented for model construction. In each classification setting, the model was trained and validated using 80% of the dataset, and the remaining 20% ​​of the dataset was reserved for testing. Data samples were randomly assigned to one of two subsets to reduce bias in the evaluation results. As shown in Figure 52, the F1 score was calculated to evaluate the model performance on the test set. The F1 score is the harmonic mean of precision and recall, and elucidates the number of predictions that the model was accurate, balancing both false negatives and false positives.

[0191] We determined a CNN model for activity level prediction. Deep learning models have been used to achieve high performance in many tasks related to the classification of biosignal data. Among the many popular deep learning architectures that are utilized in such tasks, CNNs are widely used due to their ability to learn patterns in structured multidimensional data (e.g., time-frequency signal representations). When applying such methodologies to the task of simulated PerfO activity level classification, a 16-class CNN classification model was developed and analyzed. These CNN models were constructed to map a 2D spectrogram representation of a simulated PerfO activity signal segment to a probability distribution across 16 classes.

[0192] Because deep learning models often require large datasets to learn generalizable functions, data augmentation was used to maximize diversity in the training set. Each time a signal segment was loaded into the training dataset, multiple random croppings of this segment were also added to the training set. To some extent, this allowed for increasing the size of the training dataset without collecting additional samples and helped combat overfitting. To maintain a constant input signal length between simulated PerfO activities with varying durations, activity segments with durations shorter than a fixed input data duration (e.g., 30 seconds) were repeated after shifting the segment according to a randomized cropping scheme, while longer segments were truncated to the fixed input data duration via randomized cropping. Data augmentation was not performed on the test set because it biases the resulting model performance estimates. Additional methods applied to reduce model variance included the use of L2 kernel regularization in convolutional and fully connected model layers, as well as the inclusion of dropout layers throughout the network. Following development and evaluation on the training and validation datasets, a shallow CNN, as shown in Figure 51, was trained and used for test purposes.

[0193] Data from this exemplary study suggest that tested biometric sensor devices, as well as similar wearable devices, may be used for the objective quantification of cranial and ocular muscle movements. Techniques disclosed herein (e.g., for identifying clinically relevant features 50) may be used to identify the capabilities and limitations of a given device based on clinical outcomes. Methods disclosed herein may be used to test the usefulness of wearable devices in disease populations, to more accurately measure disease progression within participants, to test how wearable device features or data relate to existing PROs, and / or to more accurately measure therapeutic effects within disease populations. The use of biometric sensor devices in long-term studies where disease progression can be measured, e.g., ongoing natural history studies, may help to elucidate which features are most important for quantifying disease effects. Exploratory use of these devices in clinical trials as part of a wearable clinical development strategy may enable more sensitive detection of therapeutic responses within disease populations. These clinical validation steps may further support strategies for using devices such as biometric sensor devices tested for passive monitoring purposes. Such monitoring may be carried out by acquiring signals from a signal acquisition device 10, identifying clinically relevant features 50 based on the data collected by the signal acquisition device 10, and / or using the clinically relevant features 50 to continuously (e.g., sequentially) provide clinical outcomes (e.g., identification of a disease or disorder and / or a treatment plan based thereon).

[0194] One or more implementations disclosed herein include a machine learning model. The machine learning models disclosed herein may be trained using the data flow 5410 in Figure 54. As shown in Figure 54, the training data 5412 may include one or more of the stage inputs 5414 and known results 5418 related to the machine learning model being trained. The stage inputs 5414 may be from any applicable source, including data inputs or outputs from the components, steps, or modules shown in Figures 1A, 1B, 2, 3, 4A, and / or 4B. The known results 5418 may be included for machine learning models generated based on supervised or semi-supervised training. Unsupervised machine learning models do not have to be trained using the known results 5418. The known results 5418 may include known or desired outputs for future inputs similar to or within the same category as the stage inputs 5414 that do not have corresponding known outputs.

[0195] The training data 5412 and the training algorithm 5420 may be provided to a training component 5430 which can apply the training data 5412 to the training algorithm 5420 to generate a machine learning model. According to one implementation, the training component 5430 may be given a comparison result 5416 which compares the previous output of the corresponding machine learning model to apply the previous result to retrain the machine learning model. The comparison result 5416 may be used by the training component 5430 to update the corresponding machine learning model. The training algorithm 5420 may utilize machine learning networks and / or models, including but not limited to deep learning networks such as deep neural networks (DNNs), convolutional neural networks (CNNs), fully convolutional networks (FCNs), and recurrent neural networks (RCNs), probabilistic models such as Bayesian networks and graphical models, and / or discriminative models such as decision forests and maximum margin methods.

[0196] Figure 55 is a simplified functional block diagram of a computer system 5500 that may be configured as a device for performing the methods disclosed herein, according to an exemplary embodiment of the present disclosure. Figure 55 is a simplified functional block diagram of a computer system that may generate features, statistics, analysis, and / or other systems, according to an exemplary embodiment of the present disclosure. In various embodiments, any of the systems disclosed herein (e.g., computer system 5500) may be an assembly of hardware including, for example, a data communication interface 5520 for packet data communication. The computer system 5500 may also include a central processing unit ("CPU") 5502 in the form of one or more processors for executing program instructions 5524. The computer system 5500 may include an internal communication bus 5508 and a storage unit 5506 (such as ROM, HDD, SSD) that can store data on a computer-readable medium 5522, but the computer system 5500 may receive programming and data via network communication (e.g., on network 110). Furthermore, the computer system 5500 may have memory 5504 (such as RAM) for storing instructions 5524 for performing the methods presented herein, although the instructions 5524 may be temporarily or permanently stored in other modules of the computer system 5500 (e.g., a processor 5502 and / or computer-readable media 5522). The computer system 5500 may also include input / output ports 5512 and / or a display 5510 for connecting to input / output devices such as a keyboard, mouse, touchscreen, monitor, and display. Various system functions may be implemented in a distributed manner on several similar platforms to distribute the processing load. Alternatively, the system may be implemented by appropriate programming on a single computer hardware platform.

[0197] The programmatic aspects of this technology can typically be considered as “products” or “manufactured goods” in the form of executable code and / or associated data carried on or embodied within a certain type of machine-readable medium. “Storage” type mediums include any or all of tangible memory such as computers, processors, or their associated modules, such as various semiconductor memories, tape drives, and disk drives, which can provide non-temporary storage for software programming at any time. All or part of the software may be communicated, in some cases, via the Internet or various other telecommunication networks. Such communication can enable, for example, the loading of software from one computer or processor to another, for example, from a management server or host computer on a mobile communication network to a server computer platform, and / or from a server to a mobile device. Therefore, other types of mediums that can carry software elements include light, electricity, and electromagnetic waves, used across physical interfaces between local devices, through wired and optical terrestrial communication network networks, and via various air links. Physical elements that carry such waves, such as wired or wireless links and optical links, can also be considered mediums that carry software. As used herein, unless limited to non-temporary tangible “storage” media, terms such as computer or machine “readable media” refer to any medium involved in providing instructions to a processor for execution.

[0198] While the methods, devices, and systems of this disclosure are described by illustrative reference to data transmission, embodiments of this disclosure may be applicable to any environment, such as desktop or laptop computers, mobile devices, wearable devices, and applications. Furthermore, embodiments disclosed herein may be applicable to any type of Internet protocol.

[0199] It will be apparent to those skilled in the art that various modifications and variations can be made to the devices and methods disclosed herein without departing from the scope of this disclosure. Furthermore, other aspects of this disclosure will be apparent to those skilled in the art from the discussion herein and the implementation of the features disclosed herein. This specification and the examples are intended to be illustrative only.

[0200] Aspects of this disclosure relate to signal-based feature analysis. In one aspect, the disclosure relates to a method comprising receiving individual electrical signals generated based on a body part, generating a plurality of extracted features based on the individual electrical signals, and identifying clinically relevant features from the plurality of extracted features, wherein the clinically relevant features satisfy a threshold determined based on a clinical outcome.

[0201] The method may also include determining a clinical outcome by applying the clinically relevant features, which may be a diagnosis or a treatment plan. The individual electrical signals may be generated based on bodily electrical signals generated by the body part. The individual electrical signals may be generated based on the movement of the body part. The individual electrical signals may be generated based on the characteristics of the body part. The multiple extracted features may be based on one or more of the following: amplitude features, zero crossing rate, standard deviation, variance, root mean square, kurtosis, frequency, bandwidth power, or skew. The individual electrical signals may be generated by a wearable device equipped with sensors, in which case the wearable device may be configured to output a mixed signal, and / or a signal separation module extracts the extracted features from the mixed signal.

[0202] For example, the signal separation module can extract features from a mixed signal by applying one or more of the following: blind signal separation, blind source separation, discrete transform, Fourier transform, integral transform, two-sided Laplace transform, Mellin transform, Hartley transform, short-time Fourier transform (or short-term Fourier transform) (STFT), rectangular mask short-time Fourier transform, charplet transform, fractional Fourier transform (FRFT), Hankel transform, Fourier-Bross-Iagornitzer transform, or linear canonical transform. A random forest algorithm may be used to score the extracted features. The threshold may be a random forest threshold, and extracted features having a random forest score equal to or greater than the random forest threshold may be identified as clinically relevant features. The threshold may be a reliability threshold, and extracted features having a reliability score equal to or greater than the reliability threshold may be identified as clinically relevant features. The reliability score may be based on one or more of the following: Spearman correlation, intraclass correlation (ICC), covariance (CV), area under the curve (AUC), clustering, or Z score.

[0203] In another embodiment, the disclosure relates to a system comprising a wearable device including a plurality of sensors, a processor, and a computer-readable data storage device storing instructions. When the processor executes the instructions, the system performs the following actions: obtain electrical activity information of a subject detected by the plurality of sensors from the wearable device, and identify clinically relevant features based on the electrical activity information.

[0204] The system may further be configured to classify the clinically relevant features as one or more diseases, determine the subject's disease based on the one or more diseases, determine the extent of the disease, and / or determine a treatment plan based on the extent of the disease. The plurality of sensors may include electroencephalography (EEG) sensors, electrooculography (EOG) sensors, electromyography (EMG) sensors, image sensors, and / or eye-tracking sensors. The clinically relevant features may be identified using machine learning algorithms. The technical concept behind this disclosure is described below. (Note 1) Receiving individual electrical signals generated based on body parts, To generate multiple extraction features based on the aforementioned individual electrical signals, Identifying clinically relevant features from the aforementioned multiple extracted features, wherein the clinically relevant features satisfy a threshold determined based on clinical outcomes, A method for providing this. (Note 2) The method according to Appendix 1, further comprising determining the clinical outcome by applying the aforementioned clinically relevant characteristics. (Note 3) The method described in Appendix 2, wherein the aforementioned clinical outcome is one of the diagnoses or treatment plans. (Note 4) The method according to Appendix 1, wherein the individual electrical signals are generated based on the bodily electrical signals generated by the bodily parts. (Note 5) The method according to Appendix 1, wherein the individual electrical signals are generated based on the movement of the body parts. (Note 6) The method according to Appendix 1, wherein the individual electrical signals are generated based on the characteristics of the body parts. (Note 7) The method according to Appendix 1, wherein the plurality of extracted features are based on one or more of the following: amplitude features, zero crossover rate, standard deviation, variance, root mean square, kurtosis, frequency, bandwidth power, or skew. (Note 8) The method according to Appendix 1, wherein the individual electrical signals are generated by a wearable device equipped with a sensor. (Note 9) The method according to Appendix 8, wherein the wearable device is configured to output a mixed signal. (Note 10) The method according to Appendix 9, wherein a signal separation module extracts the extracted features from the mixed signal. (Note 11) The method according to Appendix 10, wherein the signal separation module extracts the extracted features from the mixed signal by applying one or more of the following: blind signal separation, blind signal source separation, discrete transform, Fourier transform, integral transform, two-sided Laplace transform, Mellin transform, Hartley transform, short-time Fourier transform (or short-time Fourier transform) (STFT), rectangular mask short-time Fourier transform, charplet transform, fractional Fourier transform (FRFT), Hankel transform, Fourier-Bross-Iagornitzer transform, or linear canonical transform. (Note 12) The method according to Appendix 1, wherein a random forest algorithm is used to score the extracted features. (Note 13) The method according to Appendix 12, wherein the threshold is a random forest threshold, and extracted features having a random forest score greater than or equal to the random forest threshold are identified as clinically relevant features. (Note 14) The method according to Appendix 1, wherein the threshold is a reliability threshold, and extracted features having a reliability score equal to or greater than the reliability threshold are identified as clinically relevant features. (Note 15) The method described in Appendix 14, wherein the reliability score is based on one or more of the following: Spearman correlation, intraclass correlation (ICC), covariance (CV), area under the curve (AUC), clustering, or Z score. (Note 16) It is a system, A wearable device containing multiple sensors, Processor and A computer-readable data storage device that stores instructions, wherein when the instructions are executed by the processor, The electrical activity information of the subject detected by the multiple sensors is acquired from the wearable device, A computer-readable data storage device that stores the instructions for the system to perform the following actions: identifying clinically relevant features based on the electrical activity information; A system equipped with these features. (Note 17) The system described in Appendix 16 is further configured to classify the aforementioned clinically relevant features as one or more diseases. (Note 18) The system according to Appendix 17, further configured to determine the disease of the subject based on the one or more diseases. (Note 19) The aforementioned system, Determine the extent of the aforementioned disease, A treatment plan is determined based on the scope of the aforementioned disease. The system described in Appendix 18 is further configured as follows. (Note 20) The system as described in Appendix 16, wherein the plurality of sensors include an electroencephalogram (EEG) sensor. (Note 21) The system as described in Appendix 16, wherein the plurality of sensors include an electrooculography (EOG) sensor. (Note 22) The system according to Appendix 16, wherein the plurality of sensors include an electromyography (EMG) sensor. (Note 23) The system according to Appendix 16, wherein the plurality of sensors include an image sensor. (Note 24) The system according to Appendix 16, wherein the plurality of sensors include an eye-tracking sensor. (Note 25) The system described in Appendix 16, wherein the aforementioned clinically relevant features are identified using a machine learning algorithm.

Claims

1. A method for identifying relevant features from detected mixed electrical signals and outputting a treatment plan, A training mixed electrical signal generated based on the detection of a body part, wherein the training mixed electrical signal having a first waveform is received by at least one of one or more processors, By performing signal filtering for signal separation on the training mixed electrical signal using a signal separation module comprising at least one of the one or more processors, a plurality of individual signals, each having a waveform different from the first waveform, are generated. The presence or absence of an event for each of the plurality of individual signals is determined by at least one of the one or more processors, thereby generating segmented individual signals. Multiple training extraction features are generated by at least one of the one or more processors based on determining the signal behavior in the time domain and frequency domain for the segmented individual signals. Based on the outcome determined from the generated training mixed electrical signal, at least one of the one or more processors automatically identifies a statistical filter to be applied to the multiple training extraction features from among multiple statistical filter processes. The process involves using the identified statistical filtering to process the plurality of training extraction features with at least one of the one or more processors, By processing the plurality of training extract features using the identified statistical filtering process, at least one of the one or more processors identifies a training-related feature which is a subset of the plurality of training extract features, wherein the training-related feature that satisfies a threshold determined based on the outcome is identified. Receiving a treatment mixed electrical signal generated by detecting an individual body part associated with the outcome in a first computing device, wherein the individual's treatment-related features and other features of the individual are extractable from the treatment mixed electrical signal, and the individual's treatment-related features are associated with the identified training-related features; Extracting treatment-related features of an individual from the treatment mixed electrical signal using at least one of the one or more processors, without extracting other features of the individual. Transmitting the individual's treatment plan, based on the individual's treatment-related characteristics, to a second computing device by at least one of the one or more processors, A method for providing this.

2. The method according to claim 1, wherein each of the plurality of training extract features is associated with a respective cluster determined based on intraclass interphase (ICC).

3. The method according to claim 2, wherein each cluster of the aforementioned training-related features satisfies the ICC significance threshold for the outcome.

4. The method according to claim 1, wherein the mixed electrical signals for training are generated based on one or more of the following: electroencephalography (EEG) sensors, electrooculography (EOG) sensors, electromyography (EMG) sensors, face recognition sensors, visual tracking sensors, image sensors, video sensors, infrared sensors, thermal sensors, and vibration sensors.

5. The method according to claim 1, wherein the plurality of training extraction features are based on one or more of the following: amplitude features, zero crossover rate, standard deviation, variance, root mean square, kurtosis, frequency, bandwidth power, or skew.

6. The method according to claim 1, wherein the mixed electrical signal for training is generated by a sensor placed in a wearable device.

7. The method according to claim 1, wherein the signal separation for extracting the plurality of training extractable features from the training mixed electrical signal includes one or more of the following: blind signal separation, blind source separation, discrete transform, Fourier transform, integral transform, two-sided Laplace transform, Mellin transform, Hartley transform, short-time Fourier transform (or short-time Fourier transform) (STFT), rectangular mask short-time Fourier transform, charplet transform, fractional Fourier transform (FRFT), Hankel transform, Fourier-Bross-Iagornitzer transform, or linear canonical transform.

8. The method according to claim 1, wherein a random forest algorithm is used to score the plurality of training features.

9. The method according to claim 8, wherein the threshold is a random forest threshold, and the plurality of training extraction features having a random forest score equal to or greater than the random forest threshold are identified as the treatment-related features.

10. The method according to claim 1, wherein the threshold is a reliability threshold, and the plurality of training extraction features having a reliability score equal to or greater than the reliability threshold are identified as the treatment-related features.

11. The method according to claim 10, wherein the reliability score is based on one or more of the following: Spearman correlation, intraclass correlation (ICC), covariance (CV), area under the curve (AUC), clustering, or Z score.

12. The method according to claim 1, wherein the treatment mixed electrical signal is generated while the subject is making facial movements, and the treatment plan for treating a neuromuscular disorder identified based on the facial movements is determined by at least one of the one or more processors.

13. The Z-scores for the plurality of training extraction features are determined by at least one of the one or more processors, A heatmap of the Z-scores arranged based on the Z-scores is displayed on a display by at least one of the one or more processors. The method according to claim 1, further comprising:

14. The method according to claim 1, wherein the signal filtering for signal separation includes blind signal separation, blind source separation, discrete transform, Fourier transform, integral transform, two-sided Laplace transform, Mellin transform, Hartley transform, short-time Fourier transform (or short-time Fourier transform) (STFT), rectangular mask short-time Fourier transform, charplet transform, fractional Fourier transform (FRFT), Hankel transform, Fourier-Bross-Iagornitzer transform, or linear canonical transform.

15. A system for identifying relevant features from detected mixed electrical signals and outputting a treatment plan, Wearable devices including sensors, Processor and A computer-readable data storage device that stores instructions, wherein when the instructions are executed by the processor, A training mixed electrical signal generated based on the detection of a body part, receiving a training mixed electrical signal having a first waveform, By performing signal filtering for signal separation on the aforementioned training mixed electrical signal using a signal separation module, a plurality of individual signals, each having a waveform different from the first waveform, are generated. By determining the presence or absence of an event for each of the aforementioned multiple individual signals, segmented individual signals are generated. Multiple training extraction features are generated based on the determination of the signal behavior in the time domain and frequency domain for the segmented individual signals. Based on the outcome determined from the generated training mixed electrical signal, the system automatically identifies a statistical filter to be applied to the multiple training extraction features from among multiple statistical filter processes. Processing the multiple training features using the identified statistical filtering process, By processing the plurality of training extract features using the identified statistical filtering process, the training related features, which are a subset of the plurality of training extract features, are identified, and the training related features that satisfy a threshold determined based on the outcome are identified. Receiving a treatment mixed electrical signal generated based on the detection of an individual body part associated with the aforementioned outcome, wherein the treatment-related features of the individual and other features of the individual are extractable from the treatment mixed electrical signal, and the treatment-related features of the individual are associated with the identified training-related features; Extracting treatment-related features of the individual from the treatment-mixed electrical signal without extracting other features of the individual, Transmitting the individual's treatment plan, based on the individual's treatment-related characteristics, to a computing device, A computer-readable data storage device that stores the instruction to cause the system to execute the instruction, A system equipped with these features.

16. The system according to claim 15, further configured to determine the disease of a subject based on the aforementioned treatment-related characteristics.

17. The aforementioned system, Determine the state of the aforementioned illness, A treatment plan is determined based on the aforementioned disease state. The system according to claim 16, further configured as follows.

18. The system according to claim 15, wherein the sensor is at least one of an electroencephalogram (EEG) sensor, an electrooculogram (EOG) sensor, an electromyogram (EMG) sensor, an image sensor, or an eye-tracking sensor.

19. A non-temporary computer-readable medium configured to store processor-readable instructions, wherein when the instructions are executed by the processor, A training mixed electrical signal generated based on the detection of a body part, receiving a training mixed electrical signal having a first waveform, By performing signal filtering for signal separation on the aforementioned training mixed electrical signal using a signal separation module, a plurality of individual signals, each having a waveform different from the first waveform, are generated. By determining the presence or absence of an event for each of the aforementioned multiple individual signals, segmented individual signals are generated. Multiple training extraction features are generated based on the determination of the signal behavior in the time domain and frequency domain for the segmented individual signals. Based on the outcome determined from the generated training mixed electrical signal, the system automatically identifies a statistical filter to be applied to the multiple training extraction features from among multiple statistical filter processes. Processing the multiple training features using the identified statistical filtering process, By processing the plurality of training extract features using the identified statistical filtering process, the training related features, which are a subset of the plurality of training extract features, are identified, and the training related features that satisfy a threshold determined based on the outcome are identified. Receiving a treatment mixed electrical signal generated by detecting an individual body part associated with the outcome in a first computing device, wherein the individual's treatment-related features and other features of the individual are extractable from the treatment mixed electrical signal, and the individual's treatment-related features are associated with the identified training-related features; Extracting treatment-related features of the individual from the treatment-mixed electrical signal without extracting other features of the individual, Transmitting the treatment plan of the individual, based on the individual's treatment-related characteristics, to a second computing device, A non-temporary computer-readable medium configured to store the instructions that perform the operations including the operation.

20. The training mixed electrical signals are generated by sensors associated with a wearable device in the non-temporary computer-readable medium according to claim 19.

Images