A Machine Learning Based Framework Using Electroretinogram for Detecting Early Stage Glaucoma

Abstract

A method for diagnosing glaucoma using a machine learning method includes determining a labeled training dataset. The labeled training dataset includes measured electroretinogram (ERG) signals from a group of subjects. The ERG signals are labeled as either glaucoma or non-glaucomatous based on the subject from which each of the ERG signals was measured. The training dataset is used to train a machine learning model, such as a decision tree model, a discriminant model, a support vector machine, a nearest neighbor algorithm, or an ensemble classifier. The resulting trained machine learning model is configured to classify an ERG signal input as glaucoma or non-glaucomatous. The model can be utilized by measuring ERGs from the subjects and inputting the measured ERGs into the trained machine learning model. The subjects can be diagnosed with glaucoma based on the output classification of glaucoma.

Description

[Technical field]

[0001] The present application relates to a machine learning based framework using electroretinograms for detecting early stage glaucoma. [Background technology]

[0002] Glaucoma is a chronic neurodegenerative disease affecting the retina and optic nerve, a leading cause of blindness and characterized by gradual and irreversible vision loss. Currently, available treatment paradigms focus primarily on a predisposing factor, high intraocular pressure ("IOP"), that does not allow for repair of the retina and optic nerve once the disease has progressed and damage has occurred.

[0003] Early stage glaucoma can be difficult to detect and diagnose. Current technology for diagnosing glaucoma includes a variety of techniques. These techniques include tonometry, which involves an instrument that measures the pressure inside the eye; ophthalmoscopy, which involves an instrument that identifies the shape and color of the optic nerve; perimetry, in which a system measures a subject's ability to see objects clearly at several points in the subject's visual field; gonioscopy, in which a lens and a lamp are used to distinguish between open-angle and closed-angle glaucoma; and pachymetry, which involves measuring the thickness of the cornea via a pachymeter. Summary of the Invention

[0004] Presented here is a framework that uses techniques / techniques to extract relevant features from electroretinogram ("ERG") counts. The framework uses the relevant features to train machine learning models to diagnose early stage glaucoma, differentiate between various stages of glaucoma progression, and provide a quantitative assessment of visual functionality by predicting retinal ganglion ("RGC") signals. This framework allows for detection of early / various stages of glaucoma impairment with high accuracy while also providing a more detailed understanding of glaucoma progression and the impact of various therapeutic interventions on this progression.

[0005] The embodiments of the present disclosure are directed to providing a mechanism, including a computer-implemented method and a non-transitory computer storage medium, for detecting early stage glaucoma using ERG signals in a machine learning-based framework. In some embodiments, the computer-implemented method collects ERG signals, which are measurements of electrical responses of cells in the retina. As the ERG signals are raw data, the method pre-processes the ERG signals by removing anomalies of the ERG signals and performing baseline adjustment. The techniques described herein extract statistical features and advanced wavelet-based features from the pre-processed ERG signals. Using feature extraction techniques, relevant features are selected from the statistical features and advanced wavelet-based features to create a training data set. The training data set can then be used to train a machine learning model to make glaucoma-based predictions. These predictions include glaucoma diagnosis predictions, which distinguish various stages of glaucoma progression and provide retinal ganglion cell ("RGC") count predictions that determine a quantitative assessment of the visual functionality of the retina.

[0006] At a high level, the present technology concerns using machine learning for better detection of early stage glaucoma. A labeled training data set is determined by measuring ERG signals from a group of subjects and labeling the measured ERG signals as either glaucomatous or non-glaucomatous based on the subject from which each of the ERG signals was measured. The training data set is used to train a machine learning model, such as a decision tree model, a discriminant model, a support vector machine, a nearest neighbor algorithm, or an ensemble classifier. The resulting trained machine learning model is configured to classify ERG signal inputs as glaucomatous or non-glaucomatous.

[0007] The trained machine learning model can be utilized by measuring an ERG signal from a subject and inputting the measured ERG signal into the trained machine learning model. The subject can be diagnosed with glaucoma based on an output classification of glaucoma by the trained machine learning model.

[0008] This summary is intended to introduce a selection of concepts in a simplified form that are further described in the Detailed Description section of this disclosure. This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended as an aid in determining the scope of the claimed subject matter. Additional objects, advantages, and novel features of the present technology will be set forth in part in the description that follows, and in part will become apparent to those skilled in the art upon examination of the disclosure or may be learned through the practice of the technology. [Brief description of the drawings]

[0009] These and other features, aspects, and advantages of the embodiments of the present disclosure will become better understood with regard to the following description, appended claims, and accompanying drawings.

[0010] [Figure 1] FIG. 1 is a diagram of an architecture for predicting early stage glaucoma using machine learning, according to an embodiment of the present disclosure. [Diagram 2] FIG. 1 is a diagram of a machine learning framework for predicting early stage glaucoma using ERG signals, according to an embodiment of the present disclosure. [Diagram 3] FIG. 1 illustrates an exemplary diagnostics system for providing a glaucoma diagnosis, according to an embodiment of the present disclosure. [Figure 4] 1 is a schematic diagram of a glaucoma diagnosis system according to an embodiment of the present disclosure. [Diagram 5] 1 is a flow chart for extracting relevant features from ERG signals for training machine learning models in glaucoma diagnosis according to an embodiment of the present disclosure. [Figure 6] 1 is a flow chart for training a machine learning model to provide a glaucoma diagnosis of ERG signals according to an embodiment of the present disclosure. [Figure 7] 1 is a flow chart for providing a glaucoma diagnosis through a diagnostics system according to an embodiment of the present disclosure. [Figure 8] FIG. 1 is a schematic diagram of an exemplary environment in which a glaucoma diagnosis system and machine learning framework can operate, according to an embodiment of the present disclosure. [Figure 9] FIG. 2 is a block diagram of an exemplary computing device according to embodiments described herein.

[0011] While the present disclosure is amenable to various modifications and alternative forms, details of the disclosure have been shown by way of example in the drawings and will be described in detail. It should be understood, however, that the intention is not to limit the particular embodiments described. Rather, the intention is to cover all modifications, equivalents, and alternatives falling within the scope of the present disclosure. Like reference numerals are used to designate like parts in the accompanying drawings. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] The present disclosure relates to the detection of early stage glaucoma, and more particularly to the detection of early stage glaucoma using electroretinogram signals in a machine learning based framework. Although the present disclosure is not necessarily limited to such applications, various aspects of the present disclosure may be appreciated through a consideration of various examples using this context.

[0013] Overview Glaucoma is a leading cause of blindness and a complex disease that damages the optic nerve resulting in gradual and irreversible loss of vision. Early diagnosis of glaucoma can significantly aid treatment, resulting in better treatment outcomes and less vision loss. However, existing diagnostic techniques focus on either surrogate markers of disease progression in subjects (intraocular pressure ("IOP")) or structural measures. Structural measures are only reliable in the later stages of the disease, when damage causing vision loss has already occurred and become irreversible (retinal thickness measured with optical coherence tomography ("OCT"). Existing techniques do not currently use electroretinograms ("ERG") to diagnose glaucoma, as current data analysis methods do not measure parameters related to glaucoma and glaucoma disease progression. Machine learning methods have great potential in automating glaucoma diagnosis and have become an active area of ​​research focused on analyzing OCT images.

[0014] Current techniques for glaucoma diagnosis are based on psychophysical and structural techniques. The former includes the Humphrey visual field technique, which uses a machine to assess a subject's response to light stimuli across a large portion of their visual field, and remains the most commonly utilized technique for glaucoma diagnosis and monitoring the progression of glaucoma disease and the effectiveness of therapeutic interventions. The latter method of glaucoma diagnosis involves instrumental measurements of retinal structure, particularly the thickness of the affected retinal layer (e.g., the retinal nerve fiber layer ("RNFL")) and morphological changes to the portion of the retina that connects to the optic nerve (optic nerve head). The field has evolved to include automated retinal image analysis ("ARIA") systems that contribute to diagnosing complex diseases such as diabetic retinopathy and glaucoma. The development of such ARIA systems, involving machine learning methods trained on optical coherence tomography ("OCT") imaging data, has led to high analytical accuracy in automatically classifying late-stage glaucoma disease phenotypes based on structural characteristics.

[0015] However, the limitations of glaucoma detection remain in these existing techniques, so that early detection of glaucoma remains difficult, taking into account the significant limitations of structural methods. For example, methods using OCT data are not suitable for diagnosing early stage glaucoma, due to the diagnosis being fundamentally dependent on structural cell damage occurring at a later stage of the disease. Furthermore, techniques that utilize the analysis of structural changes for glaucoma diagnosis are based on measuring RNFL thickness in OCT images of the retina. This analysis technique may vary greatly and correlate weakly with RGC counts, even though RNFL thickness is a surrogate marker of RGC degeneration and optic nerve fiber loss (i.e., a hallmark of glaucoma development).

[0016] Moreover, RGC loss often occurs early during development without measurable RNFL thinning. Functional measures such as visual field and ERG tests provide more sensitive examination methods that can measure RGC loss earlier during development. Functional measures are sensitive to subtle changes in RGC function and RGC damage, suggesting a significant possibility for enabling early detection of glaucoma even in the absence of high IOP, as seen in normotensive glaucoma subjects. As a result, glaucoma intervention must be initiated before irreversible damage occurs, allowing optimization of treatment strategies based on improving RGC function.

[0017] ERG data are complex, multidimensional biomedical data relevant to diagnosing glaucoma, yet are not currently considered during routine clinical practice or in clinical research. This is due in part to multiple barriers associated with clinical ERG data collection, including limited reproducibility, high costs of both equipment and individual tests, long test duration, complex test administration resulting in reduced subject acceptance and compliance, and the need for highly trained professionals to administer the tests.

[0018] Newer diagnostics are often time-consuming and labor-intensive, and are focused on parameters developed to address a small subset of specific ocular genetic diseases that mainly affect pediatric subject populations. Furthermore, currently available methods are often not suitable for analyzing large data sets and databases, making currently available methods unable to utilize complex and rich data sets. As a result, early detection of glaucoma is not possible with currently available techniques at the early stages of glaucoma development, when cellular changes occur that have not yet caused structural damage or visual impairment.

[0019] Embodiments of the present disclosure improve upon existing techniques described herein (as well as other techniques) by providing a novel machine learning framework using ERG signal data for glaucoma detection, including early stage detection. Embodiments of the framework provide technical solutions by extracting and identifying relevant features (i.e., predictors) from ERG signals to train and develop machine learning models to diagnose glaucoma (binary classification), distinguish between different stages of glaucoma progression when glaucoma is detected (multi-class classification), and provide a quantitative assessment of visual function by predicting RGC counts from ERG signals.

[0020] The embodiments disclosed herein take advantage of changes in the activity of RGCs, the retinal neurons most affected by glaucoma, an early feature of glaucoma progression and an aspect of early onset that often occurs in the absence of measurable RNFL thinning. Although retinal function can be measured with ERG, analysis of subtle changes in RGC activity indicative of glaucoma cannot be measured with currently available commercial methods for ERG analysis, creating an urgent clinical need for more sensitive methods. ERG signals that are sensitive to subtle changes in RGC function due to glaucoma are reliably detected by the disclosed technology even in the absence of high IOP or retinal and RNFL thinning, allowing for early detection of glaucoma.

[0021] More specifically, the embodiments improve upon existing techniques by developing a predictive model for early stage glaucoma diagnosis based on a machine learning algorithm that utilizes relevant features from ERG signals as predictors. The glaucoma framework provides a novel technique for pre-processing ERG signal data, extracting relevant features of the ERG signal data, and training a machine learning model using the relevant features. The resulting machine learning model has the technical effect of achieving higher accuracy of early stage glaucoma diagnosis while providing a more detailed understanding of disease progression and also providing a better understanding of the impact of therapeutic interventions on existing techniques.

[0022] In some embodiments, the machine learning framework trains a glaucoma diagnostic machine learning ("ML") model using relevant features extracted from the ERG signal. The glaucoma diagnostic ML model is trained to provide a binary classification (i.e., classifying into glaucoma or non-glaucoma) based on the ERG signal used as input. In some implementations, the machine learning framework uses relevant statistical features in making its predictions. The relevant statistical features include cone correlation, mean and median flicker, high rods (Hi-Rods) and cone skewness, and cone standard deviation. Throughout various implementations, the glaucoma diagnostic ML model can be configured as a decision tree model, a discriminant classifier, a support vector machine ("SVM") model, an ensemble classifier model, or a naive Bayes model.

[0023] In some embodiments, the machine learning framework trains the glaucoma progression ML model using relevant features extracted from the ERG signal. The glaucoma progression ML model is trained to provide multi-class classification (i.e., classifying various progressions / stages of glaucoma) based on the ERG signal used as input. In some implementations, the glaucoma progression ML model uses statistical features in its multi-class prediction. The statistical features include cone correlation, number of troughs in Hi cones, kurtosis of scotopic threshold potential ("STR"), and mean flicker. In some implementations, the glaucoma progression ML model uses advanced wavelet-based features in making its prediction. These advanced wavelet-based features include rod wavelet variance and Shannon entropy value and AR coefficient for maximum overlap discrete wavelet packet transform ("MOD-PWT"). Throughout various implementations, the glaucoma progression ML model is configured as a decision tree model, a discriminant classifier, a support vector machine ("SVM") model, an ensemble classifier model, or a naive Bayes model.

[0024] In some embodiments, the glaucoma diagnosis framework trains an RGC count ML model using relevant features extracted from the ERG signal. The RGC count model is trained to provide a classification (i.e., classify the RGC count) based on the ERG signal used as an input. In some implementations, the RGC count model uses regression analysis (e.g., Gaussian process regression) to predict the RGC count from the ERG signal. In some other implementations, the machine learning model is an artificial neural network ("ANN"). In some embodiments, the machine learning framework uses mRmR sequential feature selection when determining feature selection for the regression analysis.

[0025] In some embodiments, the glaucoma diagnosis framework includes pre-processing the raw ERG signal. The raw ERG data may contain some anomalies, such as different start times of the signal recording, missing data in the signal recording, different sampling frequencies of the signal recording, noise in the signal recording, unequal length of the signal recording, etc. To take the anomalies into account, the glaucoma diagnosis framework provides pre-processing means, including performing baseline adjustment, feature extraction, missing data correction, outlier correction, feature scaling, feature selection, etc.

[0026] In some embodiments, the glaucoma diagnosis framework extracts statistical features from the ERG signal. The statistical features include, but are not limited to, measures of central tendency, spread, shape, peak, derivative, and correlation extracted from the ERG signal. These statistical features can describe the general behavior of the ERG signal.

[0027] In some embodiments, the glaucoma diagnosis framework extracts advanced wavelet-based features using an autoregressive model. An autoregressive model can describe a particular time-varying process and specifies that an output variable depends linearly on its previous values ​​and a stochastic term. In some implementations, the autoregressive model operates under the assumption that past values ​​affect current values.

[0028] In some embodiments, the glaucoma diagnosis framework extracts advanced wavelet-based features using Shannon entropy, which is an information-theoretic measure of the signal Shannon entropy value for MOD-PWT using a four-level wavelet decomposition, calculated on the end nodes of the wavelet.

[0029] In some embodiments, the glaucoma diagnosis framework extracts advanced wavelet-based features using multifractal wavelet leader estimates and cumulant scaling exponents. The wavelet leader is a time / space-localized suprema of the absolute values ​​of the discrete wavelet coefficients. These supremas are used to calculate the Hölder exponents that characterize the local regularity. In addition, a second cumulant of the scaling exponents is obtained. The scaling exponents are scale-dependent exponents that describe the power-law behavior of the signal at various resolutions.

[0030] The techniques described herein provide various technical improvements over conventional methods of glaucoma diagnosis. For example, embodiments providing mechanisms for binary classification of glaucoma diagnosis provide greater accuracy of diagnosis by utilizing relevant features extracted from ERG signals. These mechanisms allow a user to provide a subject's ERG signal to retrieve a glaucoma diagnosis with greater accuracy than conventional methods. Furthermore, these mechanisms also provide accurate diagnosis of early stage glaucoma, allowing treatment to occur before permanent damage to the retina occurs. Furthermore, embodiments describing techniques for extracting relevant features from ERG signals also enable training of machine learning models that provide predictions capable of distinguishing between various stages of glaucoma progression. The machine learning models also provide a quantitative assessment of visual function by predicting RGC counts.

[0031] Exemplary Machine Learning Frameworks FIG. 1 illustrates a diagram of a machine learning framework 100 implementing a glaucoma diagnostic system 105 configured to extract relevant features from ERG signals and train machine learning models in glaucoma detection and diagnosis according to an embodiment of the present disclosure. As discussed, the glaucoma diagnostic system 105 uses multiple machine learning models to diagnose glaucoma, including early stages of glaucoma, distinguish various stages of glaucoma progression, and provide a quantitative assessment of retinal visual functionality by predicting RGC counts from ERG signals. The machine learning predictions enable the glaucoma diagnostic system 105 to achieve higher accuracy over current conventional methods of glaucoma diagnosis using the novel techniques described herein and provide a more detailed understanding of disease progression. Thus, the glaucoma diagnostic system 105 uses the subject's ERG signal to provide a glaucoma diagnosis prediction once the machine learning models are trained using a training dataset along with relevant features extracted from the ERG signal. During application, if the presence of glaucoma is predicted, additional machine learning models can also predict the subject's current glaucoma progression and RGC counts.

[0032] The machine learning framework 100 and / or the glaucoma diagnosis system 105 can be implemented as a standalone application or as part of another application or suite of applications. For example, in some embodiments, the machine learning framework 100 and / or the glaucoma diagnosis system 105 are implemented as part of a diagnostics application to enable a user of the diagnostics application to diagnose and status of a subject's glaucoma. Alternatively, once a subject has received a glaucoma diagnosis, the glaucoma diagnosis system 105 can utilize the subject's ERG signal to provide the subject's stage of glaucoma progression and / or RGC counts.

[0033] The glaucoma diagnostic framework 100 includes an input ERG signal 102, a glaucoma diagnostic system 105, and an output 165. The glaucoma diagnostic system 105 includes a data pre-processor 110, an advanced feature extractor 120, a glaucoma diagnostic ML model 140, a glaucoma progression ML model 150, and an RGC count ML model 160. The output 165 includes a glaucoma diagnosis prediction 170, a glaucoma progression prediction 180, and an RGC count prediction 190. As shown, FIG. 1 provides an exemplary method that may be implemented by the computing device of FIG. 9 and is suitable for achieving the described advantages to detect early stage glaucoma. In an embodiment, one or more computer storage media having computer executable instructions embodied in the one or more computer storage media, which when executed by one or more processors, cause the one or more processors to perform the operations illustrated in FIG. 1.

[0034] As shown in FIG. 1, the glaucoma diagnostic system 105 receives an input ERG signal 102 at number 1. As discussed further below, the glaucoma diagnostic system 105 trains a number of machine learning models to provide a glaucoma diagnosis prediction, a glaucoma progression prediction, and an RGC count prediction using a number of ERG signals, converted to training data. The input signal 192 may include ERG signals measured from a subject and associated with a label. Each of the ERG signals can be associated with a glaucomatous or non-glaucomatous label. The subjects may include animal subjects, such as mammalian subjects, including mice, as well as human subjects.

[0035] The ERG signal includes measurements such as the ocular potential ("OP") and scotopic threshold potential ("STR"), which represent important ERG components indicative of RGC cell function. The OP is a small rhythmic wavelet superimposed on the rising b-wave of the ERG signal. The STR is a negative corneal deviation that occurs in a fully dark-adapted eye to a dim stimulus. The International Society for Clinical Electrophysiology of Vision ("ISCEV") has standardized an ERG protocol that includes several tests to measure the function of various retinal cell types. These tests include rod response, standard rod-cone response, high intensity rod, cone response, high intensity cone response, flicker, and Hi flicker.

[0036] In number 2, a data pre-processor 110 implements data pre-processing techniques such as baseline adjustment 111, feature extraction 112, missing data handling 113, outlier handling 114, feature scaling 115, and feature selection 116 on the input ERG signal 102. In number 3, the pre-processed ERG signal is provided to an advanced feature extractor 120 to extract statistical features 124 and advanced wavelet-based features 126. Unlike conventional techniques that utilize morphological and transitional characteristics of the ERG signal, embodiments utilize statistical and wavelet-based features to train a machine learning model in glaucoma diagnosis prediction. In particular, embodiments train a machine learning model to predict early stage glaucoma, where cellular changes occur that have not yet caused structural damage or visual impairment. As discussed, this includes training a glaucoma diagnosis ML model 140 to provide a binary classification that predicts whether glaucoma is present using relevant features (e.g., statistical and wavelet-based features) extracted from the ERG signal.

[0037] For example, the advanced feature extractor 120 includes statistical feature techniques 124 and advanced wavelet-based feature techniques 126 that extract features from the preprocessed ERG signals. As discussed further below, this can include statistical features such as measures of central tendency, spread, shape, peaks, derivatives, and correlation in number 4. This also includes advanced wavelet-based features such as autoregressive coefficients, Shannon entropy, and multifractal wavelet leader estimates in number 5.

[0038] In various embodiments, the resulting relevant features output by the advanced feature extractor 120 in the form of a training dataset are then provided to ML models 140, 150, 160, respectively, at numeral 6. The glaucoma diagnosis ML 140 model can be trained using ML training techniques such as supervised learning to provide a binary classification that predicts 170 a glaucoma diagnosis, at numeral 7. The glaucoma progression ML model 150 can be trained using ML techniques to provide a multi-class classification that predicts 180 glaucoma progression, at numeral 8. The RGC count ML model 160 can be trained using ML techniques to provide a classification that predicts 190 RGC count.

[0039] In various embodiments, the ML models described herein can be implemented as several types of machine learning models. These models include, but are not limited to, decision trees, discriminants, support vector machines, nearest neighbors, and ensemble classifiers. These machine learning models can perform both classification and regression. Decision tree-based models predict the target variable by learning decision rules. Discriminant classifiers are based on the assumption that each of the classes has a different Gaussian distribution of data, and classification is performed based on Gaussian distribution parameters estimated by a fitting function. Support vector machines ("SVM") are based on the Vapnik-Chervonenkis theory, where a hyperplane that separates the classes is determined. SVM is an efficient algorithm suitable for compact data sets. Nearest neighbor algorithms are based on the assumption that similar things exist nearby. Ensemble methods such as bagged trees (or random forests) combine the predictions of several learning algorithms with improved generalization. Regression analysis can also be performed together with classification, based on the above techniques. Regression analysis is a set of statistical procedures for estimating the relationship between a dependent variable (e.g., the labels in a training dataset) and one or more independent variables (also called predictors or features).

[0040] In various embodiments, the ML models described herein may be implemented as multi-layer perceptrons ("MLPs"), convolutional neural networks ("CNNs"), or other neural networks. Alternatively, some neural networks may be implemented as MLPs, while others are implemented as CNNs or other combinations of neural networks. A neural network may include a machine learning model that can be adjusted (e.g., trained) based on training inputs (e.g., relevant features extracted from ERG signals) to approximate an unknown function. In particular, a neural network may include a model of interconnected digital neurons that communicate and learn to approximate a complex function and generate an output based on multiple inputs provided to the model. For example, a neural network includes one or more machine learning algorithms. In other words, a neural network is an algorithm that implements deep learning techniques, i.e., machine learning, that utilizes a set of algorithms to attempt to model high-level abstractions of data.

[0041] FIG. 2 illustrates a diagram of a machine learning framework 100 for glaucoma diagnosis using a machine learning model, according to an embodiment of the present disclosure. As discussed, the embodiment uses pre-processing techniques and feature extraction methods to create unique and relevant features extracted from the ERG signal to train an ML model in glaucoma diagnosis. As shown, the machine learning framework 100 includes an input ERG signal 102, a glaucoma diagnosis system 105, and an output 165. The glaucoma diagnosis system includes a data pre-processor 110, an advanced feature extractor 120, a glaucoma diagnosis ML model 140, a glaucoma progression model 150, and an RGC count ML model 160. This combination of components enables the machine learning framework 100, through the glaucoma diagnosis system 105, to train a machine learning model to provide a glaucoma-related diagnosis given a subject's ERG signal.

[0042] Previous techniques have utilized neural networks to automate glaucoma diagnosis based on ERG signals, but the previous techniques only utilized morphological and transitional characteristics of the ERG signal as features (13 features) to train the model of the neural network. These methods were limited to basic morphological characteristics of mfERG recordings. Previous techniques also utilized neural networks for ERG-based glaucoma diagnosis, but used continuous wavelet transformed coefficients. This previous technique was limited to wavelet features only. In contrast, the embodiments utilize several advanced features extracted from the ERG signal other than morphological and transitional characteristics to provide higher accuracy for glaucoma in subjects and earlier detection of glaucoma in subjects, as further described in more detail below.

[0043] The input ERG signal 102 includes measurements of the electrical responses of different types of cells in the retina. These measurements include, but are not limited to, rod response (scotopic 0.01 ERG), maximal or standard combined rod-cone response (scotopic 3.0 ERG), OP (scotopic 3.0 pulsatillary wavelet, single-flash cone response (phototopic 3.0 ERG), flicker (phototopic 3.0 Flicker ERG), macular or focal ERG, multifocal ERG, pattern ERG, early receptor potential, STR, direct current ERG, long-duration light-adapted ERG (on-off response), double flash ERG, chromatic stimulus ERG (s-cone ERG), scotopic and photopic ERG, scotopic and photopic luminance response analysis, and saturated a-wave slope analysis.

[0044] The names of stimuli (and responses) are described by the state of light adaptation, and the flash intensity is given in cd·s·m -2 For example, 3.0 cd s m -2A dark-adapted response to 15.0 cd·s·m is called a "dark-adapted 3.0 ERG." In addition, descriptive terms (such as "rod response," "mixed rod-cone response," etc.) can be used. This scheme of naming also applies to non-standard stimuli that may be used for special protocols or due to equipment limitations (e.g., 15.0 cd·s·m -2 If flashes are used under dark-adapted conditions, they may be designated as "dark-adapted 15.0 ERG").

[0045] Each of the ERG signals may include a representative waveform of each of the standard ERGs displayed with amplitude and time calibration. The ERG signals may also be labeled with stimulus variables and light-adapted and dark-adapted conditions. In some implementations, the ERG signals include a baseline of at least 20 ms prior to the stimulus for single-flash responses and indicate each stimulus of the flash with a mark or line. Two responses from each of the stimulus conditions may be displayed to indicate the degree of consistency or variability.

[0046] The glaucoma diagnosis system 105 is a component of the machine learning framework 100 configured to develop a training dataset with relevant features extracted from the input ERG signal 102 and train a machine learning model using the relevant features. The glaucoma diagnosis system 105 develops a glaucoma diagnosis ML model 140 for early glaucoma diagnosis based on a machine learning algorithm by utilizing the highly relevant features from the input ERG signal 102 as predictors. First, the data pre-processor 110 pre-processes the input ERG signal 102 and the feature extractor 120 extracts the highly relevant features (e.g., statistical features 121, advanced wavelet-based features 130).

[0047] The data pre-processor 110 is a component of the glaucoma diagnosis system 105 configured to pre-process the input ERG signal 102. Pre-processing converts the raw data into a usable format by removing anomalies such as different start times of the signal recording, missing data of the signal recording, different sampling frequencies of the signal recording, noise of the signal recording, and unequal lengths of the signal recording. In some implementations, the data pre-processor 110 performs a baseline adjustment 111 on the input ERG signal as part of the pre-processing process. The baseline (start time) of the ERG signal can be different for different subjects and testing protocols. Thus, during the baseline adjustment 111, the measurements can be brought to a common baseline (start time offset to 0) across the baseline adjustment. In some embodiments, the data pre-processor 110 performs baseline adjustment techniques such as a median filter, a linear phase high-pass filter, and an average median filter to perform the baseline adjustment 111 on the input ERG signal 102.

[0048] In some implementations, the data preprocessor 110 performs feature extraction 112 and feature selection on the input ERG signal as part of the preprocessing process. Feature extraction involves computing a reduced set of values ​​from a high-dimensional signal that can summarize most of the information contained in the signal. Feature extraction techniques deploy a transformation of the input space into a low-dimensional subspace that attempts to preserve the most relevant information. In feature selection, the input dimensions that contain the most relevant information to solve a particular problem are selected. These methods aim to improve performance such as estimated accuracy, visualization, and understandability. The advantage of feature selection is that important information related to a single feature is not lost, but if a small set of features is needed and the original features are highly diverse, information may be lost as some of the features have to be omitted. On the other hand, with dimensionality reduction, also known as feature extraction, the size of the feature space can often be reduced without losing information about the original feature space.

[0049] In some implementations, the data pre-processor 110 applies feature extraction techniques to perform some transformation on the original features to generate other features that are more meaningful. These feature extraction techniques include, but are not limited to, Minimum Redundancy Maximum Relevance ("mRmR"), Relief, Conditional Mutual Information Maximization ("CMIM"), Correlation Coefficient, Between-Within Ratio ("BW Ratio"), Interaction, Genetic Algorithm ("GA"), Support Vector Machine Recursive Feature Elimination ("SVM-REF"), Principal Component Analysis ("PCA"), Nonlinear Principal Component Analysis, Independent Component Analysis, and Correlation-Based Feature Selection. These feature extraction techniques are useful for machine learning because they can reduce the complexity of the input data and provide a simple representation of the data, representing each of the variables in the feature space as a linear combination of the original input variables.

[0050] In some embodiments, the data preprocessor implements missing data techniques 113 to handle gaps in the input ERG signal 102. These missing data techniques 113 include complete case analysis, single imputation, log-linear model and estimation using the EM algorithm, propensity score matching, and multiple imputation. This technique focuses on cases where all variables are observed in the complete case analysis. In the single implicit imputation method, missing values ​​are replaced with values ​​from similar responding units in the sample. Similarity is determined by considering observed variables for both respondent and non-respondent data. Multiple imputation replaces each missing value with a vector of at least two imputed values ​​from at least two draws. These draws typically come from a probabilistic imputation procedure. In the log-linear model, the cell counts of the contingency table are directly modeled. The assumption can be that the cell counts follow independent multivariate Poisson distributions, given the expected values ​​of each of the cells. Conditional on the total sample size, the counts follow a multinomial distribution.

[0051] In some embodiments, the data pre-processor 110 implements outlier detection and correction techniques to handle outlier data in the input ERG signal 102. Outliers, by being different from other cases, typically have a disproportionate effect on substantive conclusions about relationships between variables. An outlier can be defined as a data point that deviates significantly from other data points.

[0052] For example, an error outlier is a data point that is distant from other data points because the error outlier results from imprecision. More specifically, error outliers include anomalies that are outside the possible range of values ​​and are caused by not being part of the target population of data, errors in observation, errors in recording, errors in preparing the data, errors in calculation, errors in coding, or errors in data manipulation. These error outliers can be handled by adjusting the data points to correct the error outlier's value or additional such data points from the data set. In some implementations, the data pre-processor 110 defines as an outlier a value that is more than three scaled median absolute deviations ("MAD") away from the median. Once defined as an outlier, the data pre-processor 110 replaces the value with the threshold value used in outlier detection.

[0053] In some embodiments, the data pre-processor 110 performs feature scaling 115 on the input ERG signal 102 as part of the process of data pre-processing. Feature scaling is a method for unifying self-variables or feature ranges in the data. Feature scaling is a necessary step in the computation of stochastic gradient descent. The data pre-processor 110 can perform a variety of feature scaling 115 techniques. These feature scaling 115 techniques include, but are not limited to, data normalization methods and interval scaling.

[0054] Data normalization is a fundamental task in data mining. Different evaluation indicators often have different dimensions, and the difference in numerical values ​​can be quite large. Without processing, the results of data analysis may be affected. Normalized processing is required to eliminate the effects of the dimensionality and range differences between indicators. Data is scaled to a certain domain to facilitate comprehensive analysis. The assumption of the normalization method is that the eigenvalues ​​follow a normal distribution, and each of the genus is transformed into a standard positive distribution with a mean of 0 and a variance of 1 by translation and scaling data transformation. Interval methods utilize boundary information to scale the range of a feature to a certain range of a feature. For example, the commonly used interval scaling method, such as [0,1], uses two extreme values ​​(maximum and minimum) for scaling.

[0055] In some embodiments, the data pre-processor 110 performs a feature selection 116 technique on the input ERG signal 102 for dimensionality reduction from the extracted features. The feature selection 116 technique can be used to reduce the computational cost of modeling to achieve a simple, easy to understand, better generalized, high-performance model. The feature extraction technique 112 can be performed to reduce the dimensionality of the signal, as discussed above. However, in some implementations, the number of resulting features may still be higher than the number of training data. Therefore, a further reduction in the dimensionality of the data can be performed using the feature selection 116 technique to identify relevant features for classification and regression. The feature selection 116 technique can reduce the computational cost of modeling, prevent the generation of complex, overfitted models with high generalization errors, and generate simple, easy to understand, high-performance models. In some embodiments, the data pre-processor 110 uses the mRmR sequential feature selection algorithm to perform the feature selection 116. The mRmR method is designed to drop redundant features, which allows for the design of compact and efficient machine learning-based models.

[0056] The advanced feature extractor 120 is a component of the glaucoma diagnosis system 105 configured to extract advanced relevant features from the input ERG signal. The advanced relevant features can capture subtle changes in the retina that indicate potential signs of glaucoma. The advanced feature extractor 120 is configured to extract features in two phases. First, statistical features 121 are extracted from the input ERG signal 102, followed by the extraction of advanced wavelet-based features 130.

[0057] In some embodiments, the statistical feature extraction 121 technique produces statistical features capable of describing the general behavior of the ERG signal extracted from the input ERG signal 102. These features include, but are not limited to, measures of central tendency, spread, shape, peaks, derivative, and correlation. Measures of central tendency included the mean, median, trimmed mean. Measures of spread included the range, standard deviation, variance, mean absolute deviation, and interquartile range. Measures of shape included skewness, kurtosis, second and third order central moments, and aspect ratio. Measures of peaks included the number of peaks and troughs in the signal. Measures of derivative included the first derivative of the signal with respect to time. Measures of correlation included the correlation coefficient of the signal with respect to time.

[0058] In some embodiments, the advanced feature extractor 120 uses an autoregressive ("AR") coefficient process to extract advanced wavelet-based features, provided that a signal x[n] at time instant n during an AR process of order p can be described as a linear combination of p earlier values ​​of the same signal. In some implementations, the AR coefficient procedure is modeled as follows:

[0059]

number

[0060] where a[i] is the i-th coefficient of the AR model, e[n] denotes white noise with mean 0, and p denotes the AR order. In some implementations, the AR coefficients of each of the blocks are estimated using the Burg method. The order can be determined as fourth order using the ARfit model order selection method. Thus, a fourth order AR model can be selected to represent each of the ERG signal components.

[0061] In some embodiments, the feature extractor 120 uses wavelet-based Shannon entropy to extract advanced wavelet-based features. Shannon entropy is an information-theoretic measure of a signal. The Shannon entropy value for MOD-PWT using four-level wavelet decomposition can be calculated on the end nodes of the wavelet.

[0062] Shannon entropy, or information entropy, measures how much information there is in an event. Generally speaking, the more certain or deterministic an event is, the less information it contains. More specifically, information is an increase in uncertainty or entropy. As an example, if someone is told something they already know, the information they receive will be extremely small. Therefore, this information will have extremely low entropy. If they are told something they know little about, they will receive a lot of new information. Therefore, this information will have high entropy. In the context of wavelet packets, in some implementations, the formula for Shannon entropy using the wavelet packet transform is:

[0063]

number

[0064] where N is the number of coefficients in the jth node, and p j,kis the normalized squared wavelet packet coefficient at the jth terminal node of the wavelet.

[0065] In some embodiments, the advanced feature extractor 120 uses the multifractal leader estimate 139 and the multiscale wavelet variance estimate to extract advanced wavelet-based features. In some implementations, the multifractal measures of the ERG signal are obtained using two wavelet methods: the wavelet leader and the cumulant of the scaling exponents. The wavelet leader is a spatiotemporal local upper limit of the absolute values ​​of the discrete wavelet coefficients. These upper limits are used to calculate the Hölder exponents that characterize the local regularity. In addition, a second cumulant of the scaling exponents can be obtained. The scaling exponents are scale-dependent exponents that describe the power-law behavior of the signal at various resolutions. The second cumulant can indicate the divergence of the scaling exponents from linearity. The wavelet variance of the ERG signal can also be obtained as a feature. The wavelet variance quantifies the degree of variability of the signal with scale, or more precisely, the degree of variability of the signal between octave band frequency intervals.

[0066] The glaucoma diagnostic ML model 140 is a component of the glaucoma diagnostic system 105 configured to predict a binary classification of a glaucoma diagnosis (e.g., glaucoma and non-glaucoma). The glaucoma diagnostic ML model 140 is trained using a training data set collected by the data pre-processor, which includes the relevant features extracted by the feature extractor 120. The glaucoma diagnostic ML model 140 can be configured as several different types of machine learning models that can be trained to classify ERG signals. These machine learning models include, but are not limited to, decision trees, discriminants, SVMs, nearest neighbors, ensemble classifiers, and neural networks. In some embodiments, the binary classifications produced by the glaucoma diagnostic ML model 140 are based on statistical features including cone correlation, flicker mean, median, and high rod and cone skewness, and cone standard deviation. In some embodiments, the binary classifications produced by the glaucoma diagnostic ML model 140 are based on wavelet-based features. These extracted wavelet features include Shannon entropy values ​​for MOD-PWT, rods and cones, rods, STR, and OP.

[0067] The glaucoma progression ML model 150 is a component of the glaucoma diagnosis system 105 configured to provide a multi-class classification of glaucoma progression. In some embodiments, the glaucoma progression classification is based on IOP as (normal<12 mmHg, high [≧12 mmHg<17 mmHg], glaucoma≧17 mmHg). In some implementations, the multi-class classification (classifying different stages, normal, high, and glaucoma) is based on statistical features extracted by the feature extractor 120. These statistical features include cone correlation, number of troughs in high cones, STR kurtosis, and flicker mean. The glaucoma progression ML model 150 can be configured as several different machine learning models. These machine learning models include, but are not limited to, SVM and ensemble-based classifiers (e.g., bagged tree models).

[0068] In some implementations, multi-class classification is performed using wavelet-based features. These extracted wavelet features include wavelet variance and Shannon entropy values ​​for rods. The AR coefficients for MOD-PWT can be used as features from high flicker, flicker, high cone, and STR. The identification of flicker ERG tests and corresponding features reaffirms the ability of the current approach in identifying relevant features among tests, which can aid in early stage diagnosis of glaucoma. Using these approaches provides improved accuracy over conventional methods, and the improved accuracy indicates that wavelet-based features can distinguish healthy subjects from subjects with glaucoma, suggesting that wavelet-based features are more sensitive to subtle changes in the ERG signal due to glaucoma. The multi-class classification capabilities of the glaucoma diagnosis system 105 reaffirm the complex nature of the ERG signal in assessing disease progression.

[0069] The RGC count ML model 160 is a component of the glaucoma diagnosis system 105 configured to predict RGC counts based on ERG signals. In some embodiments, the RGC count ML model 160 uses regression machine learning techniques to predict RGC counts from ERG signals. These regression machine learning techniques include, but are not limited to, Gaussian process regression ("GPR"), linear regression, decision trees, support vector regression, lasso regression, and random forests. In some embodiments, feature selection for the RGC count ML model 160 is performed using mRmR sequential feature selection, trained with statistical features and wavelet-based features, individually and / or collectively. In some implementations, the resulting trained RGC count ML model 160 shows that RGC counts are strongly correlated with STR and OP, making STR and OP the primary features selected for RGC regression.

[0070] In some embodiments, the glaucoma diagnosis system 105 evaluates the performance of the ML models 140, 150, and 160 during the training process. Various performance evaluation metrics can be utilized to compare the performance of different machine learning algorithms. These metrics include, but are not limited to, accuracy, sensitivity, specificity, precision, recall, f-score, and root mean square error.

[0071] In some implementations, the following mathematical formulations may be used to calculate different evaluation metrics: A true positive ("TP") may represent a case where the model correctly predicted the positive (i.e., glaucoma) class. A true negative ("TN") may represent a case where the model correctly predicted the negative (i.e., non-glaucoma) class. A false positive ("FP") may represent a case where the model incorrectly predicted the positive (i.e., glaucoma) class. A false negative ("FN") may represent a case where the model incorrectly predicted the negative (i.e., non-glaucoma) class. Accuracy is the proportion of correctly classified observations, as calculated below:

[0072]

number

[0073] Sensitivity / recall can estimate the proportion of actual positives (eg, actual glaucoma) identified as calculated below:

[0074]

number

[0075] Here, recall estimates the ability of the model to correctly reject healthy subjects without glaucoma. Precision can estimate the proportion of correct positive predictions (eg, predictions of glaucoma) as calculated below:

[0076]

number

[0077] The F-score can be estimated as the harmonic mean of precision and recall, as calculated below:

[0078]

number

[0079] Root mean square error ("RMSE") can also be used as a performance evaluation metric for regression analysis (e.g., analysis of the RGC count ML model 160). In some implementations, the RSME is the standard deviation of the prediction errors (residuals), as calculated below:

[0080]

number

[0081] where N is the number of observations.

[0082] It should be noted that Figure 2 is intended to illustrate major, representative components of machine learning framework 100. However, in some embodiments, the individual components may have more or less complexity than depicted in Figure 2, there may be other or additional components than those shown in Figure 2, and the number, type, and configuration of such components may vary.

[0083] FIG. 3 illustrates an exemplary diagnostic evaluation of an ERG signal using a diagnostics system 300, according to an embodiment of the present disclosure. As discussed, the techniques described herein provide a glaucoma diagnosis 170 (e.g., glaucoma, non-glaucoma), a glaucoma progression diagnosis 180 (e.g., normal, high, and glaucoma), and an RGC count analysis 190. Thus, in some embodiments, a user can provide ERG signal data 102 to train the ML models 140, 150, 160 of the glaucoma diagnostic system 105 to generate a glaucoma diagnostics report 330 that can include the glaucoma diagnosis 170, the glaucoma progression 180, and the RGC count 190. As shown in FIG. 3, optionally, the glaucoma diagnostic system 105 is implemented as part of the diagnostics system 300. Alternatively, the glaucoma diagnostic system 105 is implemented in a separate system that provides at least an output 165 to the diagnostics system 300 for evaluation by a user.

[0084] Once the output 165 representing the glaucoma prediction of the ERG signal is obtained by the diagnostics system 300, the output 165 can be evaluated by a user (e.g., a medical professional). In some implementations, user input 302 (e.g., ERG signal data) can be received by a diagnostics manager 304 of the diagnostics system 300. The user input 302 can include any information regarding the potential subject, including the potential subject's ERG signal data. The diagnostics manager 304 can generate a diagnostics report 330 that includes diagnostics known in the art and may include the output 165 produced by the glaucoma diagnostic system 105 as a result of evaluating the user input 302.

[0085] In some implementations, the resulting classification output 165 generated by the glaucoma diagnostic system 105 may be output to a different or downstream diagnostic system for evaluation and responsive action. The responsive action may take any known or subsequently developed form, including output to a medical professional, such as via a user interface, when the ERG signal data indicates a diagnosis of glaucoma. The user interface may comprise user interface elements for drilling down into the details of the notification, including identifying the subject's particular ERG signal data and their corresponding classifications produced by the corresponding ML models 140, 150, and 160. In this manner, the medical professional may identify which features of the ERG signal data contributed to the diagnosis, as well as the progression of the glaucoma condition. Moreover, probability / confidence values ​​for the classification, weighting values, etc. may be provided in the notification to provide more evidence for the classification associated with the ERG signal data. User interface elements may be provided to allow a medical professional to provide input indicating the correctness / incorrectness of the classification of the ERG signal data, and this information may then be stored for the creation of new training data sets for updating the training of the ML models 140, 150, 160 at future times.

[0086] 4 illustrates a schematic diagram of a glaucoma diagnosis system 400 (e.g., the "glaucoma diagnosis system 105" described above) according to an embodiment of the present disclosure. As shown, the glaucoma diagnosis system 400 includes, but is not limited to, a user interface manager 402, a training manager 404, a data pre-processor 405, a machine learning component 406, and a memory manager 408. The machine learning component 406 includes a glaucoma diagnosis ML model 410, a glaucoma progression ML model 412, and an RGC count ML model 414. The memory manager 408 includes an input ERG signal 418, a glaucoma diagnosis prediction 420, a glaucoma progression prediction 422, and an RGC count prediction 424.

[0087] As shown in FIG. 4, the glaucoma diagnosis system 400 includes a user interface manager 402. For example, the user interface manager 402 allows a user to input 418 an ERG signal into the glaucoma diagnosis system 400. In some embodiments, the user interface manager 402 provides a user interface through which a user can upload an input ERG signal 418 representing a subject's ERG signal data, as discussed above. Alternatively or additionally, the user interface may allow a user to download the input ERG signal 418 from a local or remote storage location (e.g., by providing an address (e.g., a URL or other endpoint) associated with the input ERG signal 418 source). In some embodiments, the user interface may allow a user to link an ERG signal capture device, such as electroretinogram equipment or other hardware, to capture ERG signal data and provide the ERG signal data to the glaucoma diagnosis system 400. In some embodiments, the user interface manager 402 also allows a user to provide the specific ERG signal data to be evaluated and analyzed. Additionally, the user interface manager 402 allows a user to request the glaucoma diagnosis system 400 to provide a specific diagnostic prediction. For example, a user may request only a glaucoma diagnosis prediction. In some embodiments, the user interface manager 402 allows a user to edit the input ERG signal data 418. Alternatively, the input ERG signal 418 can be evaluated in a separate diagnostics system that is separate from the glaucoma diagnosis system 400, as discussed above.

[0088] As shown in FIG. 4, the glaucoma diagnosis system 400 also includes a training manager 704. The training manager 704 can teach, guide, adjust, and / or train one or more machine learning models. In particular, the training manager 704 can train the machine learning models based on a plurality of training data (e.g., input ERG signals 418). As discussed, the input ERG signals 418 include ERG signal data captured from various subjects. More specifically, the training manager 704 can access, identify, generate, create, and / or determine training inputs and utilize the training inputs to train and fine-tune the machine learning models. For example, the training manager 704 can train the glaucoma diagnosis ML model 410, the glaucoma progression ML model 412, and the RGC count ML model 414, as well as provide evaluation metrics, as discussed above. As discussed, the ML models are trained specifically for a particular diagnosis in some embodiments. For example, an ML model is trained (or an existing ML model is retrained) to diagnose subjects with glaucoma, and another ML model is trained to diagnose glaucoma progression in subjects.

[0089] 4, the glaucoma diagnosis system 400 also includes a data pre-processor 405 (e.g., the “data pre-processor 110” described above). As discussed, the data pre-processor 405 implements data pre-processing techniques to generate training data sets for training the machine learning models by the training manager 704. The data pre-processor 405 can generate a number of relevant features from the input ERG signal 418 to create training data sets for the glaucoma diagnosis ML model 410, the glaucoma progression ML model 412, and the RGC count ML model 414, respectively. The resulting training data sets are provided to the training manager 704 for training, as discussed.

[0090] As shown in FIG. 4, the glaucoma diagnosis system 400 also includes a machine learning component 406. The machine learning component 406 may host multiple machine learning models, such as a glaucoma diagnosis ML model 410, a glaucoma progression ML model 412, and an RGC count ML model 414, or other machine learning models. The machine learning component 406 may include an execution environment, libraries, and / or any other data required to execute the machine learning models. In some embodiments, the machine learning component 406 may be associated with dedicated software and / or hardware resources to execute the machine learning models. As discussed, the glaucoma diagnosis ML model 410, the glaucoma progression ML model 412, and the RGC count ML model 414 can be implemented as a decision tree model, a discriminant model, a support vector machine, a nearest neighbor algorithm, or an ensemble classifier, or a combination of these or other types of ML models, as well as a combination of MLPs, CNNs, or a combination of these or other types of neural networks.

[0091] 4 as being hosted by the machine learning component 406, in various embodiments the ML models may be hosted in multiple machine learning components and / or as part of different components. For example, the glaucoma diagnosis manager may host a glaucoma diagnosis ML model 410. Similarly, the glaucoma progression manager may host a glaucoma progression ML model 412 and an RGC count ML model 414. In various embodiments, the glaucoma diagnosis manager and the glaucoma progression manager may each include their own machine learning components or other host environments in which the respective ML models execute.

[0092] As shown in FIG. 4, the glaucoma diagnosis system 400 also includes a memory manager 408. The memory manager 708 maintains data for the glaucoma diagnosis system 400. The memory manager 408 can hold any type, size, or variety of data as needed to perform the functions of the glaucoma diagnosis system 400. The memory manager 408, as shown in FIG. 4, includes an input ERG signal 418. The input ERG signal 418 can include multiple ERG signals associated with various subjects, as discussed in further detail above. In particular, in one or more embodiments, the input ERG signal 418 includes an ERG signal that is utilized by the training manager 404 to train multiple ML models to generate a glaucoma diagnosis prediction 420, a glaucoma progression prediction 422, and an RGC count prediction 424.

[0093] Each of the components 402-408 of the glaucoma diagnostic system 400 (shown in FIG. 4) and their corresponding elements may communicate with one another using any suitable communication technology. Although the components 402-408 and their corresponding elements of the components 402-408 are shown as separate in FIG. 4, it will be appreciated that any of the components 402-408 and their corresponding elements of the components 402-408 may be combined into fewer components, such as a single facility or module, split into more components, or configured into different components that may service a particular embodiment.

[0094] The components 402-408 and corresponding elements of the components 402-408 may comprise software, hardware, or both. For example, the components 402-408 and corresponding elements of the components 402-408 may comprise one or more instructions stored in a computer-readable storage medium and executable by a processor of one or more computing devices. When executed by one or more processors, the computer-executable instructions of the glaucoma diagnosis system 400 may cause a client device and / or a server device to perform the methods described herein. Alternatively, the components 402-408 and corresponding elements of the components 402-408 may comprise hardware, such as a special purpose processing device, to perform a function or group of functions. Furthermore, the components 402-408 and corresponding elements of the components 402-408 may comprise a combination of computer-executable instructions and hardware.

[0095] Additionally, the components 402-408 of the glaucoma diagnosis system 400 may be implemented, for example, as one or more standalone applications, as one or more modules of an application, as one or more plug-ins, as one or more library functions or functions that may be called by other applications, and / or as a cloud computing model. Thus, the components 402-408 of the glaucoma diagnosis system 400 may be implemented as standalone applications, such as desktop or mobile applications. Additionally, the components 402-408 of the glaucoma diagnosis system 400 may be implemented as one or more web-based applications hosted on a remote server.

[0096] 1-4, corresponding text, and examples provide several different systems and devices that enable glaucoma diagnosis and progression using relevant features extracted from ERG signals that enable early stage detection of glaucoma in subjects. In addition to the above, embodiments may also be described with respect to flowcharts that include acts and steps for achieving a particular result. For example, FIGS. 5, 6, and 7 show flowcharts of exemplary methods according to one or more embodiments. The methods described with respect to FIGS. 5, 6, and 7 may be performed with fewer or more steps / acts, or the steps / acts may be performed in a different order. Additionally, the steps / acts described herein may be repeated or performed in parallel with each other or with different instances of the same or similar steps / acts.

[0097] Example Flow Diagram 5-7, flow diagrams illustrating various methods are provided. Each of the blocks of methods 500-700 and any other methods described herein includes computing operations implemented using any combination of hardware, firmware, and / or software. For example, in some embodiments, various functions are performed by a processor executing instructions stored in a memory. In some cases, the methods are embodied as computer usable instructions stored on a computer storage medium. In some implementations, the methods are provided by a standalone application, a service or hosted service (standalone or in combination with another hosted service), or a plug-in to another product, to name a few examples.

[0098] 5 illustrates a flowchart 500 of a series of acts in a method for extracting relevant features from a set of ERG signals to create a training data set for training a machine learning model in early stage glaucoma diagnosis according to an embodiment of the present disclosure. In one or more embodiments, the method 500 is implemented in a digital media environment including a glaucoma diagnosis system 400. The method 500 is intended to illustrate one or more methods according to the present disclosure and is not intended to limit possible embodiments. Alternative embodiments may include additional, fewer, or different steps than those depicted in FIG. 5.

[0099] As shown in Figure 5, the method 500 includes collecting 510 a set of ERG signals. As discussed, the ERG signals can include ERG signal data that measures the electrical response of cells in the retina from animals and / or humans. The ERG signals include measurements of retinal OP, STR, rods, rods and cones, high rods and cones, cones, high cones, flicker, and high flicker.

[0100] Method 500 also includes preprocessing the ERG signal by removing anomalies contained within the data of the ERG signal by the data preprocessor. This is shown in step 520. Preprocessing includes baseline adjustment, feature extraction, missing data handling, outlier handling, feature scaling, and feature selection on the input ERG signal. In some implementations, the data preprocessor performs baseline adjustment on the input ERG signal as part of the process of preprocessing. The baseline (start time) of the ERG signal can be different for different animals and testing protocols. Thus, during baseline adjustment, measurements can be brought to a common baseline (start time offset to 0) across baseline adjustment. In some embodiments, the data preprocessor performs baseline adjustment techniques such as a median filter, a linear phase high pass filter, and an average median filter to perform baseline adjustment on the input ERG signal.

[0101] The method 500 further includes extracting statistical features and advanced wavelet-based features from the ERG signal using a feature extractor, as shown in step 530. The statistical feature extraction 121 technique produces statistical features capable of describing the general behavior of the ERG signal extracted from the input ERG signal 102. These features include, but are not limited to, measures of central tendency, spread, shape, peak, derivative, and correlation.

[0102] The feature extractor can implement various techniques to extract advanced wavelet-based features. These techniques include, but are not limited to, AR coefficient processing, wavelet-based Shannon entropy, multifractal leader estimates, and multiscale wavelet variance estimates. As discussed, Shannon entropy, or information entropy, measures how much information there is in an event. Generally, the more certain or deterministic an event is, the less information it contains. Wavelet leaders are spatiotemporal local upper bounds on the absolute values ​​of discrete wavelet coefficients. These upper bounds are used to calculate the Hölder exponent, which characterizes the local regularity. Wavelet variance quantifies the degree of variability of a signal with scale, or more precisely, the degree of variability of a signal between octave band frequency intervals.

[0103] Method 500 also includes extracting relevant features from the relevant features (e.g., statistical and wavelet-based features) using feature selection and extraction techniques. This is shown in step 540. Feature selection and extraction techniques include, but are not limited to, mRmR relief, CMIM correlation coefficient, Bw ratio, interaction, GA, SVM-REF, PCA, nonlinear PCA, independent component analysis, and correlation-based feature selection. A training dataset is collected and created with the extracted relevant features to train a machine learning model in glaucoma diagnosis. This is shown in step 550.

[0104] 6 illustrates a flowchart 600 of a series of acts in a method of training a machine learning model to provide a glaucoma diagnosis classification according to an embodiment of the present disclosure. In one or more embodiments, the method 600 is implemented in a digital media environment that includes the glaucoma diagnosis system 400. The method 600 is intended to illustrate one or more methods according to the present disclosure and is not intended to limit possible embodiments. Alternative embodiments may include additional, fewer, or different steps than those depicted in FIG. 6.

[0105] As shown in FIG. 6, the method 600 includes a step 610 of accessing a training data set including ERG signals. As discussed, the ERG signals can include ERG signal data that measures electrical responses of cells in the retina of an animal and / or a human. The ERG signals include measurements of retinal OP, STR, rods, rods and cones, high rods and cones, cones, high cones, flicker, and high flicker. Additionally, each of the ERG signals is associated with a binary label that indicates whether the ERG signal is glaucomatous or non-glaucomatous.

[0106] The method 600 further includes training the machine learning model on the training dataset. This is shown in step 620. The training manager of the glaucoma diagnosis system can teach the machine learning model and guide, adjust, and / or train the machine learning model. In particular, the training manager can train the machine learning model based on the training dataset (e.g., the input ERG signal). More specifically, the training manager can access, identify, generate, create, and / or determine the training inputs and utilize the training inputs to train and fine-tune the machine learning model. In some embodiments, the training manager trains the glaucoma diagnosis ML model, the glaucoma progression ML model, and the RGC count ML model, as discussed above, and provides the evaluation metrics. As discussed, in some embodiments, the ML models are specifically trained for a particular diagnosis. For example, a machine learning model is trained (or an existing ML model is retrained) to diagnose a subject with glaucoma, and another ML model is trained to diagnose glaucoma progression in a subject.

[0107] In some embodiments, the training manager evaluates the binary classification prediction using a performance evaluation metric and retrains the machine learning model until a stopping criterion is reached. The training manager can also evaluate the multi-class classification prediction using a performance evaluation metric and retrain the glaucoma progression ML model until a stopping criterion is reached. The training manager can also evaluate the ERG count prediction using a performance evaluation metric and retrain the RGC count ML model until a stopping criterion is reached.

[0108] Method 600 also includes measuring the subject's ERG signal. This is shown in step 630. Using known techniques, equipment, and devices, measurements of the ERG signal are collected. As discussed, the ERG signal can include ERG signal data that measures the electrical response of cells in the animal and / or human retina. The ERG signal includes measurements of retinal OP, STR, rods, rods and cones, high rods and cones, cones, high cones, flicker, and high flicker.

[0109] Once measured, the ERG signal is provided as an ERG signal input to a trained machine learning model. This is shown in step 640. Once input, the machine learning model provides a binary classification of a glaucoma diagnosis (e.g., glaucoma and non-glaucoma). If the binary classification returns glaucoma, the classification can act as a diagnosis for the subject. This is shown in step 650.

[0110] In some embodiments, the measured ERG signal is provided as an ERG signal to a glaucoma progression ML model. Once input, the glaucoma progression ML model provides a multi-class classification of glaucoma progression in the subject. Additionally or alternatively, the measured ERG signal is provided as an ERG signal to an RGC count ML model. Once input, the RGC count ML model provides a prediction of the subject's RGC count based on the provided ERG signal.

[0111] 7 illustrates a flow chart of a series of acts in a method of glaucoma diagnosis using a machine learning framework using ERG signals, according to one or more embodiments. In one or more embodiments, method 700 is implemented in a digital media environment including glaucoma diagnosis system 400. Method 700 is intended to illustrate one or more methods according to the present disclosure and is not intended to limit possible embodiments. Alternative embodiments may include additional, fewer, or different steps than those depicted in FIG. 7.

[0112] 7, the method 700 includes receiving 710 a request to provide a glaucoma diagnosis generated by a glaucoma diagnosis system that includes a plurality of machine learning models. The glaucoma diagnosis system generates the glaucoma diagnosis by inputting the ERG signal data into a trained machine learning model that provides a binary classification of glaucoma or non-glaucoma.

[0113] In some embodiments, the glaucoma diagnostic system includes a glaucoma diagnosis ML model for predicting a glaucoma diagnosis (e.g., glaucoma or non-glaucoma), a glaucoma progression ML model for predicting glaucoma progression based on IOP as normal, high, or glaucomatous, and an RGC count ML model for predicting RGC count based on regression analysis. In some embodiments, the glaucoma diagnostic system trains the machine learning model using a set of ERG signals. The ERG signals contain data including OP, STR, rods, rods and cones, high rods and cones, cones, high cones, flicker, and high flicker data.

[0114] 7, the method 700 includes acquiring 720 an ERG signal of a subject. For example, the ERG signal may include information related to the retina, as discussed above. The glaucoma diagnosis system may be implemented as part of a diagnostics system or as a separate system that makes a glaucoma diagnosis available to the diagnostics system for evaluation.

[0115] 7, the method 700 includes generating 730 a glaucoma diagnosis based on the ERG signal and the request. In some embodiments, generating the glaucoma diagnosis further includes generating a glaucoma progression prediction and an RGC count prediction. In some implementations, the glaucoma diagnosis, glaucoma progression, and RGC count prediction are provided in a diagnostics report.

[0116] Exemplary Computing Environment 8 illustrates a schematic diagram of an example computing environment 800 in which the glaucoma diagnosis system 400 may operate, according to one or more embodiments of the present disclosure. In one or more embodiments, the computing environment 800 includes a service provider 802, which may include one or more servers 804 connected to a plurality of client devices 806A-806C via one or more networks 808. The client devices 806A-806C, the one or more networks 808, the service provider 802, and the one or more servers 804 may communicate with each other or with other components using any communications platform and technology suitable for transporting data and / or communications signals, including any known communications technologies, devices, media, and protocols supporting remote data communications, examples of which are described in more detail below with respect to FIG. 9.

[0117] Although FIG. 8 illustrates a particular arrangement of client devices 806A-806C, one or more networks 808, service provider 802, and one or more servers 804, various additional arrangements are possible. For example, client devices 806A-806C may communicate directly with one or more servers 804, bypassing network 808. Or, alternatively, client devices 806A-806C may communicate directly with each other. Service provider 802 may be a public cloud service provider that owns and operates its own infrastructure in one or more data centers and provides this infrastructure on demand to customers and end users to host applications on one or more servers 804. The servers may include one or more hardware servers (e.g., hosts), each of which has its own computing resources (e.g., processors, memory, disk space, networking bandwidth, etc.), which may be securely partitioned among multiple customers, each of which hosts its own applications on one or more servers.

[0118] In some embodiments, the service provider may be a private cloud provider that maintains a cloud infrastructure for a single entity. The one or more servers 804 may also include one or more hardware servers, each having its own computing resources that are divided among applications hosted by the one or more servers for use by members of the entity or their customers.

[0119] 8 is shown as having various components, computing environment 800 may have additional or alternative components. For example, environment 800 may be implemented on a single computing device along with glaucoma diagnosis system 400. In particular, glaucoma diagnosis system 400 may be implemented in whole or in part on client device 802A.

[0120] As shown in Figure 8, environment 800 may include client devices 806A-806C. Client devices 806A-806C may comprise any computing device. For example, client devices 806A-806C may comprise one or more personal computers, laptop computers, mobile devices, mobile phones, tablets, special purpose computers, TVs, or other computing devices, including the computing devices described below with respect to Figure 8. Although three client devices are shown in Figure 8, it will be appreciated that client devices 806A-806C may comprise any number of client devices (more or less than shown).

[0121] Moreover, as shown in FIG. 8, the client devices 806A-806C and the one or more servers 804 may communicate via one or more networks 808. The one or more networks 808 may represent a single network or a collection of networks, such as the Internet, a corporate intranet, a virtual private network (VPN), a local area network (LAN), a wireless local network (WLAN), a cellular network, a wide area network (WAN), a metropolitan area network (MAN), or a combination of two or more such networks. Thus, the one or more networks 808 may be any suitable network through which the client devices 806A-806N may access the service provider 802 and the server 804, or vice versa. The one or more networks 808 are discussed in more detail below with respect to FIG. 9.

[0122] In addition, the environment 800 may also include one or more servers 804. The one or more servers 804 may generate, store, receive, and transmit any type of data, including an input ERG signal 418, a glaucoma diagnosis prediction 420, a glaucoma progression prediction 422, an RGC count prediction 424, or other information. For example, the server 804 may receive data from a client device, such as client device 806A, and send data to another client device, such as client device 802B and / or 802C. The server 804 may also transmit electronic messages between one or more users of the environment 800. In an exemplary embodiment, the server 804 is a data server. The server 804 may also comprise a communication server or a web hosting server. Additional details regarding the server 804 are discussed below with respect to FIG. 9.

[0123] As discussed above, in one or more embodiments, the one or more servers 804 can include or implement at least a portion of the glaucoma diagnostic system 400. In particular, the glaucoma diagnostic system 400 can comprise an application running on the one or more servers 804, or portions of the glaucoma diagnostic system 400 can be downloaded from the one or more servers 804. For example, the glaucoma diagnostic system 400 can include a web hosting application that enables the client devices 806A-806C to interact with content hosted on the one or more servers 804. To illustrate, in one or more embodiments of the environment 800, the one or more client devices 806A-806C can access web pages supported by the one or more servers 804. In particular, the client device 806A can run a web application (e.g., a web browser) to enable a user to access, view, and / or interact with web pages or websites hosted on the one or more servers 804.

[0124] When the client device 806A accesses a web page or other web application hosted on the one or more servers 804, in one or more embodiments, the one or more servers 804 can provide access to one or more digital images (e.g., input ERG signal 418) stored on the one or more servers 804. Additionally, the client device 806A can receive a request (i.e., via user input) to perform a glaucoma diagnosis and provide the request to the one or more servers 804. Upon receiving the request, the one or more servers 804 can automatically perform the methods and processes described above. The one or more servers 804 can provide all or a portion of the glaucoma diagnosis prediction to the client device 806A for display to the user. The one or more servers 804 can also host a diagnostics application used to provide a diagnosis to the subject.

[0125] As just described, the glaucoma diagnostic system 400 may be implemented, in whole or in part, by individual elements 802-808 of the computing environment 800. Although some components of the glaucoma diagnostic system 400 have been described in the previous examples with respect to particular elements of the computing environment 800, it will be appreciated that various alternative implementations are possible. For example, in one or more embodiments, the glaucoma diagnostic system 400 is implemented on any of the client devices 806A-C. Similarly, in one or more embodiments, the glaucoma diagnostic system 400 may be implemented on one or more servers 804. Moreover, different components and functions of the glaucoma diagnostic system 400 may be implemented separately among the client devices 806A-C, the one or more servers 804, and the network 808.

[0126] Embodiments of the present disclosure may comprise or utilize special purpose or general purpose computers including computer hardware, such as, for example, one or more processors and system memory, as discussed in further detail below. Embodiments within the scope of the present disclosure also include physical and other computer readable media for carrying or storing computer executable instructions and / or data structures. In particular, one or more of the processes described herein may be embodied, at least in part, in a non-transitory computer readable medium and implemented as instructions executable by one or more computing devices (e.g., any of the media content access devices described herein). In general, a processor (e.g., a microprocessor) receives instructions from a non-transitory computer readable medium (e.g., a memory, etc.) and executes those instructions, thereby performing one or more processes, including one or more of the processes described herein.

[0127] A computer-readable medium can be any available medium that can be accessed by a general-purpose or special-purpose computer system. A computer-readable medium that stores computer-executable instructions is a non-transitory computer-readable storage medium (device). A computer-readable medium that carries computer-executable instructions is a transmission medium. Thus, by way of example and not limitation, embodiments of the present disclosure can comprise at least two distinctly different kinds of computer-readable media: a non-transitory computer-readable storage medium (device) and a transmission medium.

[0128] Non-transitory computer-readable storage media (devices) include RAM, ROM, EEPROM, CD-ROM, (e.g., RAM-based) solid-state drives (SSD), flash memory, phase-change memory (PCM), other types of memory, other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired program code means in the form of computer-executable instructions or data structures and that can be accessed by a general-purpose or special-purpose computer.

[0129] A "network" is defined as one or more data links that enable the transport of electronic data between computer systems and / or modules and / or other electronic devices. When information is transferred or provided to a computer over a network or another communications connection (either hardwired, wireless, or a combination of hardwired or wireless), the computer properly views the connection as a transmission medium. Transmission media can include networks and / or data links that can be used to carry desired program code means in the form of computer-executable instructions or data structures and that can be accessed by a general-purpose or special-purpose computer. Combinations of the above should also be included within the scope of computer-readable media.

[0130] Furthermore, upon reaching various computer system components, program code means in the form of computer executable instructions or data structures can be automatically transferred from transmission media to non-transitory computer readable storage media (devices) (or vice versa). For example, computer executable instructions or data structures received over a network or data link can be buffered in RAM in a network interface module (e.g., "NIC") and then eventually transferred to computer system RAM and / or to less volatile computer storage media (devices) in the computer system. It should therefore be understood that non-transitory computer readable storage media (devices) can be included in computer system components that also (or even primarily) utilize transmission media.

[0131] Computer-executable instructions include, for example, instructions and data that, when executed on a processor, cause a general-purpose computer, a special-purpose computer, or a special-purpose processing device to perform a certain function or group of functions. In some embodiments, computer-executable instructions are executed on a general-purpose computer to transform the general-purpose computer into a special-purpose computer that implements elements of the present disclosure. Computer-executable instructions may be, for example, binaries, intermediate format instructions such as assembly language, or even source code. Although the subject matter has been described in language specific to structural features and / or methodological acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the described features or acts described above. Rather, the described features and acts are disclosed as exemplary forms of implementing the claims.

[0132] Those skilled in the art will appreciate that the present disclosure may be practiced in a networked computing environment with many types of computer system configurations, including personal computers, desktop computers, laptop computers, message processors, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, cell phones, PDAs, tablets, pagers, routers, switches, etc. The present disclosure may also be practiced in a distributed system environment where tasks are performed by both local and remote computer systems that are linked through a network (either by hardwired data links, wireless data links, or a combination of hardwired and wireless data links). In a distributed system environment, program modules may be located in both local and remote memory storage devices.

[0133] The embodiments of the present disclosure can also be implemented in a cloud computing environment. In this specification, "cloud computing" is defined as a model for enabling on-demand network access to a shared pool of configurable computing resources. For example, cloud computing can be utilized in the market to provide ubiquitous and convenient on-demand access to a shared pool of configurable computing resources. The shared pool of configurable computing resources can be rapidly provisioned through virtualization, released with low management effort or service provider interaction, and then scaled accordingly.

[0134] The cloud computing model can consist of various characteristics, such as, for example, on-demand self-service, broad network access, resource pooling, rapid elasticity, measured services, etc. The cloud computing model can also exhibit various service models, such as, for example, Software as a Service ("SaaS"), Platform as a Service ("PaaS"), Infrastructure as a Service ("IaaS"), etc. The cloud computing model can also be deployed using various deployment models, such as private cloud, community cloud, public cloud, hybrid cloud, etc. As used herein and in the claims, a "cloud computing environment" is an environment in which cloud computing is utilized.

[0135] Example Operating Environment Having described an overview of embodiments of the present technology, an exemplary operating environment in which embodiments of the present technology may be implemented is described to provide a general context for various aspects of the present technology. Referring now to FIG. 9, in particular, an exemplary operating environment for implementing embodiments of the present technology is shown and generally designated as computing device 900. The computing device 900 is only one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the present technology. Neither should the computing device 900 be interpreted as having any dependency or requirement relating to any one or combination of illustrated components.

[0136] The techniques of this disclosure may be described in the general context of computer code or machine usable instructions, including computer executable instructions, such as program modules, executed by a computer or other machine, such as a personal data assistant or other handheld device. Generally, program modules, including routines, programs, objects, components, data structures, etc., refer to code that performs particular tasks or implements particular abstract data types. The techniques may be practiced in a variety of system configurations, including handheld devices, consumer electronics, general purpose computers, more specialized computing devices, etc. The techniques may also be practiced in distributed computing environments where tasks are performed by remote processing devices linked through a communications network.

[0137] Referring to FIG. 9, a computing device 900 includes a bus 910 that directly or indirectly couples the following devices: memory 912, one or more processors 914, one or more presentation components 916, input / output ports 918, input / output components 920, and an exemplary power supply 922. The bus 910 represents what may be one or more buses (such as an address bus, a data bus, or a combination thereof). While the various blocks in FIG. 9 are shown with lines for clarity, in practice, it is not very clear to represent the various components diagrammatically, and metaphorically, the lines would more accurately be gray and fuzzy. For example, one may consider a presentation component, such as a display device, or an I / O component. In addition, a processor has memory. We recognize that such is the nature of the art, and again state that the diagram of FIG. 9 merely illustrates an exemplary computing device that may be used in connection with one or more embodiments of the present technology. No distinction is made between categories such as "workstations," "servers," "laptops," "handheld devices," etc., all as contemplated within the scope of FIG. 9 and within the reference to "computing devices."

[0138] Computing device 900 typically includes a variety of computer-readable media. Computer-readable media can be any available media that can be accessed by computing device 900 and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer-readable media can comprise computer storage media and communication media.

[0139] Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by the computing device 800. Computer storage media inherently excludes signals.

[0140] Communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and other wireless media. Combinations of any of the above should also be included within the scope of computer-readable media.

[0141] The memory 912 includes computer storage media in the form of volatile or non-volatile memory. The memory may be removable, non-removable, or a combination thereof. Examples of hardware devices include solid state memory, hard drives, optical disk drives, and the like. The computing device 900 includes one or more processors that read data from various entities, such as the memory 912 or the I / O components 920. The presentation components 916 present data indications to a user or other devices. Examples of presentation components include a display device, a speaker, a printing component, a vibrating component, and the like.

[0142] The I / O ports 918 allow the computing device 900 to be logically coupled to other devices, including I / O components 920, some of which may be built-in. Example components include microphones, joysticks, game pads, satellite dishes, scanners, printers, wireless devices, and the like.

[0143] Having identified various components in the present disclosure, it should be understood that any number of components and arrangements may be utilized to achieve the desired functionality within the scope of the present disclosure. For example, the components in the illustrated embodiments are shown in lines for conceptual clarity. Other arrangements of these and other components may also be implemented. For example, while some components are shown as single components, many of the elements described herein may be implemented as separate or distributed components or in conjunction with other components, and in any suitable combination and location. Some elements may be omitted entirely. Moreover, various functions described herein as being performed by one or more entities may be performed by hardware, firmware, and / or software, as described below. For example, various functions may be performed by a processor executing instructions stored in a memory. Thus, other arrangements and elements (e.g., machines, interfaces, functions, orders, and groupings of functions, etc.) in addition to or in place of those illustrated can be used.

[0144] The subject matter of the present invention is described herein with particularity to meet statutory requirements. However, the specification itself is not intended to limit the scope of this patent. Rather, the inventors contemplate that the claimed subject matter may be otherwise comprised, in conjunction with other current or future technologies, to include different steps or combinations of steps that are similar to those described in this document. Moreover, although the terms "step" and / or "block" may be used herein to imply different elements of a method used, this term should not be construed as implying any particular order within or between the various steps disclosed herein, unless and when the order of individual steps is explicitly described. For purposes of this disclosure, words such as "a" and "an" include the plural as well as the singular, unless otherwise indicated. Thus, for example, the requirement of "a feature" is met if there is one or more features.

[0145] The present invention has been described in relation to particular embodiments which are intended in all respects to be illustrative rather than restrictive, and alternative embodiments will become apparent to those skilled in the art to which the invention pertains without departing from the scope of the invention.

[0146] From the foregoing, it will be seen that this invention is well adapted to attain all of the ends and objectives set forth above, together with other advantages which are obvious and inherent to the system and method. It will be understood that certain features and subcombinations are of utility and may be employed independently of other features and subcombinations, which are contemplated by and within the scope of the appended claims.

[0147] The following embodiments represent exemplary embodiments of the concepts contemplated herein. Any one of the following embodiments may be combined in a multiple dependent manner so as to be dependent on one or more other clauses. Additionally, any combination of dependent embodiments (e.g., clauses that are explicitly dependent on a previous clause) may be combined while remaining within the scope of the aspects contemplated herein. The following clauses are exemplary and not limiting in nature.

[0148] Clause 1. A method comprising: collecting a set of electroretinogram (ERG) signals having measurement information related to electrical responses of cells in the retina; preprocessing the ERG signals by removing anomalies contained in the data of the ERG signals; extracting statistical and wavelet-based features from the preprocessed ERG signals; extracting features from the statistical and wavelet-based features based at least in part on using feature selection and extraction techniques; and generating a training data set using the features to train at least one machine learning model in glaucoma diagnosis prediction. In this manner, the exemplary embodiment provides technical improvements over conventional techniques by implementing a glaucoma diagnosis system that extracts more efficient and relevant features (e.g., advanced statistical and advanced wavelet-based features) for providing glaucoma diagnosis, glaucoma progression, and retinal ganglion cell count prediction.

[0149] Clause 2. The method of claim 1, further comprising: training a machine learning model to produce a binary classification prediction of glaucoma using the training dataset, evaluating the binary classification prediction using a performance evaluation metric, and retraining the machine learning model until a stopping criterion is reached.

[0150] Clause 3. The method of claim 1 or 2, further comprising the steps of training a machine learning model to generate a multi-class classification prediction related to a stage of glaucoma progression using the training dataset, evaluating the multi-class classification prediction using a performance evaluation metric, and retraining the machine learning model until a stopping criterion is reached.

[0151] Clause 4. The method of claim 1 or 2, or 3, further comprising: training a machine learning model to generate retinal ganglion cell (RGC) count predictions using the training dataset to provide a quantitative assessment of visual function; evaluating the ERG count predictions using a performance evaluation metric; and retraining the machine learning model until a stopping criterion is reached.

[0152] Clause 5. The method of claim 1 or 2, or 3, or 4, wherein the machine learning model is an artificial neural network.

[0153] Clause 6. The method of claim 1 or 2, or 3, or 4, or 5, wherein the wavelet-based features include coefficients from an autoregressive model describing the power law behavior at various resolutions and wavelet variances of the ERG signal.

[0154] Clause 7. The method of claim 1 or 2, or 3, or 4, or 5, or 6, wherein the wavelet-based features include a Shannon entropy value for a maximal overlap discrete wavelet packet transform (MOD-PWT).

[0155] Clause 8. The method of claim 1 or 2, or 3, or 4, or 5, or 6, or 7, wherein the wavelet-based features include multifractal wavelet leader estimates of the second cumulant of the scaling exponent and various Hölder exponents.

[0156] Clause 9. A system for detecting glaucoma, the system comprising at least one processor and one or more computer storage media having computer executable instructions stored thereon, the computer executable instructions, when executed by the at least one processor, cause the at least one processor to perform operations including: accessing a training dataset including electroretinogram (ERG) signals, each of the ERG signals being associated with a binary label selected from glaucoma and non-glaucoma; training a machine learning model on the training dataset to generate a trained machine learning model; obtaining ERG signal data related to a subject; providing the ERG signal data as an ERG signal input to the trained machine learning model; and receiving a classification from the machine learning model determined in response to the ERG signal input and related to the subject. In this manner, the exemplary embodiment provides technical improvements over conventional techniques by implementing a glaucoma diagnosis system that trains a machine learning model using more efficient and relevant features (e.g., advanced statistical features and advanced wavelet-based features) that provide glaucoma diagnosis, glaucoma progression, and retinal ganglion cell count predictions.

[0157] Clause 10. The system of claim 9, further comprising reducing the dimensionality of the ERG signals of the training data set by extracting ERG signal features.

[0158] Clause 11. The system of claim 9 or 10, wherein the machine learning model is a decision tree model, a discriminant model, a support vector machine, a nearest neighbor algorithm, or an ensemble classifier.

[0159] Clause 12. The system of claim 9 or 10, or 11, further comprising treating the subject for glaucoma based on the classification of glaucoma.

[0160] Clause 13. The system of claim 9 or 10, or 11, or 12, wherein training the machine learning model includes training the machine learning model to produce a binary classification prediction of glaucoma using the training dataset, evaluating the binary classification prediction using a performance evaluation metric, and retraining the machine learning model until a stopping criterion is reached.

[0161] Clause 14. The system of claim 9 or 10, or 11, or 12, or 13, wherein training the machine learning model includes training the machine learning model to produce a multi-class classification prediction related to a stage of glaucoma progression using the training dataset, evaluating the multi-class classification prediction using a performance evaluation metric, and retraining the machine learning model until a stopping criterion is reached.

[0162] Clause 15. The system of claim 9 or 10, or 11, or 12, or 13, or 14, wherein training the machine learning model includes training the machine learning model to produce retinal ganglion cell (ERG) count predictions using the training dataset to provide a quantitative assessment of visual function, evaluating the ERG count predictions using a performance evaluation metric, and retraining the machine learning model until a stopping criterion is reached.

[0163] Clause 16. The system of claim 9 or 10, or 11, or 12, or 13, or 14, or 15, wherein the training data set includes extracted statistical features and wavelet-based features based at least in part on using feature selection and extraction techniques on the ERG signals.

[0164] Clause 17. A non-transitory computer-readable storage medium comprising instructions stored thereon, the instructions, when executed by a processor, causing the processor to receive a request to provide a glaucoma diagnosis prediction generated by a glaucoma diagnosis system including a machine learning model, the glaucoma diagnosis system generating the glaucoma diagnosis prediction using a machine learning model trained using a training dataset with advanced features extracted from an electroretinogram ("ERG") signal, obtain ERG signal data based on the request, and generate the glaucoma diagnosis prediction based at least in part on inputting the ERG signal data into the machine learning model of a glaucoma diagnosis framework. In this manner, the exemplary embodiment provides technical improvements over conventional techniques by implementing a glaucoma diagnosis system providing a machine learning model trained using more efficient and relevant features (e.g., advanced statistical features and advanced wavelet-based features) to provide glaucoma diagnosis, glaucoma progression, and retinal ganglion cell count predictions.

[0165] Clause 18. The non-transitory computer-readable storage medium of claim 17, further comprising instructions for causing the processor to receive a second request to provide a glaucoma progression prediction generated by a glaucoma diagnostic system including a second machine learning model, wherein the glaucoma diagnostic system generates the glaucoma progression prediction using the second machine learning model trained using the training dataset, and generate the glaucoma progression prediction by inputting the ERG signal data into the second machine learning model of the glaucoma diagnostic framework.

[0166] Claim 19. The non-transitory computer-readable storage medium of claim 17 or 18, further comprising instructions for causing a processor to receive a second request to provide a retinal ganglion cell (RGC) count prediction generated by a glaucoma diagnostic system including a second machine learning model, where the glaucoma diagnostic system generates the RGC count prediction using the second machine learning model trained using the training dataset, and generate the RGC count prediction by inputting the ERG signal data into the second machine learning model of the glaucoma diagnostic framework.

[0167] Clause 20. The non-transitory computer-readable storage medium of claim 17 or 18 or 19, wherein the advanced features include extracted statistical features and wavelet-based features based at least in part on using feature selection and extraction techniques on the ERG signal.

Claims

1. 1. A method comprising: collecting a set of electroretinogram (ERG) signals having measurement information related to electrical responses of cells in a retina; preprocessing the ERG signals by removing anomalies contained within data of the ERG signals; extracting statistical and wavelet-based features from the preprocessed ERG signals; extracting features from the statistical and wavelet-based features based at least in part on using feature selection and extraction techniques; and generating a training dataset using the features for training at least one machine learning model for glaucoma diagnosis prediction.

2. 2. The method of claim 1, further comprising: training the machine learning model to produce a binary classification prediction of glaucoma using the training dataset; evaluating the binary classification prediction using a performance evaluation metric; and retraining the machine learning model until a stopping criterion is reached.

3. 2. The method of claim 1, further comprising: training the machine learning model to generate a multi-class classification prediction related to a stage of glaucoma progression using the training dataset; evaluating the multi-class classification prediction using a performance evaluation metric; and retraining the machine learning model until a stopping criterion is reached.

4. 2. The method of claim 1, further comprising: training the machine learning model to produce retinal ganglion cell (RGC) count predictions using the training dataset to provide a quantitative assessment of visual function; evaluating the ERG count predictions using a performance evaluation metric; and retraining the machine learning model until a stopping criterion is reached.

5. The method of claim 4 , wherein the machine learning model is an artificial neural network.

6. The method of claim 1 , wherein the wavelet-based features include coefficients from an autoregressive model describing power law behavior at various resolutions and wavelet variance of the ERG signal.

7. The method of claim 1 , wherein the wavelet-based features include a Shannon entropy value for a maximal overlap discrete wavelet packet transform (MOD-PWT).

8. The method of claim 1 , wherein the wavelet-based features include multifractal wavelet leader estimates of a second cumulant of a scaling exponent and various Holder exponents.

9. 1. A system for detecting glaucoma comprising: at least one processor; and one or more computer storage media having computer-executable instructions stored thereon that, when executed by the at least one processor, cause the at least one processor to perform operations including: accessing a training dataset including electroretinogram (ERG) signals, each of the ERG signals being associated with a binary label selected from glaucoma and non-glaucoma; training a machine learning model on the training dataset to generate a trained machine learning model; obtaining ERG signal data relating to a subject; providing the ERG signal data as an ERG signal input to the trained machine learning model; and receiving a classification from the machine learning model determined in response to the ERG signal input and relating to the subject.

10. 10. The system of claim 9, further comprising reducing the dimensionality of the ERG signals of the training data set by extracting ERG signal features.

11. 10. The system of claim 9, wherein the machine learning model is a decision tree model, a discriminant model, a support vector machine, a nearest neighbor algorithm, or an ensemble classifier.

12. 10. The system of claim 9, further comprising treating the subject for glaucoma based on the classification of glaucoma.

13. 10. The system of claim 9, wherein training the machine learning model includes training the machine learning model to produce a binary classification prediction of glaucoma using the training dataset, evaluating the binary classification prediction using a performance evaluation metric, and retraining the machine learning model until a stopping criterion is reached.

14. 10. The system of claim 9, wherein training the machine learning model includes training the machine learning model to produce a multi-class classification prediction related to a stage of glaucoma progression using the training dataset, evaluating the multi-class classification prediction using a performance evaluation metric, and retraining the machine learning model until a stopping criterion is reached.

15. 10. The system of claim 9, wherein training the machine learning model includes training the machine learning model to produce retinal ganglion cell (ERG) count predictions using the training dataset to provide a quantitative assessment of visual function, evaluating the ERG count predictions using a performance evaluation metric, and retraining the machine learning model until a stopping criterion is reached.

16. 10. The system of claim 9, wherein the training data set includes extracted statistical and wavelet-based features based at least in part on using feature selection and extraction techniques on the ERG signals.

17. 1. A non-transitory computer-readable storage medium comprising instructions stored thereon that, when executed by a processor, cause the processor to: receive a request to provide a glaucoma diagnosis prediction generated by a glaucoma diagnosis system including a machine learning model, wherein the glaucoma diagnosis system generates the glaucoma diagnosis prediction using the machine learning model trained using a training dataset along with advanced features extracted from electroretinogram ("ERG") signals; obtain ERG signal data based on the request; and generate the glaucoma diagnosis prediction based at least in part on inputting the ERG signal data into the machine learning model of the glaucoma diagnosis framework.

18. 20. The non-transitory computer-readable storage medium of claim 17, further comprising instructions for causing the processor to: receive a second request to provide a glaucoma progression prediction generated by the glaucoma diagnostic system including a second machine learning model, wherein the glaucoma diagnostic system generates the glaucoma progression prediction using the second machine learning model trained using the training dataset; and generate the glaucoma progression prediction by inputting the ERG signal data into the second machine learning model of the glaucoma diagnostic framework.

19. 20. The non-transitory computer-readable storage medium of claim 17, further comprising instructions for causing the processor to: receive a second request to provide a retinal ganglion cell (RGC) count prediction generated by the glaucoma diagnostic system including a second machine learning model, the glaucoma diagnostic system generating the RGC count prediction using the second machine learning model trained using the training dataset; and generate the RGC count prediction by inputting the ERG signal data into the second machine learning model of the glaucoma diagnostic framework.

20. 20. The non-transitory computer-readable storage medium of claim 17, wherein the advanced features include extracted statistical features and wavelet-based features based at least in part on using feature selection and extraction techniques on the ERG signals.

Images