Systems and methods for predicting occurrence and progression of glaucoma using retinal photographs

Abstract

Depth learning-based systems and methods for predicting and layering the risk of glaucoma onset and progression based on color fundus photographs (CFP) are disclosed. Methods are clinically validated by external population queues in which a machine learning classifier is applied to predict the likelihood of a patient's occurrence or progression of glaucoma in the future (e.g., over similar periods of time of several years), which has been trained using a dataset of CFP of a longitudinal patient queue, and which is used to predict the likelihood of the patient's occurrence or progression of glaucoma in the future (e.g., over similar periods of time of several years). The dataset is related to glaucoma development of individual patients in the queue over a period of time (e.g., during a few years of process).

Description

Background Art

[0001] Glaucoma is a major chronic eye disease characterized by optic nerve damage and visual field loss (1, 2). Its onset is usually insidious, with the risk of irreversible visual field loss before symptoms appear (3). Timely detection and treatment of glaucoma by lowering intraocular pressure (IOP) can reduce the risk of disease progression (4, 5). Predicting the onset and progression of glaucoma is a major clinical challenge. Previous studies have demonstrated that biological parameters such as baseline IOP, vertical cup-to-disc ratio, mean deviation (in Humphrey visual field testing), and pattern standard deviation are helpful in predicting the occurrence and progression of glaucoma (6-12). However, IOP measurements and visual field testing are not generally available in primary care settings. In contrast, color fundus photographs (CFPs) are widely available and rapidly acquired, with the potential to allow artificial intelligence (AI)-based diagnosis of optic nerve, retinal, and systemic diseases (including chronic kidney disease, diabetes) (13). Smartphones can also be adapted to capture CFPs, making them a promising tool for future disease screening (14, 15). Thus, it would be advantageous if the onset and progression of glaucoma could be based solely on CFP rather than relying on multiple testing modalities. Summary of the invention

[0002] In one aspect, a computer-implemented method is provided, comprising: receiving, using at least one computer processor, one or more color fundus photographs (CFPs) of a patient, and applying a machine learning classifier to classify the received CFPs of the patient to diagnose whether the patient has glaucoma, the machine learning classifier having been trained using a dataset of CFPs of a cohort of patients who have been classified as having glaucoma.

[0003] In another aspect, a computer-implemented method is provided, comprising: using at least one computer processor to: receive one or more color fundus photographs (CFPs) of a patient; and apply a machine learning classifier to predict the likelihood of the patient developing or progressing glaucoma in the future (e.g., within a similar time period of several years), the machine learning classifier having been trained using a dataset of CFPs of a longitudinal patient cohort, the dataset being associated with the development of glaucoma for each patient in the cohort over a time period (e.g., over the course of several years). This method can be combined with the aforementioned methods to determine whether a patient currently has glaucoma and to predict whether a patient will develop glaucoma in the future or whether the patient's existing glaucoma will progress in the future.

[0004] In some embodiments of the method, the machine learning classifier includes a segmentation module that segments anatomical structures including retinal blood vessels, macula, optic cup, and optic disc based on the received CFP.

[0005] In some embodiments, the segmentation module has been independently trained by manual annotation or segmentation of anatomical structures including retinal blood vessels, macula, optic cup, and optic disc.

[0006] In some embodiments, one or more CFPs of the received patient are obtained from fundus images of the patient taken by a smartphone.

[0007] In some embodiments, a dataset of CFPs of a longitudinal patient cohort has been stratified into a low-risk group and a high-risk group for glaucoma occurrence or progression.

[0008] In some embodiments, the method further includes using at least one computer processor to classify the patient as belonging to a low-risk group or a high-risk group for future glaucoma occurrence or progression.

[0009] In some embodiments, the machine learning classifier includes a deep learning model, which may include the architecture of a convolutional neural network (CNN). Brief Description of the Drawings

[0010] Figure 1 : Development and validation of a deep learning system for glaucoma diagnosis, glaucoma occurrence, and progression prediction. A: Data collection and ground truth labeling for CFP-based glaucoma diagnosis; B: Pipeline for glaucoma diagnosis; C: Data collection and ground truth labeling for glaucoma occurrence and progression; D: Pipeline for predicting glaucoma development and progression. CFP: Color fundus photograph; VF: Visual field.

[0011] Figure 2 : Area under the receiver operating characteristic (AUROC) curve of the AI model for predicting glaucoma onset. A to C: Prediction performance of the AI model in the validation set (n = 1191), external test set 1 (n = 955), and external test set 2 (n = 719).

[0012] Figure 3 : Area under the receiver operating characteristic (AUROC) curve of the AI model for predicting glaucoma progression. A to C: Prediction performance of the AI model in the validation set (n = 422), external test set 1 (n = 337), and external test set 2 (n = 513).

[0013] Figure 4: Salient maps of the deep learning model. Visual interpretation of key regions of the model for diagnostic prediction. a and b: Heatmaps of typical samples of eyes with (a) and without (b) glaucoma development; c and d: Heatmaps of typical samples of eyes with (c) and without (d) glaucoma progression. In both tasks, the salient maps indicate that the AI model focuses on the optic disc margin and the regions along the superior and inferior vascular arches, which is consistent with the clinical method, whereby nerve fiber loss at the superior or inferior disc margin provides key diagnostic clues. AI-based prediction also seems to involve retinal arterioles and venules.

[0014] Figure 5 : Detailed architecture of PredictNet. PredictNet consists of an image preprocessing and analysis module. First, in the preprocessing stage, the original fundus image is enhanced using contrast-limited adaptive histogram equalization (CLAHE) and color normalization (NORM). A trained Unet is used to semantically segment important retinal structures, including the optic disc, optic cup, macula, and blood vessels. The multi-channel anatomical masks output from the Unet are merged into a single-channel mask, which is then fused with the green and red channels of the CLAHE image to form a CLAHE-normalized attention-based image. The NORM image is fused with the green and red channels of the original image to form an anatomy-based attention image. Second, in the analysis stage, the CLAHE-normalized attention-based image and the anatomy-based attention image are fed into two convolutional neural networks (i.e., ConvNet-based model 1 and model 2). The final prediction is obtained by integrating the two ConvNet-based models in a linear combination.

[0015] Figure 6 : Representative samples of the automatic segmentation of the optic disc, optic cup, macula, and blood vessels. a to d: Segmentation of the optic disc, optic cup, macula, and blood vessels. From left to right: original image, manual segmentation, automatic segmentation

[0016] Figure 7 : Confusion matrix, which shows the prediction accuracy of the model across datasets in the prediction of glaucoma onset.

[0017] a to c: Prediction accuracy in the validation set and external test sets 1 and 2. 0 and 1 are the labels for eyes without and with glaucoma occurrence, respectively.

[0018] Figure 8 : Kaplan-Meier curve for predicting the accuracy of glaucoma development.

[0019] a to c: Prediction accuracy in the validation set and external test sets 1 and 2. The blue and green survival curves represent the high-risk and low-risk subgroups stratified by the upper quartile. The one-sided log-rank test between the two subgroups was used to calculate the P-value, and all P-values were less than 0.001.

[0020] Figure 9 : Distribution of the risk scores of the prediction model across all datasets in the prediction of glaucoma onset.

[0021] The black dotted line represents the low-high risk score threshold (0.3561). The red bars represent the proportion of eyes without glaucoma development, while the blue bars represent the proportion of eyes with glaucoma development. a to c: Glaucoma onset in the validation set and external test sets 1 and 2.

[0022] Figure 10 : Confusion matrix showing the prediction accuracy of the model across datasets in the prediction of glaucoma progression.

[0023] a to c: Prediction accuracy in the validation set and external test sets 1 and 2. 0 and 1 are the labels for eyes without and with glaucoma progression, respectively.

[0024] Figure 11 : AUC curve of the model based on clinical metadata for the prediction of glaucoma progression. a to c: Prediction performance of the model in the validation set and external test sets 1 and 2.

[0025] Figure 12 : Kaplan-Meier curves for predicting the accuracy of glaucoma progression. a to c: Prediction accuracy in the validation set and external test sets 1 and 2. The blue and green survival curves represent the high-risk and low-risk subgroups stratified by the upper quartile. The one-sided log-rank test between the two subgroups was used to calculate the P-value, and all P-values were less than 0.001.

[0026] Figure 13 : Distribution of the risk scores of the prediction model across all datasets in the prediction of glaucoma progression. The black dotted line represents the low-high risk score threshold (2.6352). The red bars represent the proportion of eyes without glaucoma progression, while the blue bars represent the proportion of eyes with glaucoma progression. a to c: Glaucoma onset in the validation set and external test sets 1 and 2.

[0027] Figure 14 : Salience map of the deep learning model for diagnosing glaucoma.

[0028] Visual interpretation of key regions of a model for diagnostic prediction. a and b: Heatmaps of typical samples of eyes with (a) and without (b) possible glaucoma. The saliency maps indicate that the AI model focuses on the optic disc margin and regions along the superior and inferior vascular arches, which is consistent with clinical methods, whereby nerve fiber loss at the superior or inferior disc margin provides key diagnostic clues. Detailed implementation

[0029] According to some aspects, diagnostic systems, computing devices, and computer-implemented methods are disclosed herein that diagnose glaucoma and predict glaucoma onset or progression in a patient (e.g., a human individual) based on color fundus photographs of the patient without using a biopsy by using a machine learning framework. In some embodiments, the machine learning framework utilizes a deep learning model such as a neural network.

[0030] In one aspect, a computer-implemented method is provided, comprising: using at least one computer processor to receive one or more color fundus photographs (CFPs) of a patient, and applying a machine learning classifier to classify the received CFPs of the patient to diagnose whether the patient has glaucoma, the machine learning classifier having been trained using a dataset of CFPs of a cohort of patients who have been classified as having glaucoma.

[0031] In another aspect, a computer-implemented method is provided, comprising: using at least one computer processor to: receive one or more color fundus photographs (CFPs) of a patient; and apply a machine learning classifier to predict the likelihood of glaucoma onset or progression in the patient in the future, the machine learning classifier having been trained using a dataset of CFPs of a longitudinal patient cohort (i.e., a dataset of CFPs taken over a period of time (e.g., several years) during which the patient may develop glaucoma). The method can be combined with the foregoing method to determine whether the patient currently has glaucoma and to predict whether the patient will develop glaucoma in the future or whether the patient's existing glaucoma will progress in the future.

[0032] In some embodiments of the method, the machine learning classifier includes a segmentation module that segments anatomical structures including retinal blood vessels, macula, optic cup, and optic disc from the received CFPs.

[0033] In some embodiments, the segmentation module has been independently trained by manual annotation or segmentation of anatomical structures including retinal blood vessels, macula, optic cup, and optic disc.

[0034] In some embodiments, the one or more CFPs of the received patient are obtained from fundus images of the patient taken by a smartphone.

[0035] In some embodiments, a dataset of CFP for a longitudinal patient cohort has been stratified into a low-risk group and a high-risk group for glaucoma onset or progression.

[0036] In some embodiments, the method further includes using at least one computer processor to classify a patient as belonging to a low-risk group or a high-risk group for future glaucoma onset or progression.

[0037] In some embodiments, the machine learning classifier includes a deep learning model, which may include an architecture of a convolutional neural network (CNN).

[0038] In some embodiments, the systems, devices, media, methods, and applications described herein include a digital processing device. For example, in some embodiments, the digital processing device is part of a point-of-care device integrated with the diagnostic software described herein. In some embodiments, the medical diagnostic device includes an imaging device, such as imaging hardware for taking a CRF (e.g., a camera, such as a camera of a smartphone). The device may include an optical lens and / or a sensor that acquires the CRF at a magnification of hundreds or thousands. In some embodiments, the medical imaging device includes a digital processing device configured to perform the methods described herein. In additional embodiments, the digital processing device includes one or more processors (or computer processors) or a hardware central processing unit (CPU) that performs the functions of the device. In other embodiments, the digital processing device further includes an operating system configured to execute executable instructions. In some embodiments, the digital processing device is optionally connected to a computer network. In additional embodiments, the digital processing device is optionally connected to the Internet such that it accesses the World Wide Web. In other embodiments, the digital processing device is optionally connected to a cloud computing infrastructure. In other embodiments, the digital processing device is optionally connected to an intranet. In other embodiments, the digital processing device is optionally connected to a data storage device. By way of non-limiting example, suitable digital processing devices include server computers, desktop computers, laptop computers, notebook computers, subnotebook computers, netbook computers, set-top box computers, handheld computers, Internet appliances, mobile smartphones, tablet computers, personal digital assistants, video game consoles, and vehicles. Those skilled in the art will recognize that many smartphones are suitable for the systems described herein.

[0039] In some embodiments, the systems, media, methods, and applications described herein include one or more non-transitory computer-readable storage media encoded with a program, the program including instructions executable by an operating system of an optionally networked digital processing device. In additional embodiments, the computer-readable storage media is a tangible component of the digital processing device. In other embodiments, the computer-readable storage media is optionally removable from the digital processing device. In some embodiments, by way of non-limiting example, the computer-readable storage media includes CD-ROMs, DVDs, flash devices, solid-state memories, disk drives, tape drives, optical disc drives, cloud computing systems and services, and the like. In some cases, the program and instructions are permanently, substantially permanently, semi-permanently, or non-transitorily encoded on the media.

[0040] In some embodiments, the systems, media, methods, and applications described herein include at least one computer program or its use. The computer program includes a sequence of instructions written to perform a specified task and executable in a CPU of a digital processing device. The computer-readable instructions may be implemented as program modules that perform a particular task or implement a particular abstract data type, such as functions, objects, application programming interfaces (APIs), data structures, and the like. Based on the disclosure provided herein, those skilled in the art will recognize that the computer program may be written in various versions of various languages.

[0041] The functionality of the computer-readable instructions may be combined or distributed according to desire in various environments. In some embodiments, the computer program includes a sequence of instructions. In some embodiments, the computer program includes multiple sequences of instructions. In some embodiments, the computer program is provided from one location. In other embodiments, the computer program is provided from multiple locations. In various embodiments, the computer program includes one or more software modules. In various embodiments, the computer program partially or fully includes one or more web applications, one or more mobile applications, one or more standalone applications, one or more web browser plugins, extensions, add-ons, or add-in devices, or combinations thereof. In some embodiments, the computer program includes a web application. Based on the disclosure provided herein, those skilled in the art will recognize that in various embodiments, the web application utilizes one or more software frameworks and one or more database systems.

[0042] In some embodiments, the systems, devices, media, methods, and applications described herein include software, server, and / or database modules, or their use. Based on the disclosure provided herein, software modules are created using known machines, software, and languages in the art through techniques known to those skilled in the art. The software modules disclosed herein are implemented in a variety of ways. In various embodiments, a software module includes a file, a code segment, a programming object, a programming structure, or a combination thereof. In various additional embodiments, a software module includes multiple files, multiple code segments, multiple programming objects, multiple programming structures, or a combination thereof. In various embodiments, by way of non-limiting example, one or more software modules include web applications, mobile applications, and stand-alone applications. In some embodiments, a software module is in one computer program or application. In other embodiments, a software module is in more than one computer program or application. In some embodiments, a software module is hosted on one machine. In other embodiments, a software module is hosted on more than one machine. In additional embodiments, a software module is hosted on a cloud computing platform. In some embodiments, a software module is hosted on one or more machines at one location. In other embodiments, a software module is hosted on one or more machines at more than one location.

[0043] Example

[0044] Deep learning techniques have been widely used in glaucoma diagnosis (16 - 19). However, there is no clinically validated algorithm for predicting the onset and progression of glaucoma. The aim of this study was to develop a clinically feasible deep learning system for diagnosing glaucoma based on CFP by validating the performance in an external population cohort ( Figure 1 A and Figure 1 B) and predicting the risk of glaucoma onset and progression ( Figure 1 C and Figure 1 D). The disclosed AI systems and methods are capable of detecting features in baseline fundus photographs that are not recognizable to the human eye, and predicting which patients will progress to glaucoma within 5 years. In addition, the AI system can be deployed at the point of care via smartphone imaging to enable widespread accessible remote glaucoma screening in the future.

[0045] Methods

[0046] Dataset characteristics

[0047] Glaucoma diagnosis cohort. In these initial cohorts, we specifically sought patients who visited ophthalmologists specializing in glaucoma and anterior segment diseases. The patient populations seen by these ophthalmologists were highly enriched for POAG patients (37, 38). We deliberately selected these initial cohorts to ensure that we could collect sufficient POAG patients as well as non-glaucoma control patients (such as cataract patients) who were otherwise appropriately matched to develop an AI-based POAG diagnosis (Table 1). Training and validation data in glaucoma diagnosis were collected from community cohorts and ophthalmology clinics in Guangzhou. To test the generality of the AI model, two independent datasets obtained from Beijing and Kashgar were used as external test sets. External test set 1 was collected from patients who had annual health examinations in Beijing, while external test set 2 was obtained through smartphones at local ophthalmology clinics in Kashgar, Xinjiang Uygur Autonomous Region.

[0048] Glaucoma incidence prediction cohort. Training and validation data in predicting glaucoma incidence were collected from the Guangzhou community cohort. To test the generality of the AI model, two independent datasets obtained from Beijing and the Guangzhou community were used as external test sets. Our longitudinal cohort for POAG incidence prediction had a POAG frequency between 1% - 2%, which was well within the criteria for the POAG prevalence in the general population.

[0049] Glaucoma progression prediction cohort. Training and validation data for predicting glaucoma progression were collected from a POAG cohort at the Zhongshan Ophthalmic Center in Guangzhou. To test the generality of the AI model, two independent cohorts consisting of PACG and POAG eyes from the Zhongshan Ophthalmic Center were used as external test sets.

[0050] Image quality control and labeling

[0051] First, all images were de-identified to remove any patient-related information. Fifteen ophthalmologists with at least 10 years of clinical experience were recruited to label CFP. First, they were asked to exclude images of poor quality. The criteria included: 1) the optic disc or macula was not fully visible; 2) the image was blurred due to refractive media. 7.1% of CFP were excluded due to poor quality. Second, the graders were asked to assign glaucoma or non-glaucoma labels to each CFP. Third, each glaucoma with longitudinal follow-up data was further analyzed to determine whether there was progression based on visual field reports during the follow-up period. Visual fields with a fixation loss of less than 20%, a false positive rate of less than 15%, and a false negative rate of less than 33% were included. Each CFP or visual field report was independently evaluated by three ophthalmologists, and the ground truth was determined by the consensus of the three ophthalmologists.

[0052] Criteria for glaucoma diagnosis and progression

[0053] Glaucoma was diagnosed using the criteria from previous population-based studies (20 - 22). Glaucomatous optic neuropathy was defined as the presence of a vertical cup-to-disc ratio ≥ 0.7, RNFL defect, rim width ≤ 0.1 disc diameter, and / or disc hemorrhage. If one of the above criteria was met, the eye was labeled as a possible glaucoma.

[0054] Glaucoma progression was determined based on changes in the visual field (23). All visual field tests were performed in the 24 - 2 standard mode (Carl Zeiss Meditec, La Jolla, California, USA) using a Humphrey visual field analyzer. At least three visual field locations that were worse than the baseline at the 5% level in two consecutive reliable visual fields or at least three visual field locations that were worse than the baseline at the 5% level in two consecutive reliable visual fields were considered as progression (23). The progression time was defined as the time from the baseline to the first visual field that confirmed progression. Three ophthalmologists examined each visual field report separately to determine progression.

[0055] Manual segmentation of anatomical structures

[0056] We randomly selected 2000 CFP for manual segmentation of anatomical structures, including the optic disc, optic cup, macula, and blood vessels. Two ophthalmologists independently annotated the CFP at the pixel level, and the final standard reference for the annotation was determined by the average of these two independent annotations.

[0057] Model design for glaucoma prediction and eye disease diagnosis

[0058] First, we developed the AI model DiagnoseNet to identify CFP as glaucoma or non - glaucoma. DiagnoseNet is a pipeline consisting of modules for segmentation and diagnosis. First, in the segmentation module, Unet (39) was used to semantically segment fundus images to generate four anatomical structures: retinal blood vessels, macula, optic cup, and optic disc. Then, the segmentation data was merged into a single - channel - focused attention layer through an element - wise bit - or operation on the four anatomical structures, which replaced the blue channel of the CFP to form a new CFP image. The backbone of the diagnosis module is EfficientNet - B0, where the last fully - connected layer was replaced by a dense layer with two output units initialized with random values, and the initial weights of the other layers were determined from the pre - trained settings of ImageNet ( Figure 1 B).

[0059] Then, we created the pipeline PredictNet to predict the onset and progression of glaucoma. PredictNet pre - processes and analyzes CFP data ( Figure 1)。First, in the preprocessing stage, the original fundus images are enhanced using Contrast Limited Adaptive Histogram Equalization (CLAHE) and color normalization (NORM). The trained Unet is used to semantically segment important retinal structures, including the optic disc, optic cup, macula, and blood vessels. The multi-channel anatomical masks output from the Unet (39) are merged into a single-channel mask, which is then fused with the green and red channels of the CLAHE image to form a CLAHE-normalized attention-based image. The NORM image is fused with the green and red channels of the original image to form an anatomy-based attention image. Second, in the analysis stage, the CLAHE-normalized attention-based image and the anatomy-based attention image are fed into two convolutional neural networks (i.e., ConvNet-based Model 1 and Model 2). Each ConvNet-based model consists of a feature extraction network and a classification network module. The feature extraction network consists of 3 convolutional blocks, which are composed of Convolution2D layers, batch normalization layers, LeakReLu layers, and MaxPooling2D layers in series, while the classification network is composed of two dense layers in series. The GlobalMaxPooling2D layer is used to connect between the feature extraction network and the classification network module. The final prediction is obtained by integrating the two ConvNet-based models in a linear combination. In the last step, PredictNet generates the probability (P) of glaucoma occurrence or progression between 0 and 1. The P is converted into a z-score using the following formula: where, represents the average P of each dataset. Then, we obtain the final standard score by adding 1 to all z-scores because some z-scores are below zero.

[0060] The model was developed using Python (version 3.8.6) and TensorFlow (version 2.1.0).

[0061] Interpretation of the AI model

[0062] Gradient-weighted class activation mapping (Grad-CAM) (40) is used to highlight the class discriminative regions in the images for making decisions of interest. We created heatmaps generated from the CFP, which indicate the key regions where the AI model classifies the CFP into the low-risk group and the high-risk group.

[0063] Statistics

[0064] Demographic characteristics of study participants were expressed as mean ± standard deviation (SD) for continuous data and frequency (percentage) for categorical variables. AUC, sensitivity, and specificity with 95% confidence intervals (CIs) were implemented to evaluate the performance of the algorithms. Sensitivity and specificity were determined by the threshold selected in the validation set. Survival curves were constructed for different risk groups, and the significance of differences between groups was tested by the log-rank test. The predictive performance of the AI model and the metadata model was performed using the DeLong test. All tested hypotheses were two-sided, and a p-value of less than 0.05 was considered significant. All statistical analyses were performed using R (version 4.0).

[0065] Study approval

[0066] Institutional review board and ethics committee approval was obtained at all sites, and all participants signed consent forms. All images were uploaded to a Health Insurance Portability and Accountability Act (HIPAA)-compliant cloud server for further grading.

[0067] Results

[0068] Definition of glaucoma, its occurrence, and progression

[0069] Diagnostic criteria for possible glaucoma based on CFP were created after a publicly available population-based study: Glaucomatous optic neuropathy was defined by the presence of a vertical cup-to-disc ratio ≥0.7, retinal nerve fiber layer (RNFL) defect, disc margin width ≤0.1 disc diameter, and / or disc hemorrhage (20 - 22). Glaucoma occurrence was defined when the baseline CFP was non-glaucomatous but the eye became a possible glaucoma during the follow-up period.

[0070] When glaucoma progression was suspected, Humphrey visual fields performed in the standard 24 - 2 pattern were analyzed (23). Glaucoma progression was defined by at least three visual field test points that were worse than baseline at the 5% level in two consecutive reliable visual field tests or at least three visual field locations that were worse than baseline at the 5% level in two subsequent consecutive reliable visual field tests (23). The time to progression was defined as the time from baseline to the first visual field test report that confirmed glaucoma progression following the above criteria. The true value of clinical progression was defined by the consensus agreement of three ophthalmologists who independently evaluated each visual field report.

[0071] Image dataset and patient characteristics

[0072] We established a large dataset consisting of CFP and visual fields collected in Guangzhou, Beijing, and Kashgar. Demographic and clinical information of study participants was summarized in Table 1. The data was randomly divided into mutually exclusive sets for training, validating, and externally testing the AL algorithm.

[0073] In the first task, we developed a model for diagnosing possible glaucoma based on 31,040 CFP images. In this task, 31,040 images from 14,905 individuals (divided into training: 20,872; validation: 3,182; external test 1: 6,162; external test 2: 824) were collected from the glaucoma and anterior segment disease ophthalmology outpatient clinics. 32.8% (10,175) of the images were diagnosed with possible glaucoma. The training and validation datasets were obtained from individuals in the glaucoma and anterior segment disease department of Zhongshan Ophthalmic Center in Guangzhou, China. External test set 1 was collected from patients in the glaucoma and anterior segment disease outpatient clinic of Jidong Hospital near Beijing. To further test the generality of the AI model, we verified its performance on CFP obtained via smartphones in Kashgar.

[0074] In the second task, we developed a model for predicting future glaucoma occurrence based on data from three longitudinal cohorts. We included a total of 13,222 eyes from 7,127 participants (training: 10,357, validation: 1,191, external test 1: 955, external test 2: 719), and all participants were diagnosed as non-glaucoma at baseline. The training and validation datasets were obtained from individuals who had annual health examinations in Guangzhou, while external test set 1 was from individuals who had annual health examinations in Beijing, and external test set 2 was from a community cohort in Guangzhou. The average follow-up duration across the datasets was 47.8 to 56.6 months. The incidence of glaucoma across the datasets was 1.1% to 2.0%.

[0075] In the third task, we developed a model for predicting glaucoma progression based on CFP from an existing glaucoma cohort. In this task, 4,275 eyes from 2,219 glaucoma patients were included (training: 3,003, validation: 422, external test 1: 337; external test 2: 513), and all patients had been diagnosed with glaucoma optic neuropathy at baseline. The training and validation datasets were obtained from a primary open-angle glaucoma (POAG) cohort at Zhongshan Ophthalmic Center. To further test the generality of the AI model for different subtypes of glaucoma, external test set 1 was collected from another POAG cohort, and external test set 2 was collected from the chronic primary angle-closure glaucoma (PACG) cohort at Zhongshan Ophthalmic Center. The average follow-up duration across the datasets was 34.8 to 41.7 months. And the proportion of glaucoma progression across the datasets was 6% to 13.5% (Table 1).

[0076] Design of the diagnostic algorithm (DiagnoseNet) and the prediction algorithm (PredicNet)

[0077] First, we developed the diagnostic algorithm DiagnoseNet for possible glaucoma ( Figure 1B). Briefly, DiagnoseNet consists of two main modules (a segmentation module and a diagnosis module). The CFP is semantically segmented by the segmentation module with four anatomical structures, including retinal blood vessels, macula, optic cup, and optic disc. The diagnosis module outputs a probability score for glaucoma.

[0078] Then, we designed a pipeline PredictNet for predicting the occurrence and progression of glaucoma. Briefly, PredictNet also consists of two main modules (a segmentation module and a prediction module). The segmentation module is the same as the one in DiagnoseNet. The prediction module generates a risk score for future glaucoma occurrence or progression ( Figure 1 D and Figure 5 ).

[0079] The diagnosis and prediction algorithms share the same segmentation module. The segmentation module is independently trained based on the manual annotations of the optic disc (1853 images), optic cup (1860 images), macula (1695 images), and blood vessels (160 images). The segmentation module shows excellent segmentation performance for the above anatomical structures, and achieves IOU of 0.847, 0.669, 0.570, and 0.538 for optic disc, optic cup, macula, and blood vessel segmentation respectively. Representative samples of the segmentation are shown in Figure 6 ...

[0080] Diagnostic performance of the AI model based on CFP captured by smartphone

[0081] To demonstrate the potential of deploying our AI model in routine healthcare, we developed and tested an AI model to diagnose possible glaucoma based on CFP not only from fundus cameras but also from smartphones. As shown in Table 2, in this validation dataset, the AI model achieved an AUC of 0.97 (0.96 - 0.97), a sensitivity of 0.98 (0.97 - 0.99), and a specificity of 0.82 (0.80 - 0.83) for differentiating glaucoma from non - glaucoma. To evaluate the generality of the algorithm, the AI model was tested on two external datasets. In external test set 1, the AI model achieved an AUC of 0.94 (0.93 - 0.94), a sensitivity of 0.89 (0.87 - 0.90), and a specificity of 0.83 (0.81 - 0.84). In external test set 2 obtained using smartphones, the AI model achieved an AUC of 0.91 (0.89 - 0.93), a sensitivity of 0.92 (0.88 - 0.96), and a specificity of 0.71 (0.67 - 0.74).

[0082] Predicting glaucoma occurrence using a longitudinal cohort

[0083] We investigated the predictive performance of an AI model for the development of glaucoma in non-glaucoma individuals over a four- to five-year period. Over a four- to five-year period, a total of 158 eyes developed glaucoma. In the validation set, the AI model achieved an AUC of 0.90 (0.81 - 0.99), a sensitivity of 0.84 (0.82 - 0.87), and a specificity of 0.82 (0.57 - 0.96) for predicting glaucoma onset (Table 2 and Figure 2 ). The AI model showed good generalizability in the external test sets, achieving an AUC of 0.89 (0.83 - 0.95), a sensitivity of 0.84 (0.81 - 0.86), a specificity of 0.68 (0.43 - 0.87), and an AUC of 0.88 (0.79 - 0.97), a sensitivity of 0.84 (0.81 - 0.86), a specificity of 0.80 (0.44 - 0.97) in external test sets 1 and 2, respectively (Table 2, Figure 2 and Figure 7 ).

[0084] There was a significant difference in the incidence of glaucoma between the low-risk and high-risk groups. In the low-risk and high-risk groups of the validation set, external test set 1, and external test set 2, the incidences were 0.2% and 5.0%, 0.6% and 5.6%, and 0.4% and 4.1%, respectively. We used the Kaplan-Meier method to stratify healthy individuals into two risk categories (low risk or high risk) of developing glaucoma based on four- to five-year longitudinal data on glaucoma development. The upper quartile of the predicted risk scores from the model in the validation set was used to create the thresholds for the high-risk and low-risk groups in the Kaplan-Meier curves and log-rank tests. In the external test sets, a significant separation between the low-risk and high-risk groups was achieved (both P < 0.001, Figure 8 ).

[0085] Figure 9 The distributions of the risk scores and thresholds (upper quartiles) for the low-risk and high-risk groups across the validation set and external test sets are presented. As shown, the threshold (risk score of 0.3561, red dotted line) well defines the boundary separating individuals who are likely and unlikely to develop glaucoma over a four- to five-year period.

[0086] The AI model did not show statistically significant performance differences between subgroups stratified by age (≥60 years vs. <60 years), gender (male vs. female), and severity of glaucoma (mean deviation > -6 dB vs. < -6 dB).

[0087] Predicting Glaucoma Progression Using a Longitudinal Cohort

[0088] We investigated the predictive performance of an AI model for glaucoma progression over a three- to four-year period. A total of 444 POAG eyes had progression over the three- to four-year period. In the validation set, the AI model achieved an AUC of 0.91 (0.88 - 0.94), a sensitivity of 0.83 (0.79 - 0.87), and a specificity of 0.79 (0.66 - 0.89) for predicting glaucoma progression (Table 2 and Figure 3 ). To validate the generality of the AI model in predicting multi-mechanism glaucoma progression, we further tested its predictive performance in two independent cohorts of PACG (external test set 1) and POAG (external test set 2). The AI model achieved excellent predictive performance with an AUC of 0.87 (0.81 - 0.92), a sensitivity of 0.82 (0.78 - 0.87), and a specificity of 0.59 (0.39 - 0.76) in external test set 1 and an AUC of 0.88 (0.83 - 0.94), a sensitivity of 0.81 (0.77 - 0.84), and a specificity of 0.74 (0.55 - 0.88) in external test set 2 (Table 2, Figure 3 and Figure 10 ).

[0089] We also trained a prediction model using only baseline clinical metadata (age, gender, intraocular pressure, mean deviation, pattern standard deviation, hypertension or diabetes status) to predict progression, which yielded AUCs of 0.76 (0.70 - 0.82), 0.73 (0.66 - 0.79), and 0.44 (0.33 - 0.54) in the validation set, external test set 1, and external test set 2, respectively ( Figure 11 ). The performance of the AI model was significantly superior to that of the prediction model based on the baseline metadata in the above datasets (all P < 0.001).

[0090] There were significant differences in the proportions of eyes with glaucoma progression between the low-risk and high-risk groups. In the low-risk and high-risk groups of the validation set, external test set 1, and external test set 2, the incidences were 3.8% and 42.4%, 4.5% and 23.9%, and 2.0% and 19.8%, respectively. Then, we performed Kaplan-Meier analysis to stratify glaucoma into two risk categories (low risk or high risk) of glaucoma progression based on three- to four-year longitudinal data on glaucoma progression. The upper quartile of the predicted risk scores from the model in the validation set was used to create the Kaplan-Meier curves and the threshold for the high-risk and low-risk groups in the log-rank test. In the external test sets, significant separation of the low-risk and high-risk groups was achieved (both P < 0.001, Figure 12 ).

[0091] Figure 13The distributions of the risk scores and thresholds (upper quartiles) of the low- and high-risk groups across the validation set and the external test set are presented. As shown, the threshold (risk score of 2.6352, red dotted line) well defines the boundary separating glaucomas that are likely and unlikely to progress over a three- to four-year period.

[0092] In addition to the AUCs of the severe and less severe subgroups in the validation and external test set 1, the AI model did not show statistical significance in all subgroups stratified by age of glaucoma (≥60 years vs. <60 years), gender (male vs. female), and severity (mean deviation > -6 dB vs. < -6 dB).

[0093] Visualization of evidence for predicting glaucoma onset and progression

[0094] To improve the interpretability of the AI model and illustrate the key regions for AI-based prediction, we used Gradient-weighted Class Activation Mapping (Grad-CAM) to generate the key regions in the CFP, which are used for diagnosing glaucoma, predicting glaucoma onset and progression. Figure 14 Representative cases of DiagnoseNet and their corresponding saliency maps are presented. The representative cases and their corresponding saliency maps are presented in Figure 14 (DiagnoseNet) and Figure 4 (PredictNet), respectively. The saliency maps show that the AI model focuses on the optic disc margin and the regions along the superior and inferior vascular arches, which is consistent with the clinical method, whereby nerve fiber loss at the superior or inferior disc margin provides key diagnostic or predictive clues. This would suggest that the AI model learns clinically relevant knowledge in assessing glaucoma diagnosis and progression. The AI-based prediction also seems to involve the retinal arterioles and venules, thereby suggesting a potential association between vascular health and the etiology of chronic open-angle glaucoma.

[0095] Discussion

[0096] More than 60 million people worldwide have glaucoma, and the number is expected to increase to 110 million by 2040 (24). Due to its insidious onset and variable progression, the monitoring of glaucoma diagnosis and treatment can be challenging and clinically time-consuming. Glaucoma screening is not universal worldwide, resulting in diagnostic delays and severe irreversible vision loss. Therefore, to facilitate early intervention, there is a high clinical need for efficient and reliable AI models to help identify high-risk individuals for glaucoma development and progression in the population.

[0097] Deep learning algorithms have been widely used in glaucoma diagnosis research (16 - 19) and have achieved outstanding diagnostic performance in detecting glaucoma. However, few studies have explored the efficacy of deep learning in predicting glaucoma onset and progression (25 - 29). In this study, our AI model demonstrated excellent glaucoma diagnostic performance for CFP, which includes photos taken with a smartphone camera using an adapter that can significantly broaden its application in the point - of - care setting. Compared with traditional statistical models (30 - 34) (such as glaucoma probability score and Moorfield regression analysis), several studies using deep learning models have achieved comparable or even better predictive performance (25 - 27). Thakur et al. developed an AI model that predicted glaucoma development approximately 1 to 3 years before clinical onset and achieved a maximum AUC of 0.88. However, these deep learning models have some limitations. First, this application was limited to onset prediction without progression prediction, which is an essential part of glaucoma management. Second, the data mainly came from hospitals or clinical trials rather than community groups, including many eyes diagnosed with ocular hypertension (elevated intraocular pressure without optic neuropathy) rather than glaucoma (25). Third, there was a lack of external validation data to demonstrate the generalizability of the model in the community.

[0098] Compared with previous studies, our study had the following advantages. First, we developed an AI model for glaucoma diagnosis, onset, and progression prediction. In the external test set, the model achieved excellent predictive performance in identifying high - risk individuals who developed glaucoma or had glaucoma progression. Second, the data in glaucoma onset prediction came from community screening settings, which better reflected the distribution characteristics of glaucoma in the population and promoted the generalizability of the model. The results in the external dataset showed that the AI model achieved excellent predictive performance for glaucoma development, thus demonstrating the strong generalizability and reliability of the AI model. Third, all patients in the glaucoma cohort for the progression prediction task received medications to reduce IOP because both the enrollment and their IOP values were controlled within the normal range. This indicates that our prediction model can identify high - risk patients who will experience glaucoma progression even with reasonably controlled IOP and facilitate timely interventions such as glaucoma surgery to save vision. Fourth, the AI model based on the structural data from CFP achieved high predictive accuracy for glaucoma progression, as determined by the gold standard of visual field test results. Visual field testing can reveal functional damage to the optic nerve and is the gold standard for monitoring glaucoma progression (35). As demonstrated in the glaucoma progression prediction task, the AI model successfully identified high - risk eyes with progressive functional deterioration from baseline CFP with high sensitivity. Additionally, the AI model showed similar predictive performance in different subtypes of glaucoma, including POAG and PACG, which share similar optic nerve structure and functional damage.

[0099] Our study has the following limitations. First, the input data for our AI model was only CFP. Clinical glaucoma evaluation typically requires a comprehensive analysis of multiple modalities (such as clinical examination, optic nerve head imaging, and visual field testing) to determine glaucoma subtypes and any progression. Our study selected CPF as the only input because of their high feasibility and wide availability. Future studies can consider incorporating other data modalities to further improve the predictive performance of the algorithm. Second, only high-quality CFP were included in the study, which limits the application of the AI model in eyes with media opacities that preclude obtaining clear CFP. Third, due to the limited prevalence of glaucoma in the general population (approximately 1% to 1.5% in the age group of 40 to 65 years)(36), the number of glaucoma cases was relatively small. To address this issue, we used a deep learning model with relatively few parameters. Fourth, the AI model showed different sensitivities and specificities across datasets, but had high AUC values. High sensitivity is more important for screening, and we can further improve the predictive performance of the AI model by using more training data in the future. Fifth, all data were from the Chinese population and need to be further validated in other populations.

[0100] In summary, our study demonstrated the feasibility of a deep learning system for disease onset and progression prediction. It provides the possibility of establishing a virtual glaucoma screening system.

[0101] References

[0102] 1. Jonas JB, Aung T, Bourne RR, Bron AM, Ritch R, and Panda-Jonas S. Glaucoma. Lancet. 2017;390(10108):2183 - 93.

[0103] 2. Zhang K, Zhang L, and Weinreb RN. Ophthalmic drug discovery: novel targets and mechanisms for retinal diseases and glaucoma. Nat Rev Drug Discov. 2012;ll(7):541 - 59.

[0104] 3. Weinreb RN, Aung T, and Medeiros FA. The Pathophysiology and Treatment of Glaucoma: A Review. JAMA. 2014;311(18):1901 - 11.

[0105] 4. Heijl A, Leske MC, Bengtsson B, Hyman L, Bengtsson B, Hussein M, et al. Reduction of Intraocular Pressure and Glaucoma Progression: Results From the Early Manifest Glaucoma Trial. Archives of Ophthalmology. 2002;120(10):1268 - 79.

[0106] 5. Jammal AA, Thompson AC, Mariottoni EB, Estrela T, Shigueoka LS, Berchuck SI, et al. Impact of Intraocular Pressure Control on Rates of Retinal Nerve Fiber Layer Loss in a Large Clinical Population. Ophthalmology. 2021;128(1):48 - 57.

[0107] 6. Gordon MO, Gao F, Huccker JB, Miller JP, Margolis M, Kass MA, et al. Evaluation of a Primary Open - Angle Glaucoma Prediction Model Using Long - term Intraocular Pressure Variability Data: A Secondary Analysis of 2 Randomized Clinical Trials. JAMA Ophthalmology. 2020;138(7):780 - 8.

[0108] 7. Medeiros FA, Alencar LM, Zangwill LM, Bowd C, Sample PA, and Weinreb RN. Prediction of functional loss in glaucoma from progressive optic disc damage. Archives of ophthalmology(Chicago, Ill:1960). 2009;127(10):1250 - 6.

[0109] 8. Coleman AL, Gordon MO, Beiser JA, and Kass MA. Baseline risk factors for the development of primary open-angle glaucoma in the Ocular Hypertension Treatment Study. American Journal of Ophthalmology. 2004;138(4):684 - 5.

[0110] 9. Validated Prediction Model for the Development of Primary Open-Angle Glaucoma in Individuals with Ocular Hypertension. Ophthalmology. 2007;114(1):10 - 9.e2.

[0111] 10. Song Y, Ishikawa H, Wu M, Liu Y - Y, Lucy KA, Lavinsky F, et al. Clinical Prediction Performance of Glaucoma Progression Using a 2 - Dimensional Continuous - Time Hidden Markov Model with Structural and Functional Measurements. Ophthalmology. 2018;125(9):1354 - 61.

[0112] 11. Daneshvar R, Yarmohammadi A, Alizadeh R, Henry S, Law SK, Caprioli J, et al. Prediction of Glaucoma Progression with Structural Parameters: Comparison of Optical Coherence Tomography and Clinical Disc Parameters. Am J Ophthalmol. 2019;208:19 - 29.

[0113] 12. De Moraes CG, Sehi M, Greenfield DS, Chung YS, Ritch R, and Liebmann JM. A Validated Risk Calculator to Assess Risk and Rate of Visual Field Progression in Treated Glaucoma Patients. Investigative Ophthalmology & Visual Science. 2012;53(6):2702 - 7.

[0114] 13. Zhang K, Liu X, Xu J, Yuan J, Cai W, Chen T, et al. Deep - learning models for the detection and incidence prediction of chronic kidney disease and type 2 diabetes from retinal fundus images. Nature Biomedical Engineering. 2021;5(6):533 - 45.

[0115] 14. Wintergerst MWM, Jansen LG, Holz FG, and Finger RP. Smartphone - Based Fundus Imaging - Where Are We Now?The Asia - Pacific Journal of Ophthalmology. 2020;9(4).

[0116] 15. Iqbal U. Smartphone fundus photography: a narrative review. International Journal of Retina and Vitreous. 2021;7(1):44.

[0117] 16. Li Z, He Y, Keel S, Meng W, Chang RT, and He M. Efficacy of a Deep Learning System for Detecting Glaucomatous Optic Neuropathy Based on Color Fundus Photographs. Ophthalmology. 2018;125(8):1199 - 206.

[0118] 17. Xiong J, Li F, Song D, Tang G, He J, Gao K, et al. Multimodal Machine Learning Using Visual Fields and Peripapillary Circular OCT Scans in Detection of Glaucomatous Optic Neuropathy. Ophthalmology. 2021.

[0119] 18. Li F, Song D, Chen H, Xiong J, Li X, Zhong H, et al. Development and clinical deployment of a smartphone - based visual field deep learning system for glaucoma detection, npj Digital Medicine. 2020;3(1):123.

[0120] 19. Li F, Yang Y, Sun X,Qiu Z, Zhang S, Tun TA, et al. Digital Gonioscopy Based on Three - dimensional Anterior - Segment OCT: An International Multicenter Study. Ophthalmology. 2021.

[0121] 20. Iwase A, Suzuki Y, Araie M, Yamamoto T, Abe H, Shirato S, et al. The prevalence of primary open-angle glaucoma in Japanese: the Tajimi Study. Ophthalmology. 2004;111(9):1641-8.

[0122] 21. He M, Foster PJ, Ge J, Huang W, Zheng Y, Friedman DS, et al. Prevalence and clinical characteristics of glaucoma in adult Chinese: a population-based study in Liwan District, Guangzhou. Invest Ophthalmol Vis Sci. 2006;47(7):2782-8.

[0123] 22. Foster PJ, Buhrmann R, Quigley HA, and Johnson GJ. The definition and classification of glaucoma in prevalence surveys. Br J Ophthalmol. 2002;86(2):238-42.

[0124] 23. Garway-Heath DF, Crabb DP, Bunce C, Lascaratos G, Amalfitano F, Anand N, et al. Latanoprost for open-angle glaucoma (UKGTS): a randomised, multicentre, placebo-controlled trial. Lancet. 2015;385(9975):1295-304.

[0125] 24. Tham YC, Li X, Wong TY, Quigley HA, Aung T, and Cheng CY. Global prevalence of glaucoma and projections of glaucoma burden through 2040: a systematic review and meta-analysis. Ophthalmology. 2014;121(11):2081-90.

[0126] 25. Thakur A, Goldbaum M, and Yousefi S. Predicting Glaucoma before Onset Using Deep Learning. Ophthalmology Glaucoma. 2020;3(4):262-8.

[0127] 26. Thakur A, Goldbaum M, and Yousefi S. Convex Representations Using Deep Archetypal Analysis for Predicting Glaucoma. IEEE Journal of Translational Engineering in Health and Medicine. 2020;8:1-7.

[0128] 27. Gupta K, Goldbaum M, and Yousefi S. 2021 IEEE / CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 2021:2259-67.

[0129] 28. Shuldiner SR, Boland MV, Ramulu PY, De Moraes CG, Elze T, Myers J, et al. Predicting eyes at risk for rapid glaucoma progression based on an initial visual field test using machine learning. PLoS One. 2021;16(4):e0249856.

[0130] 29. Wen JC, Lee CS, Keane PA, Xiao S, Rokem AS, Chen PP, et al. Forecasting future Humphrey Visual Fields using deep learning. PLoS One. 2019;14(4):e0214875.

[0131] 30. Zangwill LM, Weinreb RN, Beiser JA, Berry CC, Cioffi GA, Coleman AL, et al. Baseline topographic optic disc measurements are associated with the development of primary open-angle glaucoma: the Confocal Scanning Laser Ophthalmoscopy Ancillary Study to the Ocular Hypertension Treatment Study. Arch Ophthalmol. 2005;123(9):1188-97.

[0132] 31. Weinreb RN, Zangwill LM, Jain S, Becerra LM, Dirkes K, Piltz-Seymour JR, et al. Predicting the Onset of Glaucoma: The Confocal Scanning Laser Ophthalmoscopy Ancillary Study to the Ocular Hypertension Treatment Study. Ophthalmology. 2010;117(9):1674-83. 32. Gordon MO, Beiser JA, Brandt JD, Heuer DK, Higginbotham EJ, Johnson CA, et al. The Ocular Hypertension Treatment Study: Baseline Factors That Predict the Onset of Primary Open-Angle Glaucoma. Archives of Ophthalmology. 2002;120(6):714-20.

[0133] 33. Salvetat ML, Zeppieri M, Tosoni C, and Brusini P. Baseline factors predicting the risk of conversion from ocular hypertension to primary open-angle glaucoma during a 10-year follow-up. Eye. 2016;30(6):784-95.

[0134] 34. Salvetat ML, Zeppieri M, Tosoni C, and Brusini P. Baseline factors predicting the risk of conversion from ocular hypertension to primary open-angle glaucoma during a 10-year follow-up. Eye (Lond). 2016;30(6):784-95.

[0135] 35. Hu R, Racette L, Chen KS, and Johnson CA. Functional assessment of glaucoma: Uncovering progression. Survey of Ophthalmology. 2020;65(6):639-61.

[0136] 36. Rudnicka AR, Mt-Isa S, Owen CG, Cook DG, and Ashby D. Variations in Primary Open-Angle Glaucoma Prevalence by Age, Gender, and Race: A Bayesian Meta-Analysis. Investigative Ophthalmology & Visual Science. 2006;47(10):4254-61.

[0137] 37. Li J. Glaucoma type proportion of glaucoma outpatient in Beijing Tongren Hospital from 2014 to 2016. Investigative Ophthalmology & Visual Science. 2018;59(9):2745-.

[0138] 38. Song YJ, Kim YW, Park KH, Kim YK, Choi HJ, and Jeoung JW. Comparison of glaucoma patients referred by glaucoma screening versus referral from primary eye clinic. PLOS ONE. 2019;14(1):e0210582.

[0139] 39. Ronneberger O, Fischer P, and Brox T. In: Navab N, Hornegger J, Wells WM, and Frangi AF eds. Medical Image Computing and Computer-Assisted Intervention - MICCAI 2015. Cham: Springer International Publishing; 2015:234-41.

[0140] 40. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, and Batra D. Proceedings of the IEEE international conference on computer vision. 2017:618-26.

[0141] Table 1 Baseline characteristics of study participants in different datasets

[0142]

[0143]

[0144]

[0145] VF: Visual field; POAG: Primary open-angle glaucoma; PACG: Primary angle-closure glaucoma

[0146] *Comparison of demographic parameters between the training and external test dataset 1 by independent t-tests (age, follow-up duration, intraocular pressure, mean deviation, pattern standard deviation, and number of visual field tests) or chi-square tests (gender, hypertension cases, diabetes cases, glaucoma diagnosis / occurrence / progression cases)

[0147] #Comparison of demographic parameters between the training and external test dataset 2 by independent t-tests (age, follow-up duration, intraocular pressure, mean deviation, pattern standard deviation, and number of visual field tests) or chi-square tests (gender, hypertension cases, diabetes cases, glaucoma diagnosis / occurrence / progression cases)

[0148] Table 2 Performance of the deep learning model in the validation and external test sets

[0149]

[0150]

Claims

1. A method, comprising using at least one computer processor to: Receive one or more color fundus photographs (CFPs) of a patient; Apply a machine learning classifier to classify the received CFPs of the patient so as to diagnose whether the patient has glaucoma, the machine learning classifier having been trained using a dataset of CFPs of a patient cohort classified according to their glaucoma status.

2. A method, comprising using at least one computer processor to: Receive one or more color fundus photographs (CFPs) of a patient; Apply a machine learning classifier to predict the likelihood of future glaucoma onset or progression in the patient, the machine learning classifier having been trained using a dataset of CFPs of a longitudinal patient cohort, the dataset being related to the development of glaucoma in each of the patients in the cohort over a period of time.

3. The method according to claim 1, wherein The machine learning classifier includes a segmentation module that segments anatomical structures including retinal blood vessels, macula, optic cup, and optic disc from the received CFPs.

4. The method according to claim 3, wherein, The machine learning classifier further includes a diagnostic module that generates a glaucoma probability score.

5. The method according to claim 2, wherein The machine learning classifier includes a segmentation module that segments anatomical structures including retinal blood vessels, macula, optic cup, and optic disc from the received CFPs.

6. The method according to claim 5, wherein, The machine learning classifier further includes a prediction module that generates a risk score for future glaucoma onset or progression in the patient.

7. The method according to claim 3 or 5, wherein The segmentation module has been independently trained by manual annotation or segmentation of the anatomical structures including retinal blood vessels, macula, optic cup, and optic disc.

8. The method according to any one of the preceding claims, wherein, The one or more CFPs of the patient received are obtained from fundus images of the patient taken by a smartphone.

9. The method according to claim 2, wherein, The dataset of CFPs of the longitudinal patient cohort has been stratified into a low-risk group and a high-risk group for glaucoma onset or progression.

10. The method according to claim 9 further comprises: Use at least one computer processor to classify the patient as belonging to a low-risk group or a high-risk group for future glaucoma onset or progression.

11. The method according to any one of the preceding claims, wherein, The machine learning classifier includes a deep learning model.

12. The method according to claim 11, wherein, The deep learning model includes a convolutional neural network (CNN).

13. The method according to any one of the preceding claims, wherein, The machine learning classifier includes segmenting the anatomical structures including retinal blood vessels, macula, optic cup, and optic disc of the patient's CFPs using a U-net architecture.

Images