Predicting future growth of geographic atrophy using retinal imaging data

Abstract

A method implemented by one or more computing devices includes receiving fundus autofluorescence (FAF) image data of a subject's retina. The FAF image data includes a first FAF image associated with a first time point. An image input for a deep learning system is generated using the FAF image data. A predicted growth output of geographic atrophy (GA) lesions in the retina is generated via the deep learning system using the image input. The predicted growth output is associated with at least one future time point after the first time point.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of and priority to the filing dates of U.S. Provisional Patent Application No. / N63 / 519,009, filed August 11, 2023, U.S. Provisional Patent Application No. 63 / 496,202, filed April 14, 2023, and U.S. Provisional Patent Application No. 63 / 434,872, filed December 22, 2022, the entire disclosures of each of which are incorporated herein by reference.

[0002] The present disclosure is generally directed to predicting how geographic atrophy lesions will change over time and generating a visual depiction of future GA growth locations. More specifically, the present disclosure provides methods and systems for using deep learning to predict the region of growth (ROG) of geographic atrophy lesions at future time points. [Background technology]

[0003] Age-related macular degeneration (AMD) is the leading cause of vision loss in patients over the age of 50. Geographic atrophy (GA) is a late form of AMD. GA is a degeneration of the retina that can interfere with daily activities, such as driving and reading. GA is characterized by the progressive and irreversible loss of the choriocapillaris, retinal pigment epithelium (RPE), and photoreceptors. The progression of GA varies from patient to patient, and currently, there is no widely accepted treatment for preventing or slowing its progression. Therefore, assessing the progression of GA in individual patients may be important for studying GA and developing effective treatments. Currently, diagnosis and monitoring of GA lesion expansion can be performed using fundus autofluorescence (FAF) images obtained by confocal scanning laser ophthalmoscopy (cSLO). This type of imaging technique can be used to measure changes in GA lesions over time. In FAF images, areas of GA can be seen as dark areas, and GA progression can be assessed based on the rate of increase in these dark areas over time.

[0004] GA growth rate, measured using FAF images, is the change in lesion area over time and is widely accepted as an anatomical indicator of GA progression in clinical trials. However, some currently available techniques for assessing GA progression using FAF images may be more time-consuming than desired, prone to human error, and / or may produce variable results depending on the knowledge and expertise of the human grader. For example, some currently available techniques rely solely on human graders, or may involve a two-step process in which a human grader manually refines software-generated GA lesion contours, and the refined images are then used by the human grader to determine GA lesion area and GA growth rate. Therefore, the embodiments described herein recognize that it may be desirable to have one or more methods and / or one or more systems that address at least some of the above-mentioned problems. Summary of the Invention

[0005] In one or more embodiments, a method for predicting the growth of geographic atrophy (GA) lesions is provided. Fundus autofluorescence (FAF) image data of a subject's retina may be received. The FAF image data may include a first FAF image associated with a first time point. Image input for a deep learning system may be generated using the FAF image data. A predicted growth output of a GA lesion in the retina may be generated via the deep learning system using the image input. The predicted growth output may be associated with at least one future time point after the first time point.

[0006] In some embodiments, the predicted growth output may include a first growth image showing a first predicted growth area of ​​the GA lesion on the subject's retina between the first reference time point and a first future time point after the first reference time point. In some embodiments, the predicted growth output may also include a second growth image showing a second predicted growth area of ​​the GA lesion on the subject's retina between the second reference time point and a second future time point after the second reference time point. In some embodiments, the second reference time point and the first reference time point may be the same time point or different time points. In some embodiments, the second future time point is different from the first future time point. In some embodiments, the first reference time point may be the first time point or a time point between the first time point and the first future time point.

[0007] In some embodiments, the predicted growth output may include a growth image showing an area of ​​the retina predicted to be affected by a GA lesion at a selected future time point. In some embodiments, the predicted growth output may include a calculated area for the entire area of ​​the retina in the FAF image of the FAF image data predicted to be affected by a GA lesion at a selected future time point. In some embodiments, the predicted growth output may include a calculated area for the entire area of ​​the retina in the FAF image of the FAF image data predicted to be affected by a GA lesion at a selected future time point. In some embodiments, the predicted growth output may include a growth image showing an area of ​​new growth of a GA lesion between two time points. In some embodiments, the predicted growth output may include a calculated area of ​​new growth between two time points.

[0008] In some embodiments, the FAF image data may also include a second FAF image associated with a second time point that is after the first time point. In some embodiments, generating the image input may include preprocessing each of the first FAF image and the second FAF image such that the image input includes the first preprocessed FAF image and the second preprocessed FAF image. In some embodiments, the predicted growth output may include a growth image showing a predicted growth area of ​​the GA lesion for the subject's retina for the selected future time point. In some embodiments, the growth image may include an image background associated with the first FAF image or the second FAF image and a mask on the image background. The mask may identify a predicted growth area for the subject's retina for the selected future time point.

[0009] In some embodiments, the deep learning system may include a trained convolutional neural network with long- and short-term memory. In some embodiments, the deep learning system may include a trained convolutional neural network (CNN). A training dataset used to train the trained CNN may include multiple training sets, each corresponding to a plurality of eyes, and each training set of the multiple training sets may include training FAF images corresponding to four different time points. In some embodiments, the training FAF images of the training dataset may be stratified by at least one of baseline lesion area, lesion growth rate, foveal involvement, or focus. In some embodiments, the training FAF images of the training set may include four FAF images spaced over time by consistent time intervals.

[0010] In one or more embodiments, a method for training a deep learning system is provided. A plurality of training sets for a plurality of retinas of a plurality of subjects may be received. Each training set of the plurality of training sets may include training FAF images for at least two different time points. The plurality of training sets may be used to generate training inputs for the deep learning system. The deep learning system may be trained to generate predicted growth outputs based on FAF image data of the retinas of selected subjects. The predicted growth outputs may indicate predicted growth of GA lesions in the retinas for at least one future time point.

[0011] In some embodiments, the predicted growth output may include a growth image showing the area of ​​the retina predicted to be affected by the GA pathology for the future time point. In some embodiments, the predicted growth output may include a calculated area for the entire area of ​​the retina predicted to be affected by the GA pathology for the future time point.

[0012] In one or more embodiments, a system may include one or more data processors and a non-transitory computer-readable storage medium including instructions that, when executed on the one or more data processors, cause the one or more data processors to perform any of the methods disclosed herein. In one or more embodiments, a computer program product tangibly embodied in a non-transitory machine-readable storage medium is provided that includes instructions configured to cause one or more data processors to perform any of the methods disclosed herein. [Brief explanation of the drawings]

[0013] For a more complete understanding of the principles disclosed herein and their advantages, reference is now made to the following descriptions taken in conjunction with the accompanying drawings.

[0014] [Figure 1] FIG. 1 is a block diagram of a predictive system including a deep learning system that receives FAF imaging data and generates a growth output, according to an exemplary embodiment.

[0015] [Figure 2] FIG. 1 is a flowchart of a method for predicting GA lesion growth using a prediction system, according to one or more embodiments.

[0016] [Figure 3] 1 is a flowchart illustrating an embodiment of a process for training a deep learning system according to one or more embodiments.

[0017] [Figure 4A] 2A-2C are schematic diagrams of different exemplary deep learning models used to implement the growth prediction system from FIG. 1 , according to one or more embodiments.

[0018] [Figure 4B] 1A-1C are schematic diagrams of different exemplary deep learning models used to implement the growth prediction system 150 from FIG. 1 , according to one or more embodiments.

[0019] [Figure 5] 1 is an exemplary workflow illustrating exemplary processing of three whole-lesion models, according to one or more embodiments.

[0020] [Figure 6] 6 is an exemplary workflow illustrating exemplary processing of three multi-class models, according to one or more embodiments. For example, FIG. 6 shows an example of an input FAF image that may be sent to a full-lesion model (e.g., fourth deep learning model 416, fifth deep learning model 418, sixth deep learning model 420) and an output that may be generated as described with respect to FIG. 4B above.

[0021] [Figure 7] 10A-10C depict exemplary images of an exemplary workflow for post-processing of a whole-lesion model, according to one or more exemplary embodiments.

[0022] [Figure 8] 1 illustrates an example image of an example workflow for post-processing of multi-class models, in accordance with one or more example embodiments.

[0023] [Figure 9] 10A-10C depict exemplary images of an exemplary workflow for post-processing of a multi-class whole-lesion model, according to one or more exemplary embodiments.

[0024] [Figure 10] 10A-10C illustrate examples of predicted growth images generated for different deep learning models, according to one or more embodiments.

[0025] [Figure 11] 1 is a table showing various results of an example experiment for each of six different models for training, validation, and test sets, according to one or more embodiments.

[0026] [Figure 12] FIG. 1 is a block diagram illustrating an example of a computer system according to one or more embodiments.

[0027] It should be understood that the figures are not necessarily drawn to scale, and that objects in the figures are not necessarily drawn to scale relative to each other. The figures are intended to provide clarity and understanding of various embodiments of the devices, systems, and methods disclosed herein. Wherever possible, the same reference numerals will be used throughout the figures to refer to the same or like parts. Furthermore, it should be appreciated that the figures are not intended to limit the scope of the present teachings in any way. DETAILED DESCRIPTION OF THE INVENTION

[0028] I. Overview Being able to accurately predict how geographic atrophy (GA) will progress over time can be useful in many different scenarios. As an example, predictions of GA progression can be used to improve patient stratification in clinical trials where the goal is to slow GA progression, thereby enabling improved evaluation of treatment efficacy. Furthermore, in some cases, predictions of GA progression can be used to understand disease pathogenesis through correlation with genotypic or phenotypic signatures. Furthermore, predictions of GA growth can be used in clinical trials for enrichment, stratification, or covariate adjustment, as well as in clinical settings for patient counseling. In addition to GA growth, the location of GA lesions affects the visual field. Therefore, predicting future growth areas of GA lesions can be useful for identifying patients at higher risk of vision loss.

[0029] GA lesions can be imaged using a variety of imaging modalities. FAF images have been used to quantify GA lesion area. GA growth rate, the change in lesion area over time measured using FAF images, is widely accepted as an anatomical indicator of GA progression in clinical trials. However, currently available techniques for assessing GA progression using FAF images rely on human graders to first manually identify portions of the FAF image that represent GA lesions. In some cases, this first step is semi-automated, relying on human graders to make manual refinements and / or corrections to the software-generated initial contour of the GA area. The identified portions of the FAF image are then evaluated to determine GA lesion area and GA growth rate. These techniques may involve a two-step process that can be more time-consuming than desired, prone to human error, less accurate than desired, and / or produce variable results depending on the knowledge and expertise of one or more human graders. Furthermore, these types of techniques are intended for individual time points and therefore cannot visually present to medical professionals (e.g., clinicians, healthcare providers, etc.) what the FAF image will look like in the future (e.g., 3 months, 6 months, 9 months, 1 year, etc.).

[0030] Therefore, there is a need for methods and systems that improve the speed, efficiency, and accuracy associated with predicting GA lesion growth and provide a way to visualize the growth area of ​​GA lesions at future time points. This disclosure describes various embodiments for using deep learning to predict future growth areas of geographic atrophy lesions from retinal imaging data, such as FAF images. In this embodiment, GA growth rate (e.g., annual growth rate) can be predicted from baseline FAF images.

[0031] In one or more embodiments, a method is provided for predicting future geographic atrophy growth based on retinal imaging data. The retinal imaging data may take the form of, for example, FAF imaging data. FAF imaging data of a subject's eye at a first time point may be received. A deep learning system processes the FAF imaging data to generate a growth image with a mask that identifies a growth region of the geographic atrophy lesion with respect to the future time point. For example, the mask may identify an area predicted to be affected by the GA lesion at the future time point. In some cases, the mask identifies new growth, i.e., new growth predicted between the baseline time point and the future time point. In some cases, the deep learning system also calculates an area (e.g., mm) for the growth region. 2 In other embodiments, the deep learning system may use the FAF imaging data to generate multiple growth images for different future time points.

[0032] Predictions can be generated with reliable accuracy for use in clinical settings. For example, growth images can be used to determine whether a subject is a candidate for a clinical trial, which clinical trial to assign the subject to, how to customize the subject's treatment, how to monitor the subject's progress during a clinical trial, or a combination thereof. The techniques described herein can be used to predict the prognosis of one or more subjects, predict the responsiveness of one or more subjects to various treatments, identify treatments that are predicted to be effective for individual subjects, assign one or more subjects to the appropriate arm within a clinical trial, or a combination thereof.

[0033] The growth image can be used to generate an output that includes an indication of whether the subject is eligible for a clinical trial to test a medical treatment for geographic atrophy. In some embodiments, this output can be used to enroll the subject in a clinical trial, to exclude the subject from participating in a clinical trial, to customize a protocol in a clinical trial for the subject, or to enroll the subject in a different clinical trial.

[0034] II. Geographic atrophy (GA) lesion prediction II.A. Exemplary Prediction System Referring now to the drawings, Figure 1 is a block diagram of a prediction system 100 according to various embodiments. Generally, the prediction system 100 is used to predict the progression of geographic atrophy (GA) lesions in a subject's retina. As shown, the prediction system 100 includes a computing platform 105, a data store 110, and a display system 115.

[0035] Computing platform 105 may take a variety of forms. For example, in one embodiment, computing platform 105 includes a single computer (or computer system), while in another embodiment, computing platform 105 includes multiple computers that communicate with each other. In other examples, computing platform 105 takes the form of a cloud computing platform. As shown, data storage 110 and display system 115 each communicate with computing platform 105. In some examples, data storage 110, display system 115, or both may be considered part of computing platform 105 or may be otherwise integrated with computing platform 105. Thus, in some examples, computing platform 105, data storage 110, and display system 115 may be separate components that communicate with each other, while in other examples, some combination of these components may be integrated together.

[0036] The computing platform 105 of the prediction system 100 is configured to receive or otherwise access an image input 120. The image input 120 may include one or more images acquired of one or more subjects. The image input 120 may include, for example, but not limited to, retinal imaging data, such as FAF imaging data 125. The FAF imaging data 125 includes one or more FAF images, such as FAF image 130, each capturing a subject's retina. Generally, the subject's retina has or is suspected to have a geographic atrophy (GA) lesion. The GA lesion can be a continuous or discontinuous area of ​​the retina suffering from degeneration (e.g., chronic progressive degeneration). The GA lesion can include a single lesion (e.g., a single contiguous lesion area) or multiple lesions (e.g., a discontinuous lesion area consisting of multiple separate lesions).

[0037] In one or more embodiments, the FAF imaging data 125 includes one or more reference FAF images captured at one or more reference time points. The one or more reference time points may include, for example, a baseline time point, 6 months after the first treatment, 3 months after the first treatment, 12 months after the first treatment, or some other type of reference time point. In one or more embodiments, the baseline time point may be a time point before treatment, the same day as a treatment dose (e.g., a dose in the first treatment), the same day as an initial diagnosis of GA, the same day as a screening performed for GA, or some other type of baseline or reference time point. For example, a first FAF image that may be used as a reference image may correspond to month 0 (“T1”). A second FAF image may be taken 6 months (“T2”) after the T1 FAF image. A third FAF image may be taken 12 months (“T3”) after the T1 FAF image (6 months after the T2 FAF image). The fourth FAF image may be taken 18 months ("T4") after the T1 FAF image (12 months after the T2 FAF image and 6 months after the T3 FAF image). FAF image 130 shown in FIG. 1 is an example of a FAF image from FAF imaging data 125. FAF image 130 corresponds to a baseline time point and captures GA lesions. While an exemplary time interval disclosed herein is 6 months, the time interval may be measured in 1 month, 3 months, 9 months, or weeks. In some embodiments, the FAF images associated with each eye are spaced at consistent time intervals (e.g., 6 months).

[0038] Prediction system 100 includes image processor 135, which may be implemented using hardware, software, firmware, or a combination thereof. In one or more embodiments, image processor 135 is implemented on computing platform 105. Image processor 135 receives image input 120 for processing. For example, image input 120 may be sent as input to image processor 135, retrieved from data storage 110 or some other type of storage (e.g., cloud storage), or received in some other manner.

[0039] In various embodiments, and as shown in FIG. 1 , image processor 135 includes a pre-processing module 140 that processes image input 120 (e.g., FAF imaging data 125) to create modified FAF images 145, and then transmits modified FAF images 145 to growth prediction system 150 to generate growth output 170. Pre-processing module 140 is shown as a separate component from growth prediction system 150 in FIG. 1 .

[0040] However, in some embodiments, growth prediction system 150 and preprocessing module 140 may be considered a single component. In such embodiments, growth prediction system 150 may receive FAF image 130 as an input and process FAF image 130 to generate corrected FAF image 145. Growth prediction system 150 may then, in some embodiments, generate a growth output 170 of the GA lesion captured in FAF image 130. Preprocessing may include scaling, resizing, cropping, horizontal flipping, vertical flipping, image intensity normalization, adding and / or removing noise, translation, and other such preprocessing operations. Resizing may include resizing FAF image 130 to a selected pixel size (e.g., 512 pixels by 512 pixels). Image intensity normalization may include normalizing the intensity values ​​of pixels in FAF image 130 to a selected scale (e.g., 0 to 1, −1 to 1, or another type of scale).

[0041] The growth prediction system 150, which may include a machine learning model (e.g., a deep learning model), may be implemented in any of several different ways. In one or more embodiments, the growth prediction system 150 may be a deep learning system including one or more deep learning models. For example, the growth prediction system 150 may include an artificial neural network (ANN), such as a perceptron, a multilayer perceptron (MLP), an autoencoder (AE), a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory (LSTM), a grated recurrent unit (GRU), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a generative adversarial network (GAN), a deep Q-network, a neural autoregressive distribution estimation (NADE), an adversarial network (AN), an attention model (AM), a spiking neural network (SNN), a deep reinforcement learning model, or other model.

[0042] In one or more embodiments, the growth prediction system 150 may be implemented using a predictive neural network (NN) system. The predictive NN system may include any number or combination of neural networks. In one or more embodiments, the predictive NN system takes the form of a convolutional neural network (CNN) that includes one or more neural networks. In some cases, the growth prediction system 150 includes multiple subsystems and / or layers, each including one or more neural networks. In other embodiments, the predictive NN system takes the form of a long short-term memory (LSTM) UNet that includes one or more neural networks. Each of these one or more neural networks may itself be a convolutional neural network.

[0043] In other embodiments, the predictive NN system takes the form of a two-dimensional U-Net CNN. The U-Net may consist of an encoder (contraction path) that converts an image into a feature map, and a decoder (augmentation path) that converts the feature map into a probability map of equal size to the input image. The image may include, for example, image input 120. The encoder may extract image features of different spatial resolutions, which may in turn be used by the decoder to derive an accurate segmentation mask.

[0044] In some embodiments, the encoder is a 34-layer residual neural network (ResNet) backbone that extracts features at different resolutions, an encoder depth of 4, and pre-trained ImageNet weights, where encoder depth refers to the number of stages, and feature size decreases with each additional stage. The decoder may use batch normalization between the convolutional and activation layers and may have a depth of 4 with decoder channels (128, 64, 32, 16).

[0045] Growth prediction system 150 may be used in either a training mode 160 with a training data set 163 (also referred to as a training input) or a prediction mode 165. In prediction mode 165, growth prediction system 150 is used to generate a growth output 170. The growth output 170 generated by growth prediction system 150 may include, for example, a set of growth images 172 (e.g., including one or more growth images, such as growth image 175) and / or one or more measurement outputs, such as measurement output 180. In one embodiment, growth image 175 identifies growth regions of geographic atrophy lesions for future time points.

[0046] In one or more embodiments, grown image 175 may include mask 185, image background 190, or both. Image background 190 may be an image layer below mask 185. Image background 190 may be, for example, FAF image 130, corrected FAF image 145, or some other representation of FAF image 130. In one or more embodiments, image background 190 has dimensions that are equal in size to or proportional to (e.g., the same aspect ratio as) FAF image 130.

[0047] Generally, the mask 185 indicates the predicted growth of geographic atrophy (GA) lesions (which may be continuous or discontinuous) at future time points. For example, the mask 185 may be overlaid on an image background 190, which may be the FAF image 130, to identify portions of the FAF image 130 that correspond to the predicted growth of geographic atrophy (GA) lesions at future time points. The mask 185 indicates areas predicted to be affected by geographic atrophy at future time points (e.g., any number of days, weeks, months, or years after the baseline time point).

[0048] In one or more embodiments, the mask 185 takes the form of a boundary or contour. The boundary or contour may be dashed, solid, dotted, or some other variation. The boundary or contour may include various thicknesses. In other embodiments, the mask 185 is an opaque block or other type of graphic feature overlaid on the FAF image 130 that identifies the entire area (or alternatively, a selected percentage of the area) predicted to be affected by GA lesions. The mask 185 may be presented using any type of grayscale, color, hash, pattern, etc. that can be distinguished from the image background 190. In one or more embodiments, the mask 185 is identified at the pixel level of the FAF image 130. In some embodiments, the mask 185 is identified by the leftmost or rightmost pixel of every row of the FAF image 130.

[0049] As described above, the mask 185 may identify regions or portions of the image background 190 that are predicted to be affected by GA lesions at future time points. If the reference time point is before treatment, the mask 185 may identify predicted regions of growth (ROG) of GA lesions at the baseline time point. In other embodiments, the mask 185 identifies differences between the regions of the image background 190 that are predicted to be affected by GA lesions at future time points and the reference time point. In this manner, the regions of growth predicted by the growth prediction system 150 may be new growth between the reference time point and the future time point. In yet other examples, the mask 185 includes multiple predictions for multiple time points after the baseline time point or other type of reference time point.

[0050] As described above, mask 185 may indicate predicted areas of growth of a GA lesion by identifying predicted areas of growth. In other embodiments, mask 185 may identify portions of FAF image 130 that are not predicted to be areas of growth of a GA lesion, thereby indicating predicted growth. For example, mask 185 may be overlaid on all portions of FAF image 130 that are predicted not to be associated with a GA lesion at a future time, such that any portion of FAF image 130 not covered by mask 185 indicates predicted growth.

[0051] In one or more embodiments, the growth prediction system 150 also generates a measurement output 180, which may be a measurement of the growth area of ​​the GA lesion. This measurement may be, for example, a calculated area (e.g., mm 2 ) The measurement may be a calculated area of ​​the entire region predicted to be affected by GA lesions at a future time point, or a calculated area of ​​new growth between the baseline time point and a future time point. In some embodiments, measurement output 180 is presented on growth image 175. In other embodiments, measurement output 180 is presented separately from growth image 175.

[0052] As described above, the growth prediction system 150 is trained when in training mode 160. In training mode 160, the growth prediction system 150 is trained using a training dataset 163. The training dataset 163 includes a FAF image dataset selected to ensure that the growth prediction system 150 can be used in prediction mode 165 with a desired level of accuracy. In one or more embodiments, the training dataset 163 includes FAF images obtained through one or more studies (e.g., clinical studies, research studies, etc.). When FAF images are obtained from multiple studies, the studies are selected so that the selection criteria for the studies are the same. Ensuring that the same selection criteria were used in the studies helps ensure a certain type of consistency across the FAF images, which improves training accuracy and, therefore, prediction accuracy.

[0053] II.B. Exemplary Methodology for Predicting GA Lesion Growth Using a Prediction System 1, the growth prediction system 150 of the prediction system 100 may be used in a training mode 160 or a prediction mode 165. Exemplary methods for using the growth prediction system 150 in these modes are described in further detail below.

[0054] II.B.1. Exemplary Methods Using a Predictive System in Predictive Mode Figure 2 is a flowchart illustration of a method for predicting GA lesion growth using a prediction system, according to one or more embodiments. In Figure 2, the prediction system used may be prediction system 100 of Figure 1. Accordingly, process 200 of Figure 2 will be described with continued reference to Figure 1 and prediction system 100 of Figure 1.

[0055] Step 205 includes receiving image data of fundus autofluorescence (FAF) of the subject's retina. In some embodiments, in step 205, prediction system 100 receives the FAF image data. In some embodiments, the received or otherwise accessed FAF image data is image input 120. As described above, FAF image data 120 may include FAF imaging data 125, which may include one or more FAF images, such as FAF image 130. Generally, one of the FAF images is associated with a first time point. The first time point or first reference time point can be, for example, a baseline time point (e.g., the time of initial diagnosis of GA, which may be referred to as time (0) or 0 months), or a time point after the baseline time point (e.g., 3 months, 6 months, 9 months, 12 months, 18 months, etc. after baseline). The first future time point can be 6 months, 1 year, 18 months, 2 years after the first reference time point, or some other time point. The first reference time point can be, for example, the first time point (e.g., the baseline time point) or another time point between the baseline time point and a future time point. In some embodiments, the received FAF image data 120 includes T1 FAF images or T2 FAF images, while in other embodiments, the received FAF image data includes T1 FAF images and T2 FAF images.

[0056] Step 210 includes using the FAF image data to generate processed image data for the deep learning system. In some embodiments, in step 210, prediction system 100 uses the received FAF imaging data to generate processed image data for the deep learning system. In some embodiments, the deep learning system is growth prediction system 150, and the processed image data includes corrected FAF image 145. In some embodiments, using FAF image data 125 to generate corrected FAF image 145 for growth prediction system 150 includes sending FAF image data 125 as input to preprocessing module 140 or growth prediction system 150 for preprocessing. As previously mentioned, preprocessing may include scaling, resizing, cropping, horizontal flipping, vertical flipping, image intensity normalization, adding and / or removing noise, translation, and other such preprocessing operations.

[0057] Step 215 includes generating 215 a predicted growth output of geographic atrophy (GA) lesions in the retina using the processed image data via a deep learning system. In some embodiments, in step 215, prediction system 100 generates 215 a predicted growth output of geographic atrophy (GA) lesions in the retina using the processed image data via growth prediction system 150. In some embodiments, the predicted growth output includes GA growth output 170, and the predictive NN system generates GA growth output 170.

[0058] The growth prediction system 150 may include one or more models from a variety of models. While additional details are provided with respect to the examples below, the predictive NN system of the growth prediction system 150 may be, for example, a total lesion model, a simple UNet model, a multi-channel UNet model, a continuous label UNet model, an LSTM UNet model, or another CNN model. In some embodiments, a total lesion model is used. In other embodiments, the growth prediction system 150 may be trained using T4 total lesions as ground truth. In some cases, the growth prediction system 150 may infer T4 total lesions from T2 FAF images. The growth prediction system 150 may be a Simple U-Net. In some embodiments, the growth prediction system 150 may infer T4 total lesions from a combination of T1 and T2 FAF images. The growth prediction system 150 may be a Multichannel U-Net. In some embodiments, the growth prediction system 150 may infer T3 and T4 total lesions, respectively, from T2 FAF images. The growth prediction system 150 may be a Sequential Label U-Net. In yet other embodiments, the growth prediction system 150 may estimate T4 total lesions from a combination of T1 and T2 FAF images. The growth prediction system 150 may be, for example, an LSTM UNet model. In some embodiments, the growth prediction system 150 includes or comprises a multi-class model. The multi-class model may be trained with multi-class ground truth. In some embodiments, the growth prediction system 150 may estimate class T2 total lesions and a 1-year area of ​​growth (ROG) (e.g., the area of ​​growth between the T4 lesion area and the T2 lesion area, i.e., T4-T2 ROG) from the T2 FAF images. The growth prediction system 150 may estimate both T2 total lesions and a 1-year ROG (e.g., T4-T2 ROG) from a combination of T1 and T2 FAF images. The growth prediction system 150 may estimate T2 total lesions and a 6-month ROG (e.g., T4-T3 ROG and T3-T2 ROG) from the T2 FAF images.

[0059] The growth prediction system 150 may be a Sequential U-Net. In each of these examples of the growth prediction system 150, the predictive NN system can provide end-to-end predictions in which inputs are automatically processed into predicted GA growth outputs 170. No human intervention is required in the prediction mode. Generally, the growth prediction system 150 is trained using a training dataset that ensures that GA lesion growth is predicted with at least a threshold level of accuracy, which may be determined, for example, based on performance metrics.

[0060] The GA growth output 170 generated by the growth prediction system 150 is associated with at least one future time point after the first time point. That is, the GA growth output 170 is associated with a time point that is chronologically after the first time point and is therefore a future time point relative to the first time point. If the received FAF image data includes a T1 FAF image, the future time point may be associated with T2, T3, T4, or later. If the received FAF image data includes a T2 FAF image, the future time point may be associated with T3, T4, or later. The GA growth output 170 indicates the predicted growth of geographic atrophy (GA) lesions in the retina at at least one future time point and may include one or more growth images. For example, the GA growth output 170 may include a first growth image and an associated first measurement output, and a second growth image and an associated second measurement output. However, as mentioned above, the measurement output may be excluded in some embodiments. In some embodiments, when the received FAF image data 125 includes a T2 FAF image in step 205, the first growth image and first measurement output are associated with a T3 lesion, and the second growth image and second measurement output are associated with a T4 lesion. However, the combinations of growth outputs vary, and additional examples are provided below.

[0061] In some embodiments, the first growth image and associated first measurement output are associated with a first future time point, and the second growth image and associated second measurement output are associated with a second future time point. The second future time point and the first future time point can be the same time point or different time points. As an example, the first future time point is associated with predicted GA lesion growth at 6 months from baseline, and the second future time point is associated with predicted GA lesion growth at 1 year from baseline.

[0062] One exemplary growth measure 180 is the calculated area (e.g., mm) for the combined affected regions. 2 ), which includes the area already affected at the first time point and the predicted area of ​​growth for the future time point. Another measurement may, for example, include the area of ​​growth (e.g., the area calculated for the predicted area of ​​growth at the future time point) and omit the area already affected at the first time point. However, in other embodiments, growth measurement 180 includes the area of ​​growth between two future time points. For example, if the growth output includes predicted growth for a first future time period and predicted growth for a second future time period, the area of ​​growth may include the difference between the area associated with the second future time point and the first future time point. In some embodiments, measurement output 180 is overlaid on or presented along with growth image 175. In other embodiments, measurement output 180 is presented separately from growth image 175. The image may include a photograph, an annotated photograph, a graphic depiction, or the like.

[0063] A growth measurement of 180 is mm 2The mask 185, expressed in units of measurement such as pixels, may identify predicted growth locations for the subject's retina at a selected future time point. That is, the mask 185 visually indicates which areas of the retina will be affected at the future time point. In one or more embodiments, the mask 185 may identify areas of the image background 190 that will be affected in addition to areas predicted to be affected by GA lesions at the future time point. In other embodiments, the mask 185 identifies differences between areas of the image background 190 predicted to be affected by GA lesions at future time points and the baseline time point (e.g., identifying areas of predicted growth after the baseline time point). In still other embodiments, the mask 185 identifies differences between areas of the FAF image background predicted to be affected by GA lesions at two different future time points.

[0064] In one or more embodiments, step 215 may be repeated to generate a third growth image and / or a third measurement output for a third future time point. Prediction system 100 is not limited to generating three growth images and / or measurement outputs for three future time points, but may generate any number of growth images and / or measurement outputs for any number of future time points.

[0065] In one or more embodiments, the GA growth output 170 may be transmitted to a display system 115 via one or more communication links (e.g., wired, wireless, and / or optical communication links), may be stored in data storage 110, or both. The display system 115 includes one or more display devices in communication with the computing platform 105. The display system 115 may be separate from the computing platform 105 or may be at least partially integrated as part of the computing platform.

[0066] In some embodiments, the GA growth output 170 is transmitted as a report that can be viewed on the display system 115. The report may include, for example, but is not limited to, at least one of a table, a spreadsheet, a database, a file, a presentation, an alert, a graph, a chart, one or more graphics, or a combination thereof.

[0067] For example, GA growth output 170 may be used to determine whether a subject is a candidate for a clinical trial, to which clinical trial to assign the subject, how to customize the subject's treatment, how to monitor the subject's progress during the clinical trial, or a combination thereof. Method 200 and / or GA growth output 170 may be used to predict the prognosis of one or more subjects, to predict the responsiveness of one or more subjects to various treatments, to identify treatments predicted to be effective for individual subjects, to assign one or more subjects to an appropriate arm within a clinical trial, or a combination thereof.

[0068] The GA growth output 170 may be used to generate an output that includes an indication of whether a subject is eligible for a clinical trial to test a medical treatment. In some embodiments, this output may be used to enroll the subject in a clinical trial, to exclude the subject from participating in a clinical trial, to customize a protocol in a clinical trial for the subject, or to enroll the subject in a different clinical trial.

[0069] II.B.2. Exemplary Methods Using a Prediction System in Training Mode 3 is a flowchart illustrating an embodiment of a process for training a deep learning system according to one or more embodiments. In one or more embodiments, method 300 may be implemented to develop a trained growth prediction system (e.g., trained growth prediction system 150 of prediction system 100 described in FIG. 1). Process 300 of FIG. 3 will be described with continued reference to prediction system 100 of FIG. 1.

[0070] Step 305 includes receiving multiple training sets for multiple retinas of multiple subjects in step 305. In some embodiments, the multiple training sets for multiple retinas of multiple subjects in step 305 are the training dataset 163 of FIG. 1. This training dataset 163 may be accessed, for example, from a database, cloud storage, or some other type of storage. The training dataset 163 may include FAF images from multiple sources, such as clinical trials or studies with the same (or substantially the same or similar) selection criteria. Ensuring that the training dataset 163 is constructed from studies that share the same (or substantially the same or similar) selection criteria improves or increases the consistency across FAF images, which can improve the accuracy of training and thereby improve the accuracy of prediction compared to using training sets from studies with different types of selection criteria. Given that the predicted growth images are drawn from the same type of distribution as the training dataset (e.g., selection criteria for a clinical trial), the accuracy of prediction may be improved. In some embodiments, the growth prediction system 150 may be selected or configured to reduce the total amount of time, processing resources, or both used for training.

[0071] Generally, each training set in the training dataset 163 includes training FAF images for at least two different time points. In one or more embodiments, a training set of the plurality of training sets includes multiple training FAF images corresponding to different time points (e.g., 2, 3, 4, 5, 6, or more time points). In one or more embodiments, the training FAF images include four FAF images spaced over time at consistent time intervals (e.g., 6-month intervals) relative to a baseline time point (e.g., time of initial or confirmatory diagnosis of GA).

[0072] Step 310 includes generating training inputs for the deep learning system using the multiple training sets. In some embodiments, generating training inputs for the deep learning system using the multiple training sets in step 310 includes performing one or more preprocessing operations to preprocess the training FAF images of the multiple training sets. Such preprocessing operations may include, but are not limited to, scaling, resizing, cropping, horizontal flipping, vertical flipping, adding and / or removing noise, translating, and other such preprocessing operations. In one or more embodiments, the preprocessing operations further include determining a portion of the multiple datasets to be used to generate the training inputs. This may include removing certain FAF images from the multiple training sets. Additional details regarding exemplary FAF image selection are provided in the examples below. For example, without limitation, approximately 80% of the multiple datasets may be used to form the training inputs, whereas approximately 20% of the multiple datasets would not be used. In general, generating training inputs for a deep learning model using multiple datasets from multiple sources improves the predictive performance of the deep learning system. For example, using deep learning models trained to analyze FAF images and automatically predict the growth of GA lesions can improve the speed and efficiency of making these predictions, as well as the accuracy of the predictions.

[0073] Step 315 includes generating a predicted growth output based on the FAF image data of the retina of the subject selected in step 315 via a deep learning system (e.g., the deep learning model in growth prediction system 150 of FIG. 1 ). In some embodiments, in step 315, the deep learning system may be trained in a variety of different ways to create growth prediction system 150. Additional details regarding various types of training are provided below.

[0074] III. Exemplary Experiments Using Multiple Types of Exemplary Deep Learning Models to Implement a Growth Prediction System The exemplary experiment involves multiple different deep learning models for use in predicting future growth areas of GA lesions using FAF images. These models were trained to predict GA lesion growth at various time points. Each of these different deep learning models is an example of an implementation of the growth prediction system 150 of FIG. 1 .

[0075] III.A. Exemplary Curation of Training Datasets via Filtering and Preprocessing to Form Training, Validation, and Test Sets An exemplary deep learning model was trained using a training input (or training dataset) (this is an example of an implementation of training dataset 163 in Figure 1). The training dataset included imaging data derived from FAF images obtained from clinical trials (e.g., lampalizumab Phase 3 clinical trials (NCT02247479 and NCT02247531) and observational studies (NCT02479386 and NCT02399072). Inclusion criteria for study eyes were: eyes with a clearly defined area of ​​GA secondary to AMD, no evidence of anterior choroidal neoplasia or active choroidal neoplasia, and a peripheral band-like or diffuse hyperautofluorescence pattern on FAF images, between 2.54 and 17.78 mm, located entirely within the blue-light FAF imaging field. 2 Inclusion criteria included a total lesion area of ​​1.27 mm (papillary area of ​​1–7). If GA lesions were multi-sited, the inclusion criteria were a focal lesion of 1.27 mm (papillary area of ​​1–7). 2 (greater than 0.5 nipple area). The clinical trial conformed to the Declaration of Helsinki and was compliant with the Health Insurance Portability and Accountability Act.

[0076] In the exemplary experiment, 30-degree (field 2) FAF images of the macula center with 786 pixels x 786 pixels or 1536 pixels x 1536 pixels were used. Because the Phase 3 trial did not observe a treatment effect on lesion growth rate, data from all treatment groups were pooled for this example analysis. GA lesion area was graded on FAF images using RegionFinder software (Heidelberg Engineering, Inc.). Before beginning the grading process for follow-up visit images, human graders longitudinally registered the follow-up visit FAF images to the screening (e.g., baseline) visit FAF images using automated registration software. GA lesion area was graded at a central reading center by at least one trained human grader (G1).

[0077] In this example experiment, the available training data included FAF images of patient eyes taken at various visits between screening and up to two years. This available training data was filtered based on various rules to arrive at a training dataset (e.g., consisting of multiple training sets for different patients) that could then be divided into training, validation, and test sets. For example, images of patients with missing GA lesion annotations were excluded from the available training data. Of the remaining available training data, patients with missing GA lesion annotations at four consecutive longitudinal visits were excluded.

[0078] Among the remaining available training data, the patient had various combinations of FAF images based on four consecutive hospital visits (T1, T2, T3, T4). For example, the patient had a combination of four images for screening (e.g., first visit / baseline visit) (coded at T1), 24 weeks (coded at T2), 48 weeks (coded at T3), and 72 weeks (coded at T4); the patient had a combination of four images for 24 weeks (coded at T1), 48 weeks (coded at T2), 72 weeks (coded at T3), and 96 weeks (coded at T4); the patient had a combination of four images for 48 weeks (coded at T1), 72 weeks (coded at T2), 96 weeks (coded at T3), and 120 weeks (coded at T4). Thus, T1 referred to the baseline visit, T2 referred to the visit six months after the baseline, T3 referred to the visit one year after the baseline, and T4 referred to the visit 1.5 years after the baseline.

[0079] Among this remaining available training data, patients with misregistration of FAF images and corresponding annotations of GA lesions over longitudinal visits were excluded. Misregistration was determined based on pairwise comparisons made between the annotations in (1) T1 and T2, (2) T2 and T3, (3) T3 and T4, and (4) T2 and T4. For each of these combinations, the Dice similarity coefficient (DSC), the change rate of lesion area (%), and the lesion growth rate (mm 2 / year) were obtained. The non-exclusion criteria were selected such that the papilla (DSC) > 0.7, -10% < change in lesion area (%) < 100%, and all annotations were within the FAF image field of view. Cases achieving 0.7 < DSC < 0.9 were manually reviewed for registration errors and peripapillary lesions, and if observed, the patient was excluded from this experiment.

[0080] Among the remaining available training data, patients who were duplicated across different combinations of longitudinal visits were excluded to ensure that the final training data set included unique combinations of patients across various combinations of longitudinal visits. In other words, there were no patients duplicated between data sets.

[0081] Preprocessing included resizing all FAF images to a size of 768 × 768 pixels. Additionally, each FAF image was z-normalized (a process used to normalize all pixels in an image so that the mean of all values ​​is 0 and the standard deviation is 1).

[0082] Of the final training dataset, the patient set was divided into training (310), validation (78), and test (209) sets. Thus, the training and validation sets formed the development set. The dataset was stratified across the training, validation, and test sets (to ensure similar distributions within each) with respect to baseline (or initial) lesion size (e.g., large or small), lesion growth rate (e.g., fast or slow), foveal involvement (e.g., non-subfoveal or subfoveal), focus (e.g., monofocal or multifocal), eye (e.g., right or left eye), study (e.g., specific clinical trial), and visit (e.g., specific longitudinal combination of visits).

[0083] III.B. Developing an Exemplary Model Each deep learning model was implemented using a 2D UNet architecture (an example of a convolutional neural network (CNN)). The UNet architecture includes an encoder (contraction path) that can convert, for example, a medical image into a feature map, and a decoder (augmentation path) that can convert the feature map into a probability map of the same size as the input image. The encoder extracts image features at different spatial resolutions, which are then used by the decoder to derive an accurate segmentation mask.

[0084] In this example experiment, the same encoder and decoder specifications were used for all implementations of the deep learning model using UNet. The encoder was a 34-layer residual neural network (ResNet) backbone extracting features at different resolutions, an encoder depth of 4, and pre-trained ImageNet weights. The encoder depth refers to the number of stages, and the feature size decreases with each additional stage. The decoder used batch normalization between the convolutional and activation layers and had a depth of 4 with decoder channels of (128, 64, 32, 16). Training parameters included, for example, data augmentation, scheduled learning rate reduction with loss plateau, a batch size of 4, a maximum of 200 epochs (with early stopping), and the optimizer AdamW. Data augmentation consisted of randomized horizontal and vertical flips, the addition of noise, and minor translations and scaling.

[0085] Model development included the development of three different whole-lesion models and three multi-class models (e.g., all developed using PyTorch). Hyperparameter optimization was performed via grid search over learning rates (1e-3, 1e-4, 5e-4, 1e-5), weight decay (1e-2, 1e-4), and loss functions, resulting in the development of 48 individual models per model type (six types total). Searches were designed to increase or maximize the papilla growth area calculated during post-processing. Model performance was evaluated using Dice scores, coefficient of determination (R 2 ), and the squared Pearson correlation coefficient (r ) between the 1-year growth area of ​​true and derived GA lesions. 2 ) was evaluated by calculating

[0086] Image segmentation can be performed to improve the accuracy of deep learning model 150. Segmentation can be performed using, for example, an autoregressive network (which can be used to predict semantic segmentation maps for unobserved future frames from past sequences of video), a temporal encoder-decoder network architecture (which can derive features from past frames and later use them to construct future semantic segmentations), a separate encoder-decoder architecture (which can be used to identify future trajectory points of an object), an LSTM component (e.g., an LSTM component in any of the aforementioned architectures), or a combination thereof to predict future lesion growth and / or potentially concurrent lesion growth over patient visits. As an example, an LSTM-backed autoregressive segmentation model can be used to predict future and potentially concurrent lesion growth over patient visits.

[0087] III.B.1. Exemplary Whole-Lesion Model Three exemplary full-lesion models are described with respect to FIG.

[0088] 4A is a schematic diagram of a different exemplary deep learning model used to implement the growth prediction system 150 from FIG. 1, according to one or more embodiments. In FIG. 4A, the growth prediction system 150 is implemented differently and trained to receive FAF images that capture GA lesions at various time points and generate predicted growth output (e.g., including growth images) for various future time points. These various time points may include, for example, T1, T2, T3, and T4 as described above, which are spaced apart by a consistent time interval of six months.

[0089] In this exemplary experiment, T1 FAF image 402 is an example of one type of image input generated at a first time point (T1) that may be transmitted to growth prediction system 150. T1 FAF image 402 may be an example of an implementation of corrected FAF image 145 of FIG. 1. T1 of T1 FAF image 402 is the baseline time point. T2 FAF image is an example of one type of image input generated at a second time point (T2) that may be transmitted to growth prediction system 150. T2 FAF image 404 may be an example of an implementation of corrected FAF image 145 of FIG. 1. T2 of T2 FAF image 402 is six months after baseline.

[0090] In this exemplary experiment, the growth prediction system 150 is implemented using a first deep learning model 410 (i.e., Model 1). The first deep learning model 410 is implemented using a simple UNet architecture trained to infer GA pathology across T4 from T2 FAF images 404. More specifically, the first deep learning model 410 receives and processes the T2 FAF images 404 to generate a T4 growth image 406. The T4 growth image 406 may be an example of an implementation for the growth image 175 of FIG. 1. The T4 growth image 406 is a growth image that shows the total area of ​​the retina predicted to be affected by GA pathology at T4.

[0091] In this exemplary experiment, the growth prediction system 150 is also implemented using a second deep learning model 414 (i.e., Model 2). The second deep learning model 414 is implemented using a Multichannel UNet trained to infer T4 whole GA lesions from a combination of T1 FAF images 402 and T2 FAF images 404.

[0092] In this exemplary experiment, the growth prediction system 150 is also implemented using a third deep learning model 414 (i.e., Model 3). The third deep learning model 414 is implemented using a Sequential Label UNet trained to infer the T3 growth image 408 and the T4 growth image 406 from the T2 FAF image 404. The T3 growth image 408 may be an example of an implementation for the growth image 175 of FIG. 1. The T3 growth image 408 is a growth image that shows the total area of ​​the retina predicted to be affected by GA pathology at T3.

[0093] These three whole-lesion models were trained using loss functions including Dice, Dice cross-entropy, Tversky (a = 0.6, b = 0.4), Tversky (a = 0.4, b = 0.6), and focal loss.

[0094] III.B.2. Exemplary Multiclass Model Three exemplary full-lesion models are described with respect to FIG. 4B.

[0095] 4B is a schematic diagram of a different exemplary deep learning model used to implement the growth prediction system 150 from FIG. 1, according to one or more embodiments. In FIG. 4B, the growth prediction system 150 is implemented differently, trained to receive FAF images capturing GA lesions at various time points and generating predicted growth output (e.g., including growth images) for various future time points. These various time points may include, for example, T1, T2, T3, and T4 as described above, which are spaced apart by a consistent time interval of six months.

[0096] In this exemplary experiment, the growth prediction system 150 is also implemented using a fourth deep learning model 416 (i.e., Model 4), a fifth deep learning model 418 (i.e., Model 5), and a sixth deep learning model 420 (i.e., Model 6).

[0097] The fourth deep learning model 416 is implemented using a Simple UNet trained to infer a T2 growth image 422 and a T4-T2 growth image 424 using the T2 FAF image 404. The T2 growth image 422 shows the predicted total lesion at T2. The T4-T2 growth image 424 shows the area representing new growth between T4 and T2. In other words, the T4-T2 growth image 424 can show the difference between the predicted GA lesion at time T4 and the predicted GA lesion at time T2.

[0098] The fifth deep learning model 418 is implemented using a Multichannel UNet trained to infer T2 growth images 422 and T4-T2 growth images 424 using the T2 FAF images 404 and T1 FAF images 402.

[0099] The sixth deep learning model 420 is implemented using a Sequential UNet trained to use the T2 FAF image 404 to infer a T2 growth image 422, a T4-T2 growth image 424, and a T3-T2 growth image 426. The T3-T2 growth image 426 shows areas representing new growth between T3 and T2. In other words, the T3-T2 growth image 426 can show the difference between the predicted GA lesion at time T3 and the predicted GA lesion at time T2.

[0100] These three multiclass models were trained using loss functions including Dice, generalized Dice, generalized Dice focal, Dice cross-entropy, Dice focal, and focal loss.

[0101] III.B.3. Exemplary Post-Processing for Whole-Lesion and Multi-Class Models Post-processing for the three exemplary whole-lesion models involved applying a sigmoid function to each prediction and setting a threshold of >0.5 to determine the final outcome prediction. The 1-year ROG for GA was derived by taking the difference between the predicted T4 lesion and grader T2 lesion annotations. This post-processing method is just one example of how post-processing could be performed.

[0102] For inference, post-processing of the three exemplary all-lesion models involved applying a softmax function per channel to each prediction with the highest probability determining the final prediction. The background (e.g., image background not considered a GA lesion) was considered a separate class, as during training. For the fourth and fifth deep learning models 416 and 418, T4-T2 growth images 424 were predicted, and no further post-processing was required. For the sixth deep learning model 420, T4-T2 growth images 424 were generated by adding the T3-T2 growth images 426 and the separate T4-T3 growth images.

[0103] Post-processing of the multi-class models also enabled the generation of predicted total lesion growth images (e.g., predicted total lesion area at future time points). For example, to obtain the total predicted GA lesions at T4, the T2 growth image 422 was combined with the T4-T2 growth image 424. Thus, these three multi-class models also enabled the generation of predicted total lesions at T4 and, when used in this manner, may be referred to as multi-class total lesion models. This method was implemented to reduce the variability in the prediction of growth areas derived from predicted T2 lesions.

[0104] III.C. Example Results The results were analyzed using the squared Pearson correlation coefficient (r 2 ) and estimation of a linear calibration function from the validation set (using ensemble least-squares linear regression). Predictions from the test set were transformed by the calibration function to obtain recalibrated predictions.

[0105] 5 is an example workflow illustrating example processing of three whole-lesion models according to one or more embodiments. For example, FIG. 5 shows example input FAF images that may be sent to the whole-lesion models (e.g., first deep learning model 410, second deep learning model 412, third deep learning model 414) and outputs that may be generated as described with respect to FIG. 4A above.

[0106] 6 is an example workflow illustrating example processing of three multi-class models according to one or more embodiments. For example, FIG. 6 shows an example input FAF image that may be sent to a full-lesion model (e.g., fourth deep learning model 416, fifth deep learning model 418, sixth deep learning model 420) and an output that may be generated as described with respect to FIG. 4B above.

[0107] 7 shows exemplary images of an exemplary workflow for post-processing of a whole-lesion model, according to one or more exemplary embodiments. In FIG. 7, image 702 is the T4 whole-lesion prediction generated by the whole-lesion model, image 704 is the T2 annotation of the whole lesion by a human grader, and image 706 is the final predicted growth region between T4 and T2 (T4-T2) generated by subtracting image 704 from image 702.

[0108] 8 shows an example workflow for post-processing a multi-class model, according to one or more exemplary embodiments. In FIG. 8, image 802 is an example of T2 total lesion and T4-T2 ROG predictions that may be generated by the fourth deep learning model 416 and the fifth deep learning model 418. Image 804 is an example of T2 total lesion, T3-T2 ROG, and T4-T3 ROG predictions that may be generated by the sixth deep learning model 420. Image 806 is a T4-T2 prediction (ROG) that may be obtained directly from the T4-T2 prediction or by adding the T3-T2 and T4-T3 predictions.

[0109] 9 shows an example workflow for post-processing a multi-class whole-lesion model, according to one or more exemplary embodiments. In FIG. 9, image 902 shows T2 whole-lesion and T4-T2 predictions that may be generated by the fourth deep learning model 416 and the fifth deep learning model 418. Image 904 shows the sum of the T2 whole-lesion and T4-T2 ROG predictions to generate a T4 whole-lesion prediction. Image 906 shows the T2 annotation by a human grader (G1). Image 908 shows the T4-T2 prediction (ROG) generated by subtracting the T2 annotation from the T4 prediction.

[0110] 10 shows examples of predicted growth images generated for different deep learning models, according to one or more embodiments. Set A of images shows predicted 1-year ROG (e.g., T4-T2 growth images) predicted for an exemplary input T2 FAF image, and shows ground truth T4 FAF images for the second deep learning model 412. Set B of images shows predicted 1-year ROG (e.g., T4-T2 growth images) predicted for an exemplary input T2 FAF image, and shows ground truth T4 FAF images for the fifth deep learning model 418 implemented for multi-class only. Set C of images shows predicted 1-year ROG (e.g., T4-T2 growth images) predicted for an exemplary input T2 FAF image, and shows ground truth T4 FAF images for the fifth deep learning model 418 implemented as multi-class whole-lesion.

[0111] FIG. 11 is a table showing various results of an example experiment for each of six different models for training, validation, and test sets, according to one or more embodiments.

[0112] IV. Example Output Use Cases In general, prediction system 100 and / or growth prediction system 150, which predicts GA growth output at a future time point, including a visual illustration (e.g., a graphical display) of the predicted location relative to the subject's retina, are an improvement to the technical field of GA lesion prediction and treatment due to the reduction in cost, time, expense, and / or computing resources that can be achieved. Furthermore, in some cases, the predicted GA growth output can be generated as accurate and / or reliable as a human grader, such that growth prediction system 150 can be successfully relied upon for use in clinical settings. For example, GA growth output 170, including mask 185, can be used to determine whether a subject is a candidate for a clinical trial, which clinical trial to assign the subject to, how to customize the subject's treatment, how to monitor the subject's progress during a clinical trial, or a combination thereof. The techniques described herein can be used to predict the prognosis of one or more subjects, predict the responsiveness of one or more subjects to various treatments, identify treatments predicted to be effective for individual subjects, assign one or more subjects to the appropriate arm within a clinical trial, or a combination thereof.

[0113] In general, prediction system 100 and / or growth prediction system 150, which predicts GA growth output at a future time point, where the GA growth output includes a visual illustration of the predicted location relative to the subject's retina, can be used to generate an output including an indication of whether the subject is eligible for a clinical trial to test a medical treatment for geographic atrophy, and thus the predicted GA growth output is an improvement to the art of GA lesion prediction and treatment. In some embodiments, this output can be used to enroll the subject in a clinical trial, exclude the subject from participating in a clinical trial, customize a protocol in a clinical trial for the subject, or enroll the subject in a different clinical trial.

[0114] In general, prediction system 100 and / or growth prediction system 150, in which the GA growth output predicts GA growth output at a future time point, including a visual illustration of the predicted location relative to the subject's retina, are an improvement to the field of GA lesion prediction and treatment because the predicted GA growth output is generated with an accuracy that can be successfully relied upon for use in clinical settings. For example, GA growth output 170, including mask 185, can be used to determine whether a subject is a candidate for a clinical trial, which clinical trial to assign the subject to, how to customize the subject's treatment, how to monitor the subject's progress during a clinical trial, or a combination thereof. The techniques described herein can be used to predict the prognosis of one or more subjects, predict the responsiveness of one or more subjects to various treatments, identify treatments predicted to be effective for individual subjects, assign one or more subjects to appropriate arms within a clinical trial, or a combination thereof.

[0115] In general, prediction system 100 and / or growth prediction system 150, which predict GA growth output at a future time point, provide the technical effect of identifying and generating a GA growth output that includes a visual illustration of the predicted location relative to the subject's retina.

[0116] V. Examples of Computer-Implemented Systems 12 is a block diagram of a computer system according to various embodiments. The computer system 1200 may be an example of one implementation for the computing platform 105 previously described in FIG.

[0117] In one or more examples, computer system 1200 may include a bus 1202 or other communication mechanism for communicating information and a processor 1204 coupled with bus 1202 for processing information. In various embodiments, computer system 1200 may also include memory, which may be random access memory (RAM) 1206 or other dynamic storage device, coupled to bus 1202 for determining instructions to be executed by processor 1204. The memory may also be used for storing temporary variables or other intermediate information during execution of instructions to be executed by processor 1204. In various embodiments, computer system 1200 may further include read-only memory (ROM) 1208 or other static storage device coupled to bus 1202 for storing static information and instructions for processor 1204. A storage device 1210, such as a magnetic disk or optical disk, may be provided and coupled to bus 1202 for storing information and instructions.

[0118] In various embodiments, computer system 1200 may be coupled via bus 1202 to a display 1212, such as a cathode ray tube (CRT) or liquid crystal display (LCD), for displaying information to a computer user. An input device 1214, including alphanumeric and other keys, may be coupled to bus 1202 for communicating information and command selections to processor 1204. Another type of user input device is a cursor control 1216, such as a mouse, joystick, trackball, gesture input device, eye-gaze-based input device, or cursor direction keys, for communicating directional information and command selections to processor 1204 and for controlling cursor movement on display 1212. This input device 1216 typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allow the device to specify a position in a plane. However, it should be understood that input devices 1214 that allow three-dimensional (e.g., x, y, and z) cursor movement are also contemplated herein.

[0119] Consistent with a particular implementation of the present teachings, results may be produced by computer system 1200 in response to processor 1204 executing one or more sequences of one or more instructions contained in RAM 1206. Such instructions may be read into RAM 1206 from another computer-readable medium or computer-readable storage medium, such as storage device 1210. Execution of the sequences of instructions contained in RAM 1206 may cause processor 1204 to perform the processes described herein. Alternatively, hard-wired circuitry may be used in place of or in combination with software instructions to implement the present teachings. Thus, implementation of the present teachings is not limited to any specific combination of hardware circuitry and software.

[0120] The terms "computer-readable medium" (e.g., data store, data storage, storage device, data storage device, etc.) or "computer-readable storage medium" as used herein refer to any medium that participates in providing instructions to processor 1204 for execution. Such a medium may take many forms, including, but not limited to, non-volatile media, volatile media, and transmission media. Examples of non-volatile media may include, but are not limited to, optical disks, solid-state disks, and magnetic disks, such as storage device 1210. Examples of volatile media may include, but are not limited to, dynamic memory, such as RAM 1206. Examples of transmission media may include, but are not limited to, coaxial cables, copper wire, and fiber optics, including the wires that comprise bus 1202.

[0121] Common forms of computer-readable media include, for example, floppy disks, hard disks, magnetic tape or any other magnetic medium, CD-ROMs, any other optical medium, punch cards, paper tape, any other physical medium with a pattern of holes, RAM, PROMs, and EPROMs, flash EPROMs, any other memory chip or cartridge, or any other tangible medium from which a computer can read.

[0122] In addition to computer-readable media, instructions or data may be provided as signals on a transmission medium included in a communication device or system to provide one or more sequences of instructions to the processor 1204 of the computer system 1200 for execution. For example, a communication device may include a transceiver having signals indicative of instructions and data. The instructions and data are configured to cause one or more processors to implement the functions outlined in the disclosure herein. Representative examples of data communication transmission connections may include, but are not limited to, a telephone modem connection, a wide area network (WAN), a local area network (LAN), an infrared data connection, an NFC connection, an optical communication connection, etc.

[0123] It should be understood that the methodologies, flowcharts, diagrams, and accompanying disclosure described herein may be implemented using computer system 1200 as a standalone device or over a distributed network of shared computer processing resources, such as a cloud computing network.

[0124] The methodologies described herein may be implemented by various means depending on the application. For example, these methodologies may be implemented in hardware, firmware, software, or any combination thereof. In the case of a hardware implementation, the processing unit may be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described herein, or combinations thereof.

[0125] In various implementations, the methods of the present teachings can be implemented as firmware and / or software programs and applications written in conventional programming languages ​​such as C, C++, Python, etc. When implemented as firmware and / or software, the embodiments described herein can be implemented on a non-transitory computer-readable medium having stored thereon a program for causing a computer to perform the above-described methods. It should be understood that the various engines described herein can be provided on a computer system, such as computer system 1200, whereby processor 1204 performs the analyses and decisions provided by these engines in response to instructions provided by any one or combination of memory components RAM 1206, ROM 1208, or storage device 1210, and user input provided via input device 1214.

[0126] VI. List of Embodiments Embodiment 1: A method comprising: receiving fundus autofluorescence (FAF) image data of a subject's retina, the FAF image data including a first FAF image associated with a first time point; generating image input for a deep learning system using the FAF image data; and generating, via the deep learning system, a predicted growth output of geographic atrophy (GA) lesions in the retina using the image input, the predicted growth output being associated with at least one future time point after the first time point.

[0127] Embodiment 2: The method of embodiment 1, wherein the predicted growth output includes a first growth image showing a first predicted growth area of ​​the GA lesion relative to the subject's retina between the first baseline time point and a first future time point after the first baseline time point.

[0128] Embodiment 3: The method of embodiment 2, wherein the predicted growth output further includes a second growth image showing a second predicted growth area of ​​the GA lesion relative to the subject's retina between the second baseline time point and a second future time point after the second baseline time point.

[0129] Embodiment 4: The method of embodiment 3, wherein the second reference time point and the first reference time point are the same or different time points, and the second future time point is different from the first future time point.

[0130] Embodiment 5: The method of any one of embodiments 2 to 4, wherein the first reference time point is a first time point or a time point between the first time point and a first future time point.

[0131] Embodiment 6: The method of any one of embodiments 1 to 5, wherein the predicted growth output comprises a growth image showing areas of the retina predicted to be affected by GA pathology at a selected future time point.

[0132] Embodiment 7: The method of any one of embodiments 1 to 6, wherein the predicted growth output comprises a calculated area for the entire region of the retina in the FAF image data that is predicted to be affected by the GA pathology at the selected future time point.

[0133] Embodiment 8: The method of any one of embodiments 1 to 7, wherein the predicted growth output comprises a calculated area for the entire region of the retina in the FAF image data that is predicted to be affected by the GA pathology at the selected future time point.

[0134] Embodiment 9: The method of embodiment 1, wherein the predicted growth output comprises a growth image showing areas of new growth of the GA lesion between the two time points.

[0135] Embodiment 10: The method of embodiment 9, wherein the predicted growth output comprises a calculated area of ​​new growth between two time points.

[0136] Embodiment 11: A method according to any one of embodiments 1 to 10, wherein the FAF image data further includes a second FAF image associated with a second time point after the first time point, and generating the image input includes preprocessing each of the first FAF image and the second FAF image so that the image input includes the first preprocessed FAF image and the second preprocessed FAF image.

[0137] Embodiment 12: The method of embodiment 10, wherein the predicted growth output includes a growth image showing predicted growth areas of GA lesions for the subject's retina for the selected future time point, the growth image comprising an image background associated with the first FAF image or the second FAF image, and a mask on the image background, the mask identifying predicted growth areas for the subject's retina for the selected future time point.

[0138] Embodiment 13: The method of embodiment 10, wherein the deep learning system includes a trained long-short-term memory convolutional neural network.

[0139] Embodiment 14: The method of any one of embodiments 1 to 13, wherein the deep learning system includes a trained convolutional neural network (CNN), and the training dataset used to train the trained CNN includes multiple training sets corresponding to multiple eyes, and each training set of the multiple training sets includes training FAF images corresponding to four different time points.

[0140] Embodiment 15: The method of embodiment 14, wherein the training FAF images of the training dataset are stratified by at least one of baseline lesion area, lesion growth rate, foveal involvement, or focus.

[0141] Embodiment 16: The method of embodiment 14 or 15, wherein the training FAF images of the training set include four FAF images spaced apart over time by consistent time intervals.

[0142] Embodiment 17: A method of training a deep learning system, the method comprising: receiving a plurality of training sets for a plurality of retinas of a plurality of subjects, wherein each training set of the plurality of training sets includes training FAF images for at least two different time points; generating training inputs for the deep learning system using the plurality of training sets; and training the deep learning system to generate predicted growth outputs based on FAF image data of the retinas of selected subjects, wherein the predicted growth outputs indicate predicted growth of geographic atrophy (GA) lesions in the retina for at least one future time point.

[0143] Embodiment 18: The method of embodiment 17, wherein the predicted growth output comprises a growth image showing areas of the retina predicted to be affected by GA pathology for future time points.

[0144] Embodiment 19: The method of embodiment 17 or embodiment 18, wherein the predicted growth output comprises a calculated area for the entire region of the retina predicted to be affected by GA pathology for the future time point.

[0145] Embodiment 20: A system comprising one or more data processors and a non-transitory computer-readable storage medium containing instructions that, when executed on the one or more data processors, cause the one or more data processors to perform a method according to any one of embodiments 1 to 19.

[0146] Embodiment 21: A computer program product tangibly embodied in a non-transitory machine-readable storage medium comprising instructions configured to cause one or more data processors to perform the method of any one of embodiments 1 to 19.

[0147] The present disclosure is not limited to these exemplary embodiments described herein, and various configurations and implementations of the elements, components, models, and steps described herein may be used to perform GA growth prediction.

[0148] VII. Illustrative Contexts and Definitions The present disclosure is not limited to these exemplary embodiments and applications or the manner in which the exemplary embodiments and applications operate or are described herein. Further, the figures may show simplified or partial views, and the dimensions of the elements in the figures may be exaggerated or out of proportion.

[0149] Unless otherwise defined, scientific and technical terms used in connection with the present teachings described herein shall have the meanings commonly understood by those of ordinary skill in the art. Furthermore, unless otherwise required by context, singular terms shall include the plural and plural terms shall include the singular. Generally, the nomenclature and techniques utilized in connection with chemistry, biochemistry, molecular biology, pharmacology, and toxicology are described herein and are well known and commonly used in the art.

[0150] As used herein, when the terms "on," "attached to," "connected to," "coupled to," or similar terms are used, an element (e.g., a component, material, layer, substrate, etc.) can be "on," "attached to," "connected to," or "coupled to" another element, regardless of whether the element is directly on, directly attached to, connected to, or coupled to the other element, or whether there are one or more intervening elements between the one and the other element. Furthermore, when a reference is made to a list of elements (e.g., elements a, b, c), such reference is intended to include any one of the listed elements by itself, any combination of fewer than all of the listed elements, and / or all combinations of the listed elements. The division of sections herein is for ease of reference only and does not limit any combination of elements described.

[0151] The term "subject" may refer to a subject of a clinical trial, a person receiving treatment, a person undergoing anti-cancer therapy, a person being monitored for remission or recovery, a person undergoing preventative health analysis (e.g., due to a medical history), or any other person or subject patient. In various instances, "subject" and "patient" may be used interchangeably herein.

[0152] As used herein, "substantially" means sufficient to function for its intended purpose. Thus, the term "substantially" allows for slight, insignificant variations from an absolute or perfect state, dimension, measurement, result, etc., as would be expected by one of ordinary skill in the art, but does not noticeably affect overall performance. With respect to a parameter or characteristic that is a number or can be expressed as a number, "substantially" means within 10%.

[0153] The term "ones" means two or more.

[0154] As used herein, the term "plurality" can be 2, 3, 4, 5, 6, 7, 8, 9, 10 or more.

[0155] As used herein, the term "set of" means one or more. For example, a set of items includes one or more items.

[0156] As used herein, the phrase "at least one of," when used in conjunction with a list of items, means that different combinations of one or more of the listed items may be used, and that only one of the items in the list may be required. An item may be a specific object, thing, step, action, process, or category. In other words, "at least one of" means that any combination or number of items from the list may be used, but not all of the items in the list may be required. For example, without limitation, "at least one of item A, item B, or item C" means item A, item A, and item B; item B; item A, item B, and item C; item B and item C; or item A and item C. In some cases, "at least one of item A, item B, or item C" means, without limitation, two items A, one item B, and ten items C, four items B, and seven items C, or some other suitable combination.

[0157] As used herein, a "model" may include one or more algorithms, one or more mathematical techniques, one or more machine learning algorithms, or a combination thereof.

[0158] As used herein, "machine learning" can be the practice of using algorithms to analyze data, learn from it, and then make decisions or predictions about something in the world. Machine learning uses algorithms that can learn from data without relying on rule-based programming.

[0159] As used herein, "artificial neural network" or "neural network" (NN) can refer to a mathematical algorithm or computational model that mimics an interconnected group of artificial nodes or neurons that process information based on a connectionist approach to computation. A neural network, also called a neural net, may use one or more layers of nonlinear units to predict an output for a received input. Some neural networks include one or more hidden layers in addition to an output layer. The output of each hidden layer is used as the input to the next layer in the network, i.e., the next hidden layer or the output layer. Each layer of the network generates an output from the received input according to the current value of each parameter set. In various embodiments, a reference to a "neural network" can be a reference to one or more neural networks.

[0160] Neural networks may process information in two ways: when they are being trained, they are in training mode; and when they put what they have learned into practice, they are in inference (or prediction) mode. Neural networks learn through a feedback process (e.g., backpropagation) that allows the network to adjust the weight coefficients of individual nodes in intermediate hidden layers (change their behavior) so that their outputs match those of the training data. In other words, neural networks learn by being fed training data (training examples) and eventually learn how to arrive at the correct output, even when presented with a new range or set of inputs. The neural network may include, for example, but is not limited to, at least one of a feedforward neural network (FNN), a recurrent neural network (RNN), a modular neural network (MNN), a convolutional neural network (CNN), a residual neural network (ResNet), an ordinary differential equation neural network (neural-ODE), a LSTM, or another type of neural network.

[0161] As used herein, "lesion" can refer to the area of ​​an organ or tissue that has been damaged through injury or disease. This area can be continuous or discontinuous. For example, as used herein, a lesion can include multiple areas. A geographic atrophy (GA) lesion is an area of ​​the retina that suffers from chronic progressive degeneration. As used herein, a GA lesion can include one lesion (for example, one continuous lesion area) or multiple lesions (for example, a discontinuous lesion area that is composed of multiple separate lesions).

[0162] As used herein, "lesion area" can be the total area covered by the lesion, regardless of whether the lesion is a continuous or discontinuous area.

[0163] As used herein, "long-term" can refer to over a period of time, which can be days, weeks, months, years, or some other time scale.

[0164] As used herein, the "growth rate" corresponding to GA lesions can be the long-term change in the lesion area of ​​GA lesions. In other words, the "growth rate" can be the change in the lesion area over time. In some cases, this growth rate can be an annual growth rate. This growth rate can also be referred to as lesion growth rate or GA growth rate.

[0165] VIII. Further Considerations Headings and / or subheadings between sections and subsections of this document are included solely to improve readability and do not imply that features cannot be combined across sections and subsections, and thus the sections and subsections do not describe separate embodiments.

[0166] While the present teachings will be described in connection with various embodiments, it is not intended that the present teachings be limited to such embodiments. On the contrary, the present teachings encompass various alternatives, modifications, and equivalents, as will be understood by those skilled in the art. The present description provides preferred exemplary embodiments and is not intended to limit the scope, applicability, or configuration of the present disclosure. Instead, the present description of preferred exemplary embodiments provides those skilled in the art with an enabling description for practicing various embodiments. It will be understood that various changes can be made in the function and arrangement of elements without departing from the spirit and scope as set forth in the appended claims. Accordingly, such modifications and variations are deemed to be within the scope set forth in the appended claims. Furthermore, the terms and expressions used are used as terms of description rather than limitation, and there is no intention in the use of such terms and expressions to exclude equivalents of the features shown and described, or portions thereof, but it will be recognized that various modifications are possible within the scope of the present disclosure.

[0167] In describing various embodiments, the specification may present a method and / or process as a particular sequence of steps. However, to the extent that the method or process does not rely on the particular order of steps set forth herein, the method or process should not be limited to the particular order of steps set forth, and as one skilled in the art will readily appreciate, the order may be changed and still remain within the spirit and scope of the various embodiments.

[0168] Some embodiments of the present disclosure include a system including one or more data processors. In some embodiments, the system includes a non-transitory computer-readable storage medium including instructions that, when executed on the one or more data processors, cause the one or more data processors to perform some or all of one or more methods and / or some or all of one or more processes disclosed herein. Some embodiments of the present disclosure include a computer program product tangibly embodied in a non-transitory machine-readable storage medium including instructions configured to cause one or more data processors to perform some or all of one or more methods and / or some or all of one or more processes disclosed herein.

[0169] Specific details are provided herein to provide an understanding of the embodiments. However, it will be understood that the embodiments may be practiced without these specific details. For example, circuits, systems, networks, processes, and other components may be shown as components in block diagram form in order to avoid obscuring the embodiments in unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring the embodiments.

Claims

1. 1. A method comprising: receiving fundus autofluorescence (FAF) image data of a subject's retina; receiving the FAF image data including a first FAF image associated with a first time point; using the FAF image data to generate image input for a deep learning system; and generating, via the deep learning system, a predicted growth output of geographic atrophy (GA) lesions in the retina using the image input; generating the predicted growth output associated with at least one future time point after the first time point; A method comprising:

2. 2. The method of claim 1, wherein the predicted growth output comprises a first growth image showing a first predicted area of ​​growth of the GA lesion relative to the retina of the subject between a first reference time point and a first future time point after the first reference time point.

3. 3. The method of claim 2, wherein the predicted growth output further comprises a second growth image showing a second predicted growth area of ​​the GA lesion relative to the retina of the subject between a second reference time point and a second future time point after the second reference time point.

4. the second reference time point and the first reference time point are the same or different time points; The method of claim 3 , wherein the second future time is different from the first future time.

5. The method according to any one of claims 2 to 4, wherein the first reference time point is the first time point or a time point between the first time point and the first future time point.

6. 6. The method of any one of claims 1 to 5, wherein the predicted growth output comprises a growth image showing areas of the retina predicted to be affected by the GA pathology at a selected future time point.

7. 7. The method of any one of claims 1 to 6, wherein the predicted growth output comprises a calculated area for the entire region of the retina in the FAF image of the FAF image data that is predicted to be affected by the GA lesion at a selected future time point.

8. 8. The method of any one of claims 1 to 7, wherein the predicted growth output comprises a calculated area for the entire region of the retina in the FAF image of the FAF image data that is predicted to be affected by the GA lesion at a selected future time point.

9. 10. The method of claim 1, wherein the predicted growth output comprises a growth image showing areas of new growth of the GA lesion between two time points.

10. 10. The method of claim 9, wherein the predicted growth output comprises a calculated area of ​​new growth between two time points.

11. the FAF image data further includes a second FAF image associated with a second time point that is after the first time point; generating the image input comprises: preprocessing each of the first and second FAF images such that the image input comprises a first preprocessed FAF image and a second preprocessed FAF image; The method according to any one of claims 1 to 10, comprising:

12. the predicted growth output includes a growth image showing a predicted area of ​​growth of the GA lesion for the retina of the subject for a selected future time point; The growth image is an image background associated with the first FAF image or the second FAF image; and a mask on the image background, the mask identifying the predicted growth region for the subject's retina for the selected future time point; The method of claim 10, comprising:

13. 11. The method of claim 10, wherein the deep learning system comprises a trained long short-term memory convolutional neural network.

14. the deep learning system includes a trained convolutional neural network (CNN); the training dataset used to train the trained CNN includes a plurality of training sets corresponding to a plurality of eyes, a training set of the plurality of training sets includes training FAF images corresponding to four different time points; The method according to any one of claims 1 to 13.

15. 15. The method of claim 14, wherein the training FAF images of the training dataset are stratified by at least one of baseline lesion area, lesion growth rate, foveal involvement, or focus.

16. 16. The method of claim 14 or 15, wherein the training FAF images of the training set include four FAF images spaced chronologically at consistent time intervals.

17. 1. A method for training a deep learning system, comprising: receiving a plurality of training sets for a plurality of retinas of a plurality of subjects; receiving, each training set of the plurality of training sets including training FAF images for at least two different time points; using the plurality of training sets to generate training inputs for a deep learning system; and training the deep learning system to generate a predicted growth output based on FAF image data of a selected subject's retina, the predicted growth output indicating a predicted growth of geographic atrophy (GA) lesions in the retina for at least one future time point; A method comprising:

18. 18. The method of claim 17, wherein the predicted growth output comprises a growth image showing areas of the retina predicted to be affected by the GA pathology for future time points.

19. 19. The method of claim 17 or claim 18, wherein the predicted growth output comprises a calculated area for the entire region of the retina predicted to be affected by the GA pathology for a future time point.

20. 1. A system comprising: one or more data processors; a non-transitory computer readable storage medium containing instructions that, when executed on said one or more data processors, cause said one or more data processors to perform the method of any one of claims 1 to 19; and A system comprising:

21. A computer program product tangibly embodied in a non-transitory machine-readable storage medium comprising instructions configured to cause one or more data processors to perform the method of any one of claims 1 to 19.

Images