Geographic atrophy progression prediction and differential gradient activation maps

Abstract

A method for assessing geographic atrophy. A set of retinal images is received. Each model of a plurality of models is trained to predict a set of geographic atrophy (GA) progression parameters for a GA lesion using the set of retinal images. A visualization output is generated for each model of the plurality of models. The visualization output for a corresponding model of the plurality of models provides information regarding how the corresponding model predicts the set of GA progression parameters using the set of retinal images.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 63 / 189,679, entitled “Geographic Atrophy Progression Prediction and Differential Gradient Activation Maps,” filed May 17, 2021, and incorporated by reference in its entirety herein.

[0002] Field This description generally relates to predicting geographic atrophy progression. More specifically, this description provides methods and systems for predicting geographic atrophy progression using various models and analyses performed on these models (e.g., gradient activation map analysis, ablation analysis). [Background technology]

[0003] background Age-related macular degeneration (AMD) is the leading cause of vision loss in patients over the age of 50. Geographic atrophy (GA) is one of two progressive stages of AMD and is characterized by progressive and irreversible loss of choriocapillaris, retinal pigment epithelium (RPE) and photoreceptors. GA progression varies from patient to patient, and currently there are no widely accepted treatments to prevent or slow down the progression of GA. Therefore, assessing the progression of GA in individual patients may be important for studying GA and developing effective treatments. Summary of the Invention

[0004] overview In one or more embodiments, a method for assessing geographic atrophy is provided. A set of retinal images is received. Each model of a plurality of models is trained to predict a set of geographic atrophy (GA) progression parameters for a GA lesion using the set of retinal images. A visualization output is generated for each model of the plurality of models. The visualization output of a corresponding model of the plurality of models provides information regarding how the corresponding model uses the set of retinal images to predict the set of GA progression parameters.

[0005] In one or more embodiments, a method for assessing geographic atrophy in the retina is provided. A set of retinal images is received. A set of geographic atrophy (GA) progression parameters for a geographic atrophy (GA) lesion in the retina is predicted using the set of retinal images and a deep learning model. A set of gradient activation maps corresponding to the set of retinal images is generated for the deep learning model. The gradient activation maps in the set of gradient activation maps of corresponding retinal images of the set of retinal images identify a set of regions in the corresponding retinal images that are relevant to the prediction of the set of GA progression parameters by the deep learning model.

[0006] In one or more embodiments, a system for managing an assessment of geographic atrophy includes a memory including a machine-readable medium including machine-executable code, and a processor coupled to the memory. The processor is configured to execute the machine-executable code to cause the processor to receive a set of retinal images, train each model of the plurality of models to predict a set of geographic atrophy (GA) progression parameters for a geographic atrophy (GA) lesion using the set of retinal images, and generate a visualization output for each model of the plurality of models. The visualization output of a corresponding model of the plurality of models provides information regarding how the corresponding model uses the set of retinal images to predict the set of GA progression parameters. [Brief description of the drawings]

[0007] For a more complete understanding of the principles disclosed herein, and the advantages thereof, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which:

[0008] [Figure 1] FIG. 1 is a block diagram of a lesion assessment system 100 according to various embodiments.

[0009] [Diagram 2] 1 is a flow chart of a process for assessing geographic atrophy according to various embodiments.

[0010] [Diagram 3] 1 is a flow chart of a process for assessing geographic atrophy according to various embodiments.

[0011] [Figure 4] 4 is a flowchart of a process 400 for improving model performance according to various embodiments.

[0012] [Diagram 5] 11 is a chart comparing gradient activation maps generated for two deep learning models where the GA lesion is a monofocal lesion, according to one or more embodiments.

[0013] [Figure 6] 1 is a chart comparing gradient activation maps generated for two deep learning models where the GA lesion is a multifocal lesion, according to one or more embodiments.

[0014] [Figure 7] 1 is a chart showing an exemplary ablation image in accordance with one or more embodiments.

[0015] [Figure 8] FIG. 1 is a block diagram of a computer system in accordance with various embodiments.

[0016] It should be understood that the figures are not necessarily drawn to scale, and that objects within the figures are not necessarily drawn to scale relative to each other. The figures are representations intended to bring clarity and understanding to the various embodiments of the devices, systems, and methods disclosed herein. Wherever possible, the same reference numbers are used throughout the drawings to refer to the same or similar parts. Furthermore, it should be understood that the drawings are not intended to limit the scope of the present teachings in any way. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] I. Overview The ability to accurately predict progression of geographic atrophy (GA) based on baseline assessments may be useful in many different scenarios. Parameters related to GA progression include lesion growth velocity and baseline lesion area. Baseline lesion area is the total area of ​​GA lesions (e.g., mm 2 Baseline lesion area has been shown to be an indicator of GA progression, which can be assessed based on lesion growth velocity. Lesion growth velocity, also referred to herein as GA lesion growth rate or growth velocity, is the change in lesion area over a period of time. Growth velocity is often annualized (e.g., in mm 2 / year).

[0018] Being able to automatically predict baseline lesion area, lesion growth rate, or both using input retinal images and models (e.g., deep learning models) may help improve patient screening, enrichment, and / or stratification in clinical trials with the goal of slowing GA progression, thereby allowing improved evaluation of treatment efficacy. Improving prediction of such GA progression parameters (e.g., lesion growth rate, baseline lesion area) may also improve the efficacy of clinical trials, for example, but not limited to, by allowing covariate adjustment during analysis. Covariate adjustment may be used to reduce the variance of treatment effect estimates in clinical trials and increase the power of clinical trials. Furthermore, in some cases, prediction of GA progression parameters may be used to understand disease pathogenesis via correlation with genotypic or phenotypic signatures.

[0019] GA lesions may be imaged by various imaging modalities, including, but not limited to, fundus autofluorescence (FAF) and optical coherence tomography (OCT). For example, fundus autofluorescence (FAF) images may be input into one or more models (e.g., one or more deep learning models) to predict baseline lesion area, lesion growth rate, or both. The FAF image may be a baseline FAF image taken at a baseline time point. The baseline time point may be the start of a clinical trial, the time of initial assessment, a time point immediately prior to the first administration of treatment, a time point coinciding with the first administration of treatment, the same day as the first administration of treatment, or some other baseline time point.

[0020] The embodiments described herein recognize that it may be desirable to understand how a deep learning model uses one or more baseline FAF images to predict lesion growth rate, baseline lesion area, or both. For example, it may be desirable to understand which regions or features of a baseline FAF image contribute to the predicted lesion growth rate. Identifying image regions or features that are relevant to (or drive) the prediction of lesion growth rate may help to identify or localize new biomarkers, gain insight into GA pathology, build confidence that the deep learning model is not focusing on spurious or irrelevant image regions or features, and / or improve the performance of the deep learning model.

[0021] Thus, embodiments described herein provide methods and systems for evaluating image regions or features of an image (e.g., a FAF image) that contribute to a model (e.g., a deep learning model) predicting GA progression parameters (e.g., lesion growth rate, baseline lesion area, etc.). In one or more embodiments, various types of visualizations are used to understand how such models process their inputs.

[0022] For example, gradient activation maps may be used to indicate which regions of an input image (e.g., a FAF image) contribute to the final output of a deep learning model. Gradient activation maps may visually identify (e.g., via color, shading, highlighting, pattern, etc.) one or more regions in an image associated with predictions made by the model (e.g., predicted growth rate and / or predicted lesion area). Such gradient activation maps may be used to validate a deep learning model by checking whether the model focuses on non-false relevant portions of the image based on what is known or expected. Furthermore, comparing gradient activation maps generated for different models may help select the best model to use to predict lesion growth rate, lesion area, or both. Generating gradient activation maps and using these gradient activation maps to evaluate which image regions were associated (e.g., most relevant) with predictions made by a deep learning model may be computationally inexpensive compared to other methods for performing such operations. In this manner, the overall time and / or computing resources required to perform such operations may be reduced. Furthermore, using the gradient activation maps described herein does not require annotation of image regions (e.g., by a human or other model), thereby making the overall process more efficient and / or more accurate.

[0023] In one or more embodiments, the gradient activation map may be used to identify modifications that can be made to a deep learning model to improve its performance. For example, the gradient activation map may be used to identify new biomarkers or localize known biomarkers, thereby narrowing the focus of the deep learning model. In some cases, the gradient activation map may be used to narrow the focus of the deep learning model to reduce the time and computational resource consumption of the deep learning model while maintaining a desired level of predictive accuracy.

[0024] One or more embodiments described herein use ablation analysis to directly derive the portions of the retinal image that contribute to one or more GA progression parameters predicted by the deep learning model. The ablation analysis may include performing segmentation of the retinal image and then ablation (e.g., removing) various combinations of the segmented regions. For example, a segmentation algorithm may be used to segment (or separately identify) the GA lesion, the edge (e.g., a 500 μm wide margin) around the GA lesion, and the background (e.g., any portion of the image that is not identified as a GA lesion or edge). Various combinations of the GA lesion, edge, and background may be ablated from the retinal image to form an ablation image that is then fed as an input to the deep learning model. Comparing the performance of models based on different types of ablation image inputs allows for the determination of which image regions or features are associated (e.g., most relevant) with one or more GA progression parameters predicted by the deep learning model.

[0025] The information provided by the ablation analysis may be used to validate the deep learning model and determine whether the deep learning model is focused on non-false relevant portions of the retinal image based on what is known or expected. In one or more embodiments, the ablation analysis may be used to generate output for use in improving the performance of the deep learning model. For example, the ablation analysis may be used to identify new biomarkers or localize known biomarkers, thereby adjusting the focus of the deep learning model more narrowly. In some cases, the ablation analysis may be used to narrow the focus of the deep learning model to reduce the time and computational resource consumption of the deep learning model while maintaining a desired level of predictive accuracy. In other embodiments, the results of the ablation analysis may be used to generate output that identifies modifications that can be made to the deep learning model to improve its accuracy and / or reliability.

[0026] Recognizing and taking into account the importance and usefulness of methodologies and systems that can provide the above-mentioned improvements, the present specification describes various embodiments for evaluating GA progression by predicting one or more GA progression parameters using one or more models and evaluating how these models make their predictions using gradient activation maps and / or ablation analysis. For example, the various embodiments described herein provide methods and systems for generating visualization outputs (e.g., gradient activation maps) that can be used to better understand how these models (e.g., deep learning models) use different parts of an image (e.g., FAF image) to predict the growth rate of GA lesions. Furthermore, the various embodiments described herein also provide methods and systems for validating models using ablation analysis and / or identifying image features that are relevant (e.g., most relevant) to predicting growth rate.

[0027] II. An Exemplary System for Predicting Geographic Atrophy (GA) Progression FIG. 1 is a block diagram of a lesion-assessing system 100 according to various embodiments. The lesion-assessing system 100 is used to assess geographic atrophy (GA) lesions in the retina of a subject. The lesion-assessing system 100 includes a computing platform 102, a data storage 104, and a display system 106. The computing platform 102 may take various forms. In one or more embodiments, the computing platform 102 includes a single computer (or computer system) or multiple computers in communication with each other. In other examples, the computing platform 102 takes the form of a cloud computing platform.

[0028] The data storage 104 and the display system 106 each communicate with the computing platform 102. In some examples, the data storage 104, the display system 106, or both may be considered part of or otherwise integrated with the computing platform 102. Thus, in some examples, the computing platform 102, the data storage 104, and the display system 106 may be separate components that communicate with each other, while in other examples some combination of these components may be integrated together.

[0029] The lesion assessment system 100 includes an image processor 108, which may be implemented using hardware, software, firmware, or a combination thereof. In one or more embodiments, the image processor 108 is implemented in the computing platform 102.

[0030] The image processor 108 receives an image input 109 for processing. The image input 109 includes one or more retinal images. In one or more embodiments, the image input 109 includes one or more retinal images generated at a baseline or reference time point. In some embodiments, the image input 109 may be referred to as a baseline image input. In one or more embodiments, the image input 109 includes a set of retinal images 110. The set of retinal images 110 may include, for example, but not limited to, a set of fundus autofluorescence (FAF) images, a set of optical coherence tomography (OCT) images, or both. The set of FAF images may be, for example, a set of baseline FAF images. The set of OCT images may be, for example, a set of baseline OCT images. The baseline images (e.g., baseline FAF or OCT images) are images taken at a baseline time point. The baseline time point may be the start of a clinical trial, the time of the first evaluation or first clinic or clinical trial visit, a time point immediately prior to the first administration of a treatment, a time point coinciding with the first administration of a treatment, the same day as the first administration of a treatment, or some other baseline time point. In other embodiments, the set of retinal images 110 may include one or more other types of retinal images (eg, color fundus (CF) photographic images, near-infrared (NIR) images, etc.).

[0031] The image processor 108 processes an image input 109 (e.g., a set of retinal images 110) using a number of models 112 to predict a set of geographic atrophy (GA) progression parameters 114. A GA progression parameter is a parameter that is associated with (is indicative of, or may be used to indicate) GA progression of a GA lesion. A GA lesion may be a continuous or discontinuous lesion. For example, the set of GA progression parameters 114 may include a lesion area 115, a growth rate 116, or both. The lesion area 115 may be a baseline lesion area of ​​a GA lesion. The growth rate 116 (or lesion growth rate) may be the change in lesion area over a defined period of time. The growth rate 116 may be annualized (e.g., in mm 2 / year).

[0032] Each of the models 112 may be implemented in any of a number of different ways, including, for example, but not limited to, using one or more deep learning models. The models 112 include, for example, a first model 117 and a second model 118. In one or more embodiments, each of the first model 117 and the second model 118 includes a deep learning model. The deep learning models of the first model 117, the second model 118, or both may include, for example, but not limited to, any number or combination of neural networks. In one or more embodiments, the deep learning models may include a convolutional neural network (CNN) system that includes one or more neural networks. Each of these one or more neural networks may itself be a convolutional neural network. In some cases, the deep learning models include multiple subsystems, each including one or more neural networks.

[0033] In one or more embodiments, image processor 108 includes a model analyzer 120. Model analyzer 120 may be used to generate visualization outputs for each of models 112. For example, model analyzer 120 may generate a first visualization output 122 for first model 117 and a second visualization output 123 for second model 118.

[0034] The first visualization output 122 may include a set of visualizations for the set of retinal images 110. For example, the first visualization output 122 may include a visualization of each image in the set of retinal images 110. The first visualization output 122 provides information about how the first model 117 uses the set of retinal images 110 to predict the set of GA progression parameters 114. The second visualization output 123 may include a set of visualizations for the set of retinal images 110. For example, the second visualization output 123 may include a visualization of each image in the set of retinal images 110. The second visualization output 123 provides information about how the second model 118 uses the set of retinal images 110 to predict the set of GA progression parameters 114.

[0035] In one or more embodiments, the first visualization output 122 includes a set 124 of gradient activation maps for the set of retinal images 110, and the second visualization output 123 includes a set 126 of gradient activation maps for the set of retinal images 110. Each gradient activation map in the set 124 of gradient activation maps associated with the first model 117 and the set 126 of gradient activation maps associated with the second model 118 may be generated using a gradient weighting activation mapping technique for a corresponding image in the set of retinal images 110. Each gradient activation map indicates a set of regions in the corresponding retinal image that contributed to the set of GA progression parameters 114 predicted by the respective associated model (e.g., the first model 117 or the second model 118).

[0036] For example, the gradient activation map may visually identify (e.g., via color, shading, highlighting, pattern, etc.) one or more regions in the image associated with the prediction of the set of GA progression parameters 114 by the corresponding model. Additionally, coloring, shading, highlighting, patterns, text and / or numerical labels, other types of indicators, or combinations thereof may be used to indicate the degree of relevance. As an example, a range of colors between red, orange, yellow, and green to blue may be used to visually identify one or more regions in the image associated with the prediction of the set of GA progression parameters 114 by the model and their degree of relevance. For example, red may be used to identify one or more regions with the highest relevance to the prediction of the set of GA progression parameters 114, and blue may be used to identify one or more regions with the lowest relevance.

[0037] In one or more embodiments, the first visualization output 122 may be used to validate the first model 117, the second visualization output 123 may be used to validate the second model 118, or both. For example, the first visualization output 122 may be used to evaluate whether a portion of the image (e.g., one or more regions of the image) identified as being relevant to a prediction of the set of GA progression parameters 114 by the first model 117 matches an expectation. Similarly, the second visualization output 123 may be used to evaluate whether a portion of the image (e.g., one or more regions of the image) identified as being relevant to a prediction of the set of GA progression parameters 114 by the second model 118 matches an expectation.

[0038] In one or more embodiments, the first visualization output 122 and the second visualization output 123 may be used to determine whether any adjustments should be made to the first model 117 and the second model 118, respectively. For example, if the first visualization output 122 identifies an area of ​​the image most relevant to the prediction of the set of GA progression parameters 114 by the first model 117 is different than expected, the model analyzer 120 may generate an output indicating that adjustments should be made to the first model 117. The adjustments may include, for example, but are not limited to, retraining the first model 117, changing layers used in the first model 117, changing the architecture of the first model 117, combining the first model 117 with another model, or a combination thereof. The second model 118 may be similarly evaluated using the second visualization output 123.

[0039] In some embodiments, a first visualization output 122 generated for a first model 117 may be compared to a second visualization output 123 generated for a second model 118 to determine similarities and / or differences in how the first model 117 and the second model 118 predict the set of GA progression parameters 114. For example, the set of gradient activation maps 124 and the set of gradient activation maps 126 may be compared to determine whether the same or different regions were most relevant to the predictions made by the first model 117 and the second model 118.

[0040] The comparison may allow for the selection of the best model for use in predicting the set of GA progression parameters 114. For example, the model analyzer 120 may be used to generate a first visualization output 122 and a second visualization output 123 after the first model 117 and the second model 118 have each been trained and tested for predicting the set of GA progression parameters 114 based on the set of retinal images 110, which may include multiple training baseline FAF images. These two visualization outputs may be used to determine whether one model is more suitable for a particular type of GA lesion than the other model, whether the models perform similarly for the same type of GA lesion, etc. The information provided by these visualization outputs may then be used to determine which model to select for use in predicting the set of GA progression parameters 114 for a particular subject or group of subjects.

[0041] In one or more embodiments, the image processor 108 includes an image modifier 128. The image modifier 128 processes the image input 109 to generate a number of ablated images 130 that are fed as inputs to one or more of the models 112. Analyzing the performance of the models 112 using these ablated images 130 can help identify or localize new biomarkers, gain insight into GA pathology, and gain confidence that the models 112 are not focusing on spurious or irrelevant image regions or features (e.g., focusing only on the background).

[0042] The ablation image 130 may be generated in a variety of ways. For example, the ablation image may be formed by assigning pixels identified as corresponding to one or more selected regions (e.g., GA lesion, edge, background) as black. These selected regions may be identified using, for example, a segmentation algorithm. In one example, the ablation image is formed by painting edge and background pixels black such that the ablation image is an image of only GA lesions (i.e., a lesion-preserving image). In another example, the ablation lesion is formed by painting GA lesion and background pixels black such that the ablation image is an image of only edges (e.g., an edge-preserving image). In yet another example, the ablation lesion is formed by painting GA lesion and edge pixels black such that the ablation image is an image of only background (e.g., a background-preserving image). In yet another example, the ablation lesion is formed by painting GA lesion pixels black such that the ablation image is an image of edges and background (e.g., an edge- and background-preserving image).

[0043] Different groupings of ablation images may be used, for example, to train the first model 117 to form different trained models. The performance of these different trained models may be evaluated, for example, using model analyzer 120. Model analyzer 120 may evaluate a number of different metrics, such as precision, accuracy, reliability, coefficient of determination (r 2 ), one or more other metrics, or a combination thereof. An example of how this training and evaluation may be performed is described in more detail with respect to FIG. 4 below. Evaluating the performance of these different trained models may help to identify or localize biomarkers, gain insight into GA pathology, build confidence that the first model 117 is not focusing on spurious or irrelevant image regions or features, and / or improve the performance of the first model 117.

[0044] In one or more embodiments, the model analyzer 120 may be used to generate an output 132 based on an evaluation of the performance of different training models formed using the ablation techniques described above and / or based on the first visualization output 122 and the second visualization output 123. The output 132 may, for example, identify a new biomarker and / or a set of localized biomarkers that can be used to help narrow the focus of one or more of the models 112. The output 132 may, for example, identify information that provides insight into GA pathology. The output 132 may, for example, indicate whether a model of the models 112 can be validated. The output 132 may, for example, identify whether a model of the models 112 is focused on non-fake relevant image regions or features. The output 132 may, for example, identify one or more modifications that can be made to a model of the models 112 to improve the performance of the model.

[0045] III. Exemplary Methods for Assessing GA Lesions and Models for Predicting GA Lesion Growth Rate 2 is a flow chart of a process 200 for assessing geographic atrophy lesions according to various embodiments. In various embodiments, the process 200 is implemented using the lesion assessment system 100 described in FIG 1. In particular, the process 200 may be used to predict one or more GA progression parameters.

[0046] Step 202 includes receiving a set of retinal images. The set of retinal images may be an example of an implementation of set of retinal images 110 of FIG. 1. The set of retinal images may include a set of FAF images, a set of OCT images, or both. In one or more embodiments, the set of retinal images includes a collection of baseline FAF images for a plurality of subjects diagnosed with geographic atrophy, or possibly a prodromal stage of geographic atrophy.

[0047] Step 204 includes training each model of the plurality of models to predict a set of geographic atrophy (GA) progression parameters for the GA lesions using the set of retinal images. The plurality of models may include, for example, but not limited to, a plurality of deep learning models. The plurality of models in step 204 may be an example of an implementation of the model 112 of FIG. 1. As an example, the plurality of models may include a first deep learning model composed of one or more convolutional neural networks and a second deep learning model composed of one or more convolutional neural networks. The set of GA progression parameters may be an example of an implementation of the set of GA progression parameters 114 of FIG. 1. The set of GA progression parameters may include growth rate, baseline lesion area, or both.

[0048] In step 206 of generating a visualization output for each model of the plurality of models, the visualization output of the corresponding model of the plurality of models provides information regarding how the corresponding model uses the set of retinal images to predict a set of GA progression parameters. For example, step 206 may include generating a gradient activation map of the corresponding retinal images in the set of retinal images of the corresponding model. The gradient activation map indicates (e.g., visually identifies) a set of regions in the corresponding retinal images that contributed to the set of GA progression parameters predicted by the corresponding model of GA pathology. For example, the gradient activation map identifies one or more regions of the retinal images that were associated with (or drove) the prediction of the set of GA progression parameters. Furthermore, the gradient activation map may visually identify the degree of relevance of these one or more regions to the prediction of the set of GA progression parameters.

[0049] In one or more embodiments, the visualization output generated for the different models may be used to validate the models. For example, one of the multiple models may be a deep learning model. The visualization output generated for this deep learning model may be used to validate the deep learning model and confirm that the deep learning model focuses on the non-fake relevant parts of the relevant image to predict a set of GA progression parameters. In some cases, a comparison of the different visualization outputs generated for the different models may be performed to select the best model for predicting a set of GA progression parameters.

[0050] In one or more embodiments, one of the multiple models may be modified to form a new model based on the visualization output generated for that model, thereby improving the performance of the model. Model performance may be measured in terms of accuracy, precision, reliability, time spent generating a prediction, amount of computational resources utilized in generating a prediction, coefficient of determination, or a combination thereof.

[0051] 3 is a flow chart of a process 300 for assessing geographic atrophy lesions according to various embodiments. In various embodiments, the process 300 is implemented using the lesion assessment system 100 described in FIG 1. In particular, the process 300 may be used to predict one or more GA progression parameters.

[0052] Step 302 includes receiving a set of retinal images. The set of retinal images may be an example of an implementation of set of retinal images 110 of FIG. 1. In one or more embodiments, the set of retinal images may belong to a single subject diagnosed with geographic atrophy, or possibly a prodromal stage of geographic atrophy. For example, the set of retinal images may include a set of baseline FAF images, a set of baseline OCT images, or both, of the same retina of the subject. These baseline images may include images taken for the same or substantially the same time point or points (e.g., within the same hour, within the same day, within the same 1-3 days, etc.).

[0053] Step 304 includes predicting a set of geographic atrophy (GA) progression parameters for GA lesions in the retina using the set of retinal images and the deep learning model, the set of GA progression parameters including at least one of a growth rate of the GA lesions or a baseline lesion area of ​​the GA lesions.

[0054] Step 306 includes generating a set of gradient activation maps corresponding to the set of retinal images of the deep learning model, where the gradient activation maps in the set of gradient activation maps of corresponding retinal images of the set of retinal images identify a set of regions in the corresponding retinal images that are relevant to the prediction of the set of GA progression parameters by the deep learning model. The gradient activation maps may visually identify a degree of relevance of one or more regions in the retinal images using, for example, coloring, shading, highlighting, patterns, text and / or numeric labels, other types of indicators, or combinations thereof.

[0055] The process 300 may optionally include generating 308 an output for use in improving the performance of the deep learning model based on the set of gradient activation maps. The output may, for example, identify a new and / or localized set of biomarkers that can be used to help narrow the focus of the deep learning model, provide insight into the GA pathology, identify information that can be used to modify the deep learning model, indicate whether the deep learning can be validated, ascertain whether the deep learning model is focused on non-fake relevant image regions or features, identify one or more modifications that can be made to the deep learning model to improve the performance of the model, or any combination thereof.

[0056] 4 is a flowchart of a process 400 for improving model performance according to various embodiments. In various embodiments, the process 400 is implemented using the lesion scoring system 100 described in FIG.

[0057] Step 402 includes receiving a plurality of retinal images for a plurality of subjects. The plurality of retinal images may be an example of an implementation of the set of retinal images 110 of FIG. 1. The plurality of retinal images may include, for example, but not limited to, baseline FAF images of the subjects. The subjects may be individuals who have been diagnosed with GA or prodromal stages of GA and for whom GA progression information (e.g., growth rate of GA lesions) is known. In one or more embodiments, each retinal image of the plurality of retinal images images a GA lesion in the retina of the corresponding subject.

[0058] Step 404 includes modifying the plurality of retinal images to form a plurality of ablation image groups, each ablation image group of the plurality of ablation image groups including a plurality of ablation images corresponding to a plurality of retinal images from which at least one of the GA lesion, the edge of the GA lesion, or the background (a portion of the image not identified as a GA lesion or edge) has been ablated. The edge may be defined, for example, as a 500 μm wide border surrounding the GA lesion. In other examples, the edge may be defined as a surrounding border having a width selected between 250 μm and 750 μm.

[0059] In one or more embodiments, cutting out the selected portion or region of the image is performed by filling in pixels corresponding to the cut out portion or region with black. For example, step 404 may include segmenting the multiple retinal images using a segmentation algorithm to identify portions of the images that represent GA lesions, portions of the images that represent edges of the GA lesions, and the background of each retinal image (portions of the images that are not identified as GA lesions or edges). Cutting out a portion of the image may include assigning pixels identified as representing the portion to a black value (e.g., a pixel value of 0). For example, the edges of the GA lesions may be cut out by assigning pixels segmented as representing GA lesions to a pixel value of 0.

[0060] The plurality of ablated image sets formed in step 404 may be formed by ablation of one or more different portions of the retinal image. For example, the ablated image set may be a lesion-preserving image set consisting of a plurality of lesion-preserving images in which the edges of the GA lesions and the background have been ablated such that only the GA lesions of the original retinal image are retained. The ablated image set may be a lesion-edge image set consisting of a plurality of edge-preserving images in which the GA lesions and the background have been ablated such that only the edges of the GA lesions of the original retinal image are retained. The ablated image set may be a background-preserving image set consisting of a plurality of retained backgrounds in which the GA lesions and the edges of the GA lesions are ablated such that only the background of the original retinal image is retained.

[0061] Another ablation image group may be a lesion ablation image group consisting of a plurality of lesion ablation images in which the GA lesions have been ablated such that the edges of the GA lesions and the background of the original retinal image are preserved. Yet another ablation image group may be an edge ablation image group consisting of a plurality of edge ablation images in which the edges of the GA lesions have been ablated such that the GA lesions and the background of the original retinal image are preserved. Yet another ablation image group may be a background ablation image group consisting of a plurality of background ablation images in which the background has been ablated such that the GA lesions and the edges of the GA lesions of the original retinal image are preserved. In this manner, different portions or combinations of portions of the retinal image may be ablated to form the ablation images.

[0062] In some embodiments, step 404 further includes shuffling pixel values ​​within whatever portion of the original retinal image is retained to form the ablation image. This shuffling may be a randomly performed rearrangement of pixel values ​​among the pixels included in the retained portion of the retinal image. For example, the ablation image may be a lesion-shuffled image in which a portion of the retinal image identified as a GA lesion is retained and the pixel values ​​of the pixels within this portion are shuffled. This shuffling preserves intensity information associated with this portion of the retinal image but removes texture information (e.g., which areas of this portion are brighter than other areas). The ablation image may be an edge-shuffled image in which a portion of the retinal image identified as an edge of a GA lesion is retained with the pixel values ​​of this portion shuffled. The ablation image may be a background-shuffled image in which a portion of the retinal image identified as a background is retained with the pixel values ​​of this portion shuffled. Thus, in some embodiments, the multiple resection image groups may include a lesion shuffled image group, an edge shuffled image group, a background shuffled image group, a lesion and edge shuffled image group, a lesion and background shuffled image group, an edge and background shuffled image group, or combinations thereof.

[0063] Step 406 includes training an initial model to predict the growth rate of the GA lesion using each of the plurality of resection image groups to form a plurality of trained models. The initial model may be, for example, a deep learning model and may include one or more neural networks. A trained model may be formed for each of the plurality of resection image groups. For example, the initial model may be trained and tested using a first resection image group of the plurality of resection image groups to form a first trained model. As another example, the initial model may be trained and tested using a second resection image group of the plurality of resection image groups to form a second trained model.

[0064] Step 408 includes evaluating the performance of the multiple trained models, for example but not limited to, accuracy, precision, coefficient of determination (r 2), the time spent by a model trained to analyze a single excision image to predict growth rate compared to the time spent by the initial model to analyze the corresponding original retinal image, the amount of computational resources spent by a model trained to analyze a single excision image to predict growth rate compared to the amount of computational resources spent by the initial model to analyze the corresponding original retinal image, one or more other types of metrics, or a combination thereof.

[0065] Step 410 includes generating an output for use in improving the performance of the initial model based on the performance of the multiple trained models. Step 410 may be performed in a variety of ways. In one or more embodiments, the output may be an identification of a set of ablation images corresponding to the trained model with the best performance. For example, a trained model corresponding to a set of edge-preserving images may be identified as having the best performance. In this example, the output may identify the edge of the GA lesion as being most relevant to predicting growth rate. The output may further identify one or more biomarkers associated with this edge region, indicating that focusing on these one or more biomarkers may improve model performance in terms of speed and computing resources utilized. In this manner, identifying the edge of the GA lesion as being most relevant to predicting growth rate may aid in localizing the biomarker of interest.

[0066] Process 400 may optionally include step 412, which includes adjusting the initial model based on the output to form a new model. In one or more embodiments, adjusting the initial model includes narrowing the biomarkers analyzed by the initial model to biomarkers associated with the region of the retina identified as most relevant to predicting growth rate (e.g., the GA lesion, the edge of the GA lesion, or the background). In some embodiments, adjusting the initial model includes integrating a supplemental model (which itself may include one or more algorithms or models) as part of the initial model to form a new model, or combining the supplemental model with the initial model to form a new model. The supplemental model may be used, for example, to segment the input retinal image and form an ablation image based on this segmentation. The new model predicts growth rate using the ablation image, which may be faster and / or consume less computing resources than predicting growth rate using a non-ablated retinal image.

[0067] IV. Exemplary Visualization Output and Ablation Images A. Example visualization output generated for two deep learning models An experiment was conducted in which two different deep learning models were trained and tested using retinal images for multiple subjects. These retinal images were baseline FAF images, each of which imaged GA lesions, which could be unifocal or multifocal GA lesions. Both the first and second deep learning models were used to predict the lesion growth rate of the GA lesions. Visualization outputs were generated for these two deep learning models. Specifically, gradient activation maps were generated for the two deep learning models to provide information about which parts of each retinal image were ultimately associated with the growth rate predicted by the corresponding deep learning model.

[0068] 5 is a chart comparing gradient activation maps generated for two deep learning models, according to one or more embodiments. In FIG. 5, a first group 502 of gradient activation maps generated for a first deep learning model (first DL model) is an example of at least a portion of the implementation of the set of gradient activation maps 124 of FIG. 1. A second group 504 of gradient activation maps generated for a second deep learning model (second DL model) is an example of at least a portion of the implementation of the set of gradient activation maps 126 of FIG. 1.

[0069] The five gradient activation maps in the first group of gradient activation maps 502 and the five gradient activation maps in the second group of gradient activation maps 504 were generated for the same group of five retinal images, each of which imaged a monofocal GA lesion. Comparing the first group of gradient activation maps 502 and the second group of gradient activation maps 504, it becomes apparent that for the retinal images of the monofocal GA lesion, different portions of these retinal images were associated with the first deep learning model compared to the second deep learning model.

[0070] 6 is a chart comparing gradient activation maps generated for two deep learning models, according to one or more embodiments. In FIG. 6, a first group 602 of gradient activation maps generated for a first deep learning model (first DL model) is an example of at least a portion of the implementation of the set of gradient activation maps 124 of FIG. 1. A second group 604 of gradient activation maps generated for a second deep learning model (second DL model) is an example of at least a portion of the implementation of the set of gradient activation maps 126 of FIG. 1.

[0071] The five gradient activation maps in the first group of gradient activation maps 602 and the five gradient activation maps in the second group of gradient activation maps 604 were generated for the same group of five retinal images, each of which imaged a multifocal GA lesion. Comparing the first group of gradient activation maps 602 and the second group of gradient activation maps 604, it becomes apparent that for the retinal images of the multifocal GA lesion, similar portions of these retinal images were associated with both the first deep learning model compared to the second deep learning model.

[0072] B. Exemplary ablation images for ablation analysis 7 is a chart illustrating exemplary ablation images according to one or more embodiments. Each of the ablation images 700 may be an example of an ablation image implementation of the ablation images 130 of FIG. 1. The ablation images 700 include a lesion ablation image 702, an edge ablation image 704, a background ablation image 706, a lesion-preserving image 708, an edge-preserving image 710, a background-preserving image 712, a lesion-shuffled image 714, an edge-shuffled image 716, and a background-shuffled image 718.

[0073] V. Computer Implemented Systems FIG. 8 is a block diagram of a computer system according to various embodiments. Computer system 800 may be an example of one implementation of computing platform 102 described above in FIG. 1. In one or more examples, computer system 800 may include a bus 802 or other communication mechanism for communicating information and a processor 804 coupled to bus 802 for processing information. In various embodiments, computer system 800 may also include memory, which may be a random access memory (RAM) 806 or other dynamic storage device, coupled to bus 802 for determining instructions to be executed by processor 804. The memory may also be used to store temporary variables or other intermediate information during execution of instructions to be executed by processor 804. In various embodiments, computer system 800 may further include a read-only memory (ROM) 808 or other static storage device coupled to bus 802 for storing static information and instructions for processor 804. A storage device 810, such as a magnetic disk or optical disk, may be provided and coupled to bus 802 for storing information and instructions.

[0074] In various embodiments, the computer system 800 may be coupled via the bus 802 to a display 812, such as a cathode ray tube (CRT) or liquid crystal display (LCD), for displaying information to a computer user. An input device 814, including alphanumeric and other keys, may be coupled to the bus 802 for communicating information and command selections to the processor 804. Another type of user input device is a cursor control device 816, such as a mouse, joystick, trackball, gesture input device, gaze-based input device, or cursor direction keys, for communicating directional information and command selections to the processor 804 and controlling cursor movement on the display 812. This input device 814 typically has two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allow the device to specify a position in a plane. However, it should be understood that input devices 814 that allow three-dimensional (e.g., x, y, and z) cursor movement are also contemplated herein.

[0075] According to a particular implementation of the present teachings, results may be provided by computer system 800 in response to execution by processor 804 of one or more sequences of one or more instructions contained in RAM 806. Such instructions may be read into RAM 806 from another computer-readable medium or computer-readable storage medium, such as storage device 810. Execution of the sequences of instructions contained in RAM 806 may cause processor 804 to perform the processes described herein. Alternatively, hard-wired circuitry may be used in place of or in combination with software instructions to implement the present teachings. Thus, implementation of the present teachings is not limited to any specific combination of hardware circuitry and software.

[0076] The terms "computer-readable medium" (e.g., data store, storage device, data storage device, etc.) or "computer-readable storage medium" as used herein refer to any medium that participates in providing instructions to the processor 804 for execution. Such media may take many forms, including but not limited to, non-volatile media, volatile media, and transmission media. Examples of non-volatile media may include, but are not limited to, optical, solid-state, and magnetic disks, such as storage device 810. Examples of volatile media may include, but are not limited to, dynamic memory, such as RAM 806. Examples of transmission media may include, but are not limited to, coaxial cables, copper wire, and fiber optics, including the wires that comprise bus 802.

[0077] Common forms of computer readable media include, for example, a floppy disk, a flexible disk, a hard disk, a magnetic tape or any other magnetic medium, a CD-ROM, any other optical medium, punch cards, paper tape, any other physical medium with a pattern of holes, RAM, PROM, and EPROM, flash EPROM, any other memory chip or cartridge, or any other tangible medium from which a computer can read.

[0078] In addition to computer-readable media, instructions or data may be provided as signals on a transmission medium included in a communication device or system to provide a sequence of one or more instructions to the processor 804 of the computer system 800 for execution. For example, a communication device may include a transceiver having signals indicative of instructions and data. The instructions and data are configured to cause one or more processors to implement the functions outlined in the disclosure herein. Representative examples of data communication transmission connections include, but are not limited to, a telephone modem connection, a wide area network (WAN), a local area network (LAN), an infrared data connection, an NFC connection, an optical communication connection, and the like.

[0079] It should be understood that the methodologies, flowcharts, diagrams, and accompanying disclosure described herein may be implemented using computer system 800 as a stand-alone device or on a distributed network of shared computer processing resources, such as a cloud computing network.

[0080] The methodologies described herein may be implemented by various means depending on the application. For example, these methodologies may be implemented in hardware, firmware, software, or any combination thereof. In the case of hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processors (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described herein, or combinations thereof.

[0081] In various embodiments, the methods of the present teachings may be implemented as firmware and / or software programs and applications written in conventional programming languages ​​such as C, C++, Python, etc. When implemented as firmware and / or software, the embodiments described herein may be implemented on a non-transitory computer-readable medium having stored thereon a program for causing a computer to execute the above-mentioned methods. It should be understood that the various engines described herein may be provided on a computer system such as computer system 800, whereby processor 804 performs the analysis and decisions provided by these engines according to instructions provided by any one or combination of memory components RAM 806, ROM 808, or storage device 810, and user input provided via input device 814.

[0082] VI. Illustrative Context and Definitions The present disclosure is not limited to these example embodiments and applications or to the manner in which the example embodiments and applications operate or are described herein. Further, the figures may show simplified or partial views, and dimensions of elements in the figures may be exaggerated or out of proportion.

[0083] Additionally, when the terms "on," "attached to," "connected to," "coupled to," or similar terms are used herein, an element (e.g., a component, material, layer, substrate, etc.) may be "on," "attached to," "connected to," or "coupled to" the other element, regardless of whether the element is directly on, directly attached to, connected to, or coupled to the other element, or whether there are one or more intervening elements between the one element and the other element. Additionally, when a list of elements (e.g., elements a, b, c) is referenced, such reference is intended to include any one of the listed elements by itself, any combination of less than all of the listed elements, and / or all combinations of the listed elements. The division of sections herein is merely for ease of review and is not intended to limit any combination of the elements described.

[0084] The term "subject" may refer to a subject in a clinical trial, a person undergoing treatment, a person undergoing anti-cancer therapy, a person being monitored for remission or recovery, a person undergoing preventive health analysis (e.g., due to their medical history), or any other person or patient of interest. In various instances, "subject" and "patient" may be used interchangeably herein.

[0085] Unless otherwise defined, scientific and technical terms used in connection with the present teachings described herein shall have the meanings commonly understood by those of ordinary skill in the art. Further, unless otherwise required by context, singular terms shall include the plural and plural terms shall include the singular. In general, the nomenclature and techniques utilized in connection with chemistry, biochemistry, molecular biology, pharmacology and toxicology are described herein and are well known and commonly used in the art.

[0086] As used herein, "substantially" may mean sufficient to function for the intended purpose. Thus, the term "substantially" allows for minor, slight variations from an absolute or perfect state, dimension, measurement, result, etc., that would be expected by one of ordinary skill in the art, but that do not significantly affect overall performance. When used in reference to a numerical value, or a parameter or characteristic that can be expressed as a numerical value, "substantially" means within 10 percent.

[0087] The term "plurality" means two or more.

[0088] As used herein, the term "plurality" can be 2, 3, 4, 5, 6, 7, 8, 9, 10 or more.

[0089] As used herein, the term "set" means one or more. For example, a set of items includes one or more items.

[0090] As used herein, the phrase "at least one of," when used with a list of items, may mean that different combinations of one or more of the listed items may be used, and only one of the items in the list may be required. The items may be specific objects, things, steps, operations, processes, or categories. In other words, "at least one of" means that any combination or number of items from the list may be used, but not all of the items in the list are required. For example, but not limited to, "at least one of item A, item B, or item C" means item A, item A and item B, item B, item A, item B, and item C, item B and item C, or item A and C. In some cases, "at least one of item A, item B, or item C" may mean, but not limited to, two of item A, one of item B, and ten of item C, four of item B and seven of item C, or some other suitable combination.

[0091] As used herein, a "model" may include one or more algorithms, one or more mathematical techniques, one or more machine learning algorithms, or a combination thereof.

[0092] As used herein, "machine learning" can be the practice of using algorithms to analyze data, learn from it, and then make decisions or predictions about something in the world. Machine learning uses algorithms that can learn from data without relying on rule-based programming.

[0093] As used herein, "artificial neural network" or "neural network" (NN) may refer to a mathematical algorithm or computational model that mimics an interconnected group of artificial neurons that processes information based on a connectionist approach to computation. A neural network, sometimes referred to as a neural net, can predict an output for a received input using one or more layers of nonlinear units. Some neural networks include one or more hidden layers in addition to an output layer. The output of each hidden layer is used as an input to the next layer in the network, i.e., the next hidden layer or the output layer. Each layer of the network generates an output from the received input according to the current value of each set of parameters. In various embodiments, a reference to a "neural network" may be a reference to one or more neural networks.

[0094] A neural network may process information in two ways. When it is being trained, it is in training mode, and when it actually executes what it has learned, it is in inference (or prediction) mode. A neural network learns through a feedback process (e.g., backpropagation) that allows the network to adjust (change its behavior) the weight coefficients of individual nodes in the intermediate hidden layers so that the output matches that of the training data. In other words, a neural network learns by being fed training data (training examples), and eventually learns how to arrive at the correct output even when presented with a new range or set of inputs. A neural network may include, for example, but is not limited to, at least one of a feedforward neural network (FNN), a recurrent neural network (RNN), a modular neural network (MNN), a convolutional neural network (CNN), a residual neural network (ResNet), an ordinary differential equation neural network (Neural-ODE), or another type of neural network.

[0095] As used herein, a "lesion" can be an area of ​​an organ or tissue that is damaged through injury or disease. This area can be a continuous or discontinuous area. For example, as used herein, a lesion can include multiple areas. A geographic atrophy (GA) lesion can be an area of ​​the retina that suffers from chronic progressive degeneration. As used herein, a GA lesion can include one lesion (e.g., one continuous lesion area) or multiple lesions (e.g., discontinuous lesion area that is composed of multiple separate lesions).

[0096] As used herein, "lesion area" may refer to the total area covered by the lesion, regardless of whether the lesion is a continuous or discontinuous area.

[0097] As used herein, "long-term" may mean over a period of time. The period may be days, weeks, months, years, or some other time scale.

[0098] As used herein, the "growth rate" of GA lesions may refer to the long-term change in the lesion area of ​​GA lesions. This growth rate may also be referred to as GA growth rate.

[0099] VII. Description of the Preferred Embodiments Embodiment 1. A method for assessing geographic atrophy, comprising: receiving a set of retinal images; training each model of a plurality of models to predict a set of geographic atrophy (GA) progression parameters for a GA lesion using the set of retinal images; and generating a visualization output for each model of the plurality of models, wherein the visualization output for a corresponding model of the plurality of models provides information regarding how the corresponding model uses the set of retinal images to predict the set of GA progression parameters.

[0100] Embodiment 2. The method of embodiment 1, wherein generating includes generating a gradient activation map of a corresponding retinal image in the set of retinal images of the corresponding model, the gradient activation map indicating a set of regions in the corresponding retinal image that contributed to the set of GA progression parameters predicted by the corresponding model of GA pathology.

[0101] Embodiment 3. The method of embodiment 1 or 2, wherein the plurality of models includes a deep learning model, and further comprising validating the deep learning model using visualization output generated for the deep learning model.

[0102] Embodiment 4. The method of any one of embodiments 1 to 3, wherein the plurality of models includes a first deep learning model and a second deep learning model, and further includes performing a comparison between the visualization output generated for the first deep learning model and the visualization output generated for the second deep learning model.

[0103] Embodiment 5. The method of embodiment 4, further comprising selecting either the first deep learning model or the second deep learning model as the best model for predicting the set of GA progression parameters based on the comparison.

[0104] Embodiment 6. The method of any one of embodiments 1 to 5, further comprising modifying a model of the plurality of models to form a new model based on visualization output generated for the model in order to improve performance of the model.

[0105] Embodiment 7. The method of any one of embodiments 1 to 6, wherein the set of GA progression parameters includes at least one of the following: growth rate of GA lesions or baseline lesion area of ​​GA lesions.

[0106] Embodiment 8. The method of any one of embodiments 1 to 7, wherein the set of retinal images includes at least one of a set of fundus autofluorescence (FAF) images or a set of optical coherence tomography (OCT) images.

[0107] Embodiment 9. The method of embodiment 8, wherein the set of fundus autofluorescence (FAF) images is a set of baseline FAF images and the set of optical coherence tomography (OCT) images is a set of baseline OCT images.

[0108] Embodiment 10. A method for assessing geographic atrophy in the retina, comprising: receiving a set of retinal images; predicting a set of geographic atrophy (GA) progression parameters for a GA lesion in the retina using the set of retinal images and a deep learning model; and generating a set of gradient activation maps corresponding to the set of retinal images of the deep learning model, wherein gradient activation maps in the set of gradient activation maps of corresponding retinal images of the set of retinal images identify a set of regions in the corresponding retinal images associated with the prediction of the set of GA progression parameters by the deep learning model.

[0109] Embodiment 11. The method of embodiment 10, further comprising generating an output for use in improving performance of a deep learning model based on the set of gradient activation maps.

[0110] Embodiment 12. The method of embodiment 10 or 11, wherein the set of GA progression parameters includes at least one of the growth rate of GA lesions or the baseline lesion area of ​​GA lesions.

[0111] Embodiment 13. The method of any one of embodiments 10 to 12, wherein the set of retinal images includes at least one of a set of fundus autofluorescence (FAF) images or a set of optical coherence tomography (OCT) images.

[0112] Embodiment 14. A system for assessing geographic atrophy comprising: a memory including a machine-readable medium including machine-executable code; and a processor coupled to the memory configured to execute the machine-executable code to cause the processor to receive a set of retinal images; train each model of a plurality of models to predict a set of geographic atrophy (GA) progression parameters for a GA lesion using the set of retinal images; and generate a visualization output for each model of the plurality of models, wherein the visualization output for a corresponding model of the plurality of models provides information regarding how the corresponding model uses the set of retinal images to predict the set of GA progression parameters.

[0113] Embodiment 15. The system of embodiment 14, wherein the visualization output includes a gradient activation map of a corresponding retinal image in the set of retinal images of the corresponding model, the gradient activation map indicating a set of regions in the corresponding retinal image that contributed to the set of GA progression parameters predicted by the corresponding model of GA pathology.

[0114] Embodiment 16. The system of embodiment 14 or 15, wherein the corresponding model is a corresponding deep learning model, and the processor is configured to execute machine-executable code to cause the processor to validate the corresponding deep learning model using the visualization output generated for the corresponding deep learning model.

[0115] Embodiment 17. The system of any one of embodiments 14 to 16, wherein the plurality of models includes a first deep learning model and a second deep learning model, and the processor is configured to execute machine executable code to cause the processor to perform a comparison between a visualization output generated for the first deep learning model and a visualization output generated for the second deep learning model, and to select either the first deep learning model or the second deep learning model as the best model for predicting a set of GA progress parameters based on the comparison.

[0116] Embodiment 18. A system as described in any one of embodiments 14 to 17, wherein the processor is configured to execute machine executable code to cause the processor to modify a model of the plurality of models to form a new model based on visualization output generated for the model to improve performance of the model.

[0117] Embodiment 19. The system of any one of embodiments 14 to 18, wherein the set of GA progression parameters includes at least one of the growth rate of GA lesions or the baseline lesion area of ​​GA lesions.

[0118] Embodiment 20. A system described in any one of embodiments 14 to 19, wherein the set of retinal images includes at least one of a set of fundus autofluorescence (FAF) images or a set of optical coherence tomography (OCT) images.

[0119] VIII. Further Considerations Headings and subheadings among the sections and subsections of this document are included merely to improve readability and do not imply that features cannot be combined across the sections and subsections, and thus the sections and subsections do not describe separate embodiments.

[0120] Some embodiments of the present disclosure include a system including one or more data processors. In some embodiments, the system includes a non-transitory computer-readable storage medium including instructions that, when executed on the one or more data processors, cause the one or more data processors to perform some or all of the methods and / or some or all of the processes disclosed herein. Some embodiments of the present disclosure include a computer program product tangibly embodied in a non-transitory machine-readable storage medium including instructions configured to cause the one or more data processors to perform some or all of the methods and / or some or all of the processes disclosed herein.

[0121] The terms and expressions used are used as terms of description and not of limitation, and in the use of such terms and expressions there is no intention to exclude equivalents of the features shown and described or portions thereof, but it is recognized that various modifications are possible within the scope of the invention as claimed. Thus, although the invention as claimed has been specifically disclosed by embodiments and optional features, it will be understood that modifications and variations of the concepts disclosed herein may be resorted to by those skilled in the art, and such modifications and variations are considered to be within the scope of the invention as defined by the appended claims.

[0122] The description provided herein provides only preferred exemplary embodiments and is not intended to limit the scope, applicability or configuration of the present disclosure. Rather, the description of the preferred exemplary embodiments provides those skilled in the art with an enabling description for implementing various embodiments. It is understood that various changes may be made in the function and arrangement of elements (e.g., elements of block diagrams or schematic diagrams, elements of flow diagrams, etc.) without departing from the spirit and scope of the appended claims.

[0123] In the above description, specific details are given to provide a thorough understanding of the embodiments. However, it will be understood that the embodiments may be practiced without these specific details. For example, circuits, systems, networks, processes, and other components may be shown as components in block diagram form so as not to obscure the embodiments in unnecessary detail. In other examples, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail so as to avoid obscuring the embodiments.

[0124] While the present teachings have been described in conjunction with various embodiments, it is not intended that the present teachings be limited to such embodiments. On the contrary, the present teachings encompass various alternatives, modifications, and equivalents, as will be appreciated by those skilled in the art.

[0125] In describing various embodiments, the specification may present a method and / or process as a particular sequence of steps. However, to the extent that the method or process does not rely on the particular order of steps described herein, the method or process should not be limited to the particular order of steps described, and as one of ordinary skill in the art can readily appreciate, the order may be varied and still remain within the spirit and scope of the various embodiments.

Claims

1. A method for evaluating geographic atrophy, comprising: receiving a set of retinal images; training each model of a plurality of models to predict a set of geographic atrophy (GA) progression parameters for geographic atrophy (GA) lesions using the set of retinal images; generating a visualization output for each model of the plurality of models, wherein the visualization output of the corresponding model of the plurality of models provides information on how the corresponding model uses the set of retinal images to predict the set of GA progression parameters.

2. The generating comprises: generating a gradient activation map of a corresponding retinal image within the set of retinal images of the corresponding model, wherein the gradient activation map indicates a set of regions within the corresponding retinal image that contributed to the set of GA progression parameters predicted by the corresponding model for the GA lesion. The method according to claim 1.

3. The plurality of models includes deep learning models, and further comprising: validating the deep learning model using the visualization output generated for the deep learning model. The method according to claim 1 or 2.

4. The plurality of models includes a first deep learning model and a second deep learning model, and further comprising: performing a comparison between the visualization output generated for the first deep learning model and the visualization output generated for the second deep learning model. The method according to claim 1 or 2.

5. Further comprising: selecting either the first deep learning model or the second deep learning model as the best model for predicting the set of GA progression parameters based on the comparison. The method according to claim 4.

6. Further comprising: modifying a model among the plurality of models to form a new model based on the visualization output generated for the model to improve the performance of the model. The method according to claim 1 or 2.

7. The set of GA progression parameters includes at least one of a growth rate of the GA lesion or a baseline lesion area of the GA lesion. The method according to claim 1 or 2.

8. The method according to claim 1 or 2, wherein the set of retinal images includes at least one of a set of fundus autofluorescence (FAF) images or a set of optical coherence tomography (OCT) images.

9. The method according to claim 8, wherein the set of fundus autofluorescence (FAF) images is a set of baseline FAF images, and the set of optical coherence tomography (OCT) images is a set of baseline OCT images.

10. A method for evaluating geographic atrophy in a retina, comprising: receiving a set of retinal images; using the set of retinal images and a deep learning model to predict a set of geographic atrophy (GA) progression parameters for geographic atrophy (GA) lesions in the retina; generating a set of gradient activation maps corresponding to the set of retinal images of the deep learning model, wherein the gradient activation maps in the set of gradient activation maps corresponding to the corresponding retinal images of the set of retinal images identify a set of regions in the corresponding retinal images related to the prediction of the set of GA progression parameters by the deep learning model, generating a set of gradient activation maps.

11. Furthermore, The method according to claim 10, comprising generating an output for use in improving the performance of the deep learning model based on the set of gradient activation maps.

12. The method according to claim 10 or 11, wherein the set of GA progression parameters includes at least one of the growth rate of the GA lesion or the baseline lesion area of the GA lesion.

13. The method according to claim 10 or 11, wherein the set of retinal images includes at least one of a set of fundus autofluorescence (FAF) images or a set of optical coherence tomography (OCT) images.

14. A system for evaluating geographic atrophy, comprising: a memory including a machine-readable medium containing machine-executable code; a processor coupled to the memory, the processor executing the machine-executable code to cause the processor to: receive a set of retinal images; train each model of a plurality of models to predict a set of geographic atrophy (GA) progression parameters for geographic atrophy (GA) lesions using the set of retinal images; Generating a visualization output for each model of the plurality of models, wherein the visualization output of the corresponding model of the plurality of models provides information regarding how the corresponding model uses the set of retinal images to predict the set of GA progression parameters, and generating a visualization output; and a processor configured to cause the above to be performed, a system.

15. The system according to claim 14, wherein the visualization output includes a gradient activation map of a corresponding retinal image in the set of retinal images of the corresponding model, and the gradient activation map shows a set of regions in the corresponding retinal image that contributed to the set of GA progression parameters predicted by the corresponding model of the GA lesion.

16. The system according to claim 14 or 15, wherein the corresponding model is a corresponding deep learning model, and the processor is configured to execute the machine-executable code to cause the processor to verify the corresponding deep learning model using the visualization output generated for the corresponding deep learning model.

17. The plurality of models includes a first deep learning model and a second deep learning model, and the processor is configured to execute the machine-executable code to cause the processor to perform a comparison between the visualization output generated for the first deep learning model and the visualization output generated for the second deep learning model; select either the first deep learning model or the second deep learning model as the best model for predicting the set of GA progression parameters based on the comparison; The system according to claim 14 or 15, which is configured to perform the above.

18. The system according to claim 14 or 15, wherein the processor is configured to execute the machine-executable code to cause the processor to change a model among the plurality of models to form a new model based on the visualization output generated for the model to improve the performance of the model.

19. The system according to claim 14 or 15, wherein the set of GA progression parameters includes at least one of a growth rate of the GA lesion or a baseline lesion area of the GA lesion.

20. The system according to claim 14 or 15, wherein the set of retinal images includes at least one of a set of fundus autofluorescence (FAF) images or a set of optical coherence tomography (OCT) images.

Images