Neural network processing of oct data to generate predictions of geographic-atrophy growth rates

Abstract

To provide neural network processing of OCT data to generate predictions of geographic-atrophy growth rates.SOLUTION: Embodiments disclosed herein generally relate to predicting geographic-atrophy lesion growth and / or geographic atrophy lesion size in an eye. The predictions can be generated by processing a data object using a neural network. The data object may include a three-dimensional data object representing a depiction of at least part of the eye or a multi-channel data object representing one or more depictions of at least part of the eye. The neural network can include a convolutional multi-task neural network that is trained to learn features that are predictive of both lesion-growth and lesion-size outputs.SELECTED DRAWING: None

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of and priority to U.S. Provisional Patent Application Nos. 62 / 944,201, filed December 5, 2019, and 62 / 988,797, filed March 12, 2020, each of which is incorporated herein by reference in its entirety for all purposes. [Background technology]

[0002] background Geographic atrophy (GA) is an advanced form of age-related macular degeneration (AMD) that results in the degeneration of photoreceptors and supporting cells and progressive vision loss. This condition affects millions of people worldwide. In developed countries, approximately 1 in 29 people aged 75 years or older has geographic atrophy. It is characterized by the progressive structural loss of the retinal pigment epithelium (RPE), adjacent photoreceptors, and choriocapillaris. The progression of geographic atrophy shows significant interpatient variability. Currently, there are no approved treatments to prevent or slow the progression of geographic atrophy.

[0003] Geographic atrophy lesions can be imaged by various imaging modalities. Traditionally, two-dimensional fundus autofluorescence (FAF) images have been used to quantify geographic atrophy lesion area. The change in FAF-derived lesion area over a period of time (geographic atrophy growth velocity) is accepted as an anatomical outcome parameter indicating whether and / or to what extent a subject's geographic atrophy is progressing. Nevertheless, the two-dimensional nature of FAF images may limit their ability to provide precise structural information regarding lesion area. Therefore, in addition to the quantification of GA lesion area generated by FAF images, improved assessment of lesion area is needed that can enhance our understanding of GA development and progression. Summary of the Invention [Means for solving the problem]

[0004] overview In some embodiments, a method is provided. A three-dimensional data object corresponding to at least a partial representation of a subject's eye is accessed. The three-dimensional data object can include a three-dimensional representation of a volume of the subject's eye. The three-dimensional data object is processed using a convolutional neural network (e.g., an Inception neural network) to generate a prediction of subsequent growth of a geographic atrophy lesion in the eye or subsequent size of a geographic atrophy lesion in the eye. The subsequent growth of a geographic atrophy lesion can include growth of one or more geographic atrophy lesions in the eye, and / or the subsequent size of a geographic atrophy lesion can include subsequent size of one or more geographic atrophy lesions in the eye. The convolutional neural network can include one or more three-dimensional convolution modules, a pooling layer, and / or one or more optional units (e.g., an attention unit). A prediction is output.

[0005] In some embodiments, a convolutional neural network can be trained to simultaneously predict current geographic atrophy lesion size and subsequent geographic atrophy state, which can include subsequent geographic atrophy lesion growth or subsequent geographic atrophy lesion size.

[0006] In some embodiments, processing the three-dimensional data object using a convolutional neural network may further yield another prediction of the current size of the geographic atrophy lesion in the eye.

[0007] In some embodiments, the method can include generating a three-dimensional data object. For example, each image in the set of two-dimensional images can be segmented to identify a segment bounded by a predicted location of Bruch's membrane and / or internal limiting membrane. Generating the three-dimensional data object can also include accessing a set of two-dimensional B-scan OCT images of the subject's eye. A set of feature maps can be generated from the set of two-dimensional B-scan OCT images. The three-dimensional data object can be generated to include the set of feature maps.

[0008] In some embodiments, generating the three-dimensional data object includes accessing a set of two-dimensional B-scan OCT images of the subject's eye. For each of the set of two-dimensional B-scan OCT images, a set of pixels that depict a particular structure of the eye (e.g., Bruch's membrane or internal limiting membrane) can be identified. Each of the set of B-scan OCT images can be flattened based on the set of pixels. The three-dimensional data object can be generated to include at least a portion of each flattened B-scan OCT image.

[0009] In some embodiments, the method can include training another convolutional neural network using the training data set to generate a first set of learned parameter values. Transfer learning can be used to train the convolutional neural network by setting the set of parameter values ​​of the convolutional neural network to the first set of learned parameter values ​​and further training the convolutional neural network using another training data set to generate a second set of learned parameter values. The convolutional neural network can be configured with the second set of learned parameter values ​​when processing the three-dimensional data object.

[0010] In some embodiments, the method can further include determining aspects of a clinical trial based on the prediction. For example, the input data can be entered by a user including or identifying a three-dimensional data object. A prediction can be received, and based on the prediction, it can be determined that the subject is eligible to participate in a particular clinical trial. As another example, after receiving the prediction, stratification can be determined for a particular clinical trial in which the subject is or will be involved. For example, stratification can be assigning individual subjects to various treatment and / or control groups so that the groups have similar default predicted geographic atrophy scores if the treatment is not administered, and / or to analyze or normalize results before making comparisons between various groups. A clinical trial can be generated based on the stratification. Clinical trial results can be output. In another example, after receiving the prediction, adjustments can be determined for a particular clinical trial in which the subject is or will be involved. The adjustments can include changes to treatment, such as changing medication, changing dosage, and / or changing the time interval between treatments. Conducting an adjusted clinical trial can be facilitated.

[0011] In some embodiments, a method is provided. The method includes detecting, at a user device, input data including or identifying a three-dimensional data object corresponding to at least a partial representation of a subject's eye. A request communication corresponding to a request to generate a predicted subsequent geographic atrophy characteristic of the subject's eye is transmitted to a remote computing system. The request communication includes the input data. In response to receiving the request communication, the remote computing system processes the three-dimensional data object using a convolutional neural network to generate a prediction of subsequent growth of a geographic atrophy lesion in the eye or a subsequent size of a geographic atrophy lesion in the eye. The prediction is transmitted from the remote computing system to a user device that receives the prediction.

[0012] In some embodiments, the method may further include collecting a set of images of the subject's eye. The three-dimensional data-object may be generated using the set of images.

[0013] Some embodiments of the present disclosure include use of a geographic atrophy prediction in treating a subject. The geographic atrophy prediction is provided by a computing device that implements a computational model based on subject data to provide the geographic atrophy prediction. The computational model includes a convolutional neural network configured to process a three-dimensional data object corresponding to at least a partial representation of the subject's eye.

[0014] In some embodiments, a method is provided. The method includes accessing a data object including at least three data channels. Each of the at least three data channels includes a two-dimensional image corresponding to at least a partial depiction of a subject's eye. The method further includes processing the data object using a convolutional neural network to generate a prediction of subsequent growth of a geographic atrophy lesion in the eye or a subsequent size of a geographic atrophy lesion in the eye, and outputting the prediction. The convolutional neural network can include one or more two-dimensional convolution modules, a pooling layer, and / or one or more optional units. The subsequent growth of a geographic atrophy lesion can include growth of one or more geographic atrophy lesions in the eye, and / or the subsequent size of a geographic atrophy lesion can include a subsequent size of one or more geographic atrophy lesions in the eye.

[0015] In some embodiments, the data object can include multiple different en face OCT-based maps of the eye. For example, the data object can include at least two en face OCT-based scans of the eye and at least one B-scan of the eye. As another example, the data object can include at least one en face OCT-based scan of the eye and at least one image obtained using a different type of imaging modality than OCT (e.g., FAF).

[0016] In some embodiments, a convolutional neural network can be trained to simultaneously predict current geographic atrophy lesion size and subsequent geographic atrophy state, which can include subsequent geographic atrophy lesion growth or subsequent geographic atrophy lesion size.

[0017] In some embodiments, processing the data object using a convolutional neural network can further generate another prediction of the current size of the geographic atrophy lesion in the eye.

[0018] In some embodiments, the method can further include determining aspects of a clinical trial based on the prediction. For example, the input data can be entered by a user, including or identifying a three-dimensional data object. A prediction can be received, and based on the prediction, it can be determined that the subject is eligible to participate in a particular clinical trial. As another example, after receiving the prediction, stratification can be determined for a particular clinical trial in which the subject is or will be involved. For example, the stratification can be assigning individual subjects to various treatment and / or control groups so that the groups have similar default predicted geographic atrophy scores if no treatment is administered, and / or to analyze or normalize results before making comparisons between various groups. A clinical trial can be generated based on the stratification. Clinical trial results can be output.

[0019] In some embodiments, a method is provided. The method includes detecting, at a user device, input data including or identifying a data object corresponding to at least a partial depiction of an eye of a subject. A request communication corresponding to a request to generate a predicted subsequent geographic atrophy characteristic of the eye of the subject is transmitted to a remote computing system. The request communication includes the input data. In response to receiving the request communication, the remote computing system processes the data object using a convolutional neural network to generate a prediction of subsequent growth of a geographic atrophy lesion in the eye or a subsequent size of a geographic atrophy lesion in the eye. The prediction is transmitted from the remote computing system to a user device that receives the prediction.

[0020] In some embodiments, the method can further include collecting a set of images of the subject's eye. The data object can be generated using the set of images.

[0021] Some embodiments of the present disclosure include use of a geographic atrophy prediction in treating a subject. The geographic atrophy prediction is provided by a computing device that implements a computational model based on subject data to provide the geographic atrophy prediction. The computational model includes a convolutional neural network configured to process a data object corresponding to at least a partial representation of the subject's eye.

[0022] Some embodiments of the present disclosure include a system including one or more data processors. The system may further include a non-transitory computer-readable storage medium including instructions that, when executed on the one or more data processors, cause the one or more data processors to perform some or all of one or more of the methods disclosed herein.

[0023] In some embodiments, a computer program product tangibly embodied in a non-transitory machine-readable storage medium is provided, which may include instructions configured to cause one or more data processors to perform some or all of one or more of the methods disclosed herein.

[0024] The present disclosure is described in conjunction with the accompanying drawings, in which: [Brief explanation of the drawings]

[0025] [Figure 1-1] A central B-scan of the OCT volume is shown.

[0026] [Figure 1-2] 1 shows exemplary FAF images used to quantify lesion area grading.

[0027] [Figure 1-3] 1 shows an exemplary identified lesion area.

[0028] [Figure 2] 1 illustrates an exemplary computing network for generating predictions of geographic atrophy lesion size and growth rate, according to some embodiments.

[0029] [Figure 3] 1 illustrates an exemplary process for predicting geographic atrophy lesion size and growth in a three-dimensional data object using a three-dimensional convolutional neural network, according to some embodiments.

[0030] [Figure 4] 1 illustrates an exemplary process of using a 2D convolutional neural network to predict geographic atrophy lesion size and growth of a data object, according to some embodiments.

[0031] [Figure 5]1 illustrates an overview of a prediction framework using a multi-task 3D Inception convolution module, according to some embodiments.

[0032] [Figure 6] 1 illustrates the architecture of a 3D Inception convolution module according to some embodiments.

[0033] [Figure 7] 1 illustrates an overview of a prediction framework using multi-task 2D convolution modules according to some embodiments.

[0034] [Figure 8-1] Detailed performance of the multitask SE model for all holdout cases is shown. (A) shows a regression plot of predicted geographic atrophy growth velocity against true growth velocity; (B) shows a regression plot of predicted geographic atrophy lesion area against true geographic atrophy lesion area; (C) shows OCT en face images and central B-scans of outliers in geographic atrophy growth velocity prediction; (D) shows OCT en face images and central B-scans of outliers in geographic atrophy lesion area and geographic atrophy growth velocity prediction. [Figure 8-2] Detailed performance of the multitask SE model for all holdout cases is shown. (A) shows a regression plot of predicted geographic atrophy growth velocity against true growth velocity; (B) shows a regression plot of predicted geographic atrophy lesion area against true geographic atrophy lesion area; (C) shows OCT en face images and central B-scans of outliers in geographic atrophy growth velocity prediction; (D) shows OCT en face images and central B-scans of outliers in geographic atrophy lesion area and geographic atrophy growth velocity prediction. DETAILED DESCRIPTION OF THE INVENTION

[0035] In the accompanying drawings, similar components and / or features may have the same reference label. Furthermore, various components of the same type may be distinguished by following the reference label with a dash and a second label that distinguishes between the similar components. When only a first reference label is used in this specification, the description is applicable to any of the similar components having the same first reference label, regardless of the second reference label.

[0036] Detailed Description I. Overview The present disclosure describes a multi-task deep learning architecture that enables (for example, simultaneous) prediction of the current geographic atrophy (GA) lesion area and geographic atrophy lesion growth rate.The progression of GA is unique to each individual in terms of lesion area and growth rate.Currently, there is no approved treatment for preventing or delaying the progression of GA.

[0037] Accurate and personalized prediction of geographic atrophy progression can be useful for addressing important clinical and clinical issues. For example, geographic atrophy progression prediction can be used to stratify subjects in clinical trials where slowing the progression of geographic atrophy is the endpoint, enabling targeted enrollment of clinical subjects and improved evaluation of treatment efficacy. Furthermore, personalized GA prognosis can be used to more efficiently manage disease and understand disease pathogenesis by correlating with genotype or phenotype signatures.

[0038] Traditionally, fundus autofluorescence (FAF) images provide two-dimensional structural data regarding GA lesion areas. Optical coherence tomography (OCT) images can provide structural information in addition to lesion area. The additional structural information provided by high-resolution three-dimensional (3D) OCT images can provide insight into the onset and progression of geographic atrophy that is not possible with FAF images alone. For example, reticular pseudodrusen, hyperreflective foci, multilayer thickness reduction, photoreceptor atrophy, and wedge-shaped subretinal hyporeflectance detected on OCT images can be potential precursors or biomarkers associated with geographic atrophy transformation. However, OCT images are difficult to process due to their 3D nature. Current OCT-derived predictions rely on features extracted from OCT volumes rather than complete OCT images due to the complexity of processing 3D image volumes. In short, current OCT predictions involve additional feature extraction, feature engineering, or image processing steps, or some other abstraction of the baseline OCT image.

[0039] This embodiment provides a method for predicting GA directly from baseline OCT volumes without additional steps. In particular, this embodiment includes a multitask 3D neural network for GA lesion size detection and growth rate prediction using baseline 3D OCT images directly. In some embodiments, the deep learning architecture of the neural network can include a convolutional neural network (CNN). The deep learning architecture can be configured to receive (as input) 3D OCT image data and / or 3-channel image data representing data acquired at a baseline time. In some cases, the deep learning architecture is configured to generate predictions corresponding to times after the baseline time, even though it does not receive input collected at later times. Furthermore, the network within the deep learning architecture can be trained end-to-end. This deep learning technique can improve the accuracy of geographic atrophy statistics compared to state-of-the-art baseline models.

[0040] One or more OCT images collected for a subject can be preprocessed. Preprocessing may be performed (for example) to flatten one or more images to compensate for the curvature of the eye. For example, images may be flattened relative to structures such as the internal limiting membrane (ILM). Preprocessing may include registering and / or combining multiple (e.g., flattened) images (e.g., to generate a three-dimensional image). For example, preprocessing may include converting a set of A-scans (e.g., representing multiple longitudinal scans) into B-scans (e.g., to flatten the B-scans along a particular biological structure). In some cases, the flattened images can be combined to generate a three-dimensional image (or flattening can be performed after combining two-dimensional images to form a three-dimensional image). Preprocessing may further include cropping and / or resizing one or more images to a predetermined size and / or by a predetermined amount.

[0041] The preprocessing can include individually generating each of a plurality of data channels. The plurality of channels can include two or more channels or three or more channels. At least one, at least two, or at least three of the channels can correspond to an OCT channel associated with a particular type of scan (e.g., an en face C-scan or a B-scan). In some cases, at least two of the at least three channels can correspond to different types of scans and / or different imaging modalities. For example, at least one channel can include images collected using OCT, and at least another channel can include images collected using fundus photography, infrared imaging, or scanning last ophthalmoscopy (SLO).

[0042] The preprocessed images can be processed by a neural network, including (for example) a deep neural network, a two-dimensional neural network, a three-dimensional neural network, and / or a convolutional neural network. The neural network can include one or more convolutional layers (e.g., where each of two or more convolutional layers or each of all convolutional layers has a convolutional filter of a different size than another convolutional layer). The neural network can include an activation layer, a normalization layer, and / or a pooling layer.

[0043] The neural network may be trained and configured to predict one or more geographic atrophy lesion statistics, such as a current volume of a geographic atrophy lesion, a future volume of a geographic atrophy lesion, and / or a future growth rate. The neural network may be a multi-tasking model trained and configured to simultaneously predict geographic atrophy lesion growth rate and geographic atrophy lesion size. The multi-tasking model may have a lower likelihood of overfitting for the growth rate prediction task than a non-multi-tasking model.

[0044] Figure 1A shows an exemplary B-scan OCT image that can be collected and processed in a deep learning workflow. Figure 1B shows an exemplary fundus autofluorescence (FAF) image. Figure 1C shows lesion grading for the FAF image of Figure 1B. The FAF image includes a geographic atrophy lesion area 105.

[0045] The predicted growth rate or volume of the geographic atrophy lesion can be output (e.g., presented or transmitted) along with the subject and / or eye identification. The predicted growth rate or volume can inform automated or human recommendations for treatment and / or clinical trial enrollment (e.g., subject stratification). For example, the predicted growth rate or volume can be used to determine (e.g., by a human or computing device) whether a subject is eligible for a particular clinical trial. The determination can be made by evaluating rules indicating that the eligibility requirement is that the predicted growth rate or volume exceeds a predetermined lower threshold and / or is below a predetermined upper threshold. As another example, the predicted growth rate or volume can be used for clinical trial stratification (e.g., assigning individual subjects to various treatment and / or control groups so that the groups have similar default predicted geographic atrophy assessments if no treatment is administered and / or to analyze or normalize results before making comparisons between various groups).

[0046] II. Definition As used herein, the term "multitask model" refers to a model that includes one or more neural networks trained to simultaneously predict multiple statistical variables (e.g., current geographic atrophy lesion size and future geographic atrophy lesion growth rate). A multitask model can include one or more neural networks. Because a multitask model is trained to simultaneously fit two sets of information, it can avoid or reduce the possibility of overfitting.

[0047] As used herein, the term "A-scan" refers to a one-dimensional image of the depth profile of the eye along the direction of incident light. Near-infrared light can be directed in a linear direction to generate data at multiple A-scan locations.

[0048] As used herein, the term "B-scan" refers to a cross-sectional view of the eye. In some cases, a B-scan is generated in response to a particular type of measurement. For example, near-infrared light can be transmitted through the eye, and a low-coherence interference signal can generate a two-dimensional B-scan image. In some cases, a B-scan is a frame composed of an array of A-scans. A B-scan can have a transverse, longitudinal, or axial orientation, depending on the orientation of the probe.

[0049] As used herein, the term "en face C-scan" refers to a lateral image of an ocular layer at a specified depth. For example, an en face C-scan can be a lateral image of the retina or choroid layer of the eye. A C-scan can be acquired by scanning fast lateral and slow axially at a given depth. In some cases, an en face C-scan is generated from a set of B-scans.

[0050] As used herein, the term "growth rate" refers to the change in size of a geographic atrophy lesion over time. The change in size may be an absolute change or a relative change. For example, the growth rate can be the change in volume or area of ​​geographic atrophy in the eye. In two dimensions (e.g., area), the growth rate is expressed as the number of units of area (e.g., mm) per unit time. 2 / year) or as a ratio or percentage (e.g., area at a later time point relative to area at baseline). In three dimensions (e.g., volume), growth rate can be measured as unit volume (e.g., mm) per unit time. 3 / year) or as a percentage or ratio (e.g., volume at a later time point relative to volume at baseline).

[0051] As used herein, the term "geographic atrophy size" refers to a measurable dimension of one or more geographic atrophy lesions of an eye. For example, in two dimensions, geographic atrophy size refers to the size of a geographic atrophy lesion (e.g., mm 2 The area of ​​the square meter (measured in mm) can be

[0052] As used herein, the term "current" refers to a baseline time. For example, the current geographic atrophy lesion size may be the geographic atrophy lesion size when the ocular image data was generated.

[0053] As used herein, the term "progression" refers to a measurable difference from a baseline time to a future time. For example, the progression of geographic atrophy can be the growth of geographic atrophy from a baseline time to a future time. In some cases, the progression of geographic atrophy can be determined from the growth rate.

[0054] III. Exemplary Computing Network 2 illustrates an exemplary personalized GA prediction network 200 for generating predictions of geographic atrophy lesion growth rates, according to some embodiments. The GA progress controller 205 can be configured to train and execute machine learning models to generate predictions regarding geographic atrophy status. More specifically, the GA progress controller 205 can be configured to receive one or more images of an eye and output a predicted growth rate or volume of a geographic atrophy lesion.

[0055] The predictions can be generated using one or more machine learning models, such as one or more CNNs. The CNNs can include 3D CNNs and / or 2D CNNs. In some cases, the machine learning models can include ensemble machine learning models that process aggregate results from 3D networks and 2D neural networks.

[0056] The GA progress controller 205 may include an OCT input controller 220 that receives and / or acquires input data including a set of input data objects. The input data objects may include data objects used in a training dataset to train a machine learning model and / or data objects processed by the trained machine learning model.

[0057] Each input data object can correspond to a particular subject, a particular eye, and / or a particular imaging date. The input data object can include (for example) an image of at least a portion of an eye. The input data object can include images generated using imaging techniques disclosed herein, such as OCT and / or fundus photography. For example, the input data object can include a set of A-scan images (e.g., each depicting a different longitudinal scan of a particular eye), a set of B-scan images (e.g., each depicting a flattened image), or a set of C-scan images (e.g., each corresponding to a particular depth).

[0058] The input data objects can be received and / or acquired from one or more imaging systems 225, which can include an OCT device and / or a fundus camera. In some cases, the imaging system 225 transmits one or more images to an image data store 230, which can be partially or fully accessible to the GA progression controller 205.

[0059] The input data objects can be preprocessed by the preprocessing controller 235. The preprocessing can include performing normalization and / or standardization. For example, the preprocessing can include histogram matching across scans or channels within a data object (e.g., a set of B-scans) to unify the intensities of the B-scans or multiple data objects within a dataset. The preprocessing controller 235 can select a B-scan within the set of B-scans to be the reference B-scan based on the average intensity level of the B-scan. The preprocessing controller 235 can perform intensity normalization by performing histogram matching on the remaining B-scans in the set of B-scans.

[0060] The preprocessing controller 235 can also perform segmentation (e.g., BM segmentation or ILM segmentation) on each B-scan during preprocessing. After segmentation, each B-scan can be flattened along a delineation of a specific biological structure (e.g., the Bruch's membrane (BM) boundary or the internal limiting membrane (ILM) boundary). Segmentation can facilitate volume truncation (e.g., removing upper and lower background portions to reduce neural network limitations) and generate an en face map. After flattening, the preprocessing controller 235 can crop each B-scan to a region of pixels around the boundary. For example, a flattened B-scan can be cropped to a region 5 pixels above the ILM and 250 pixels below the ILM, or to a region 120 pixels above the BM and 135 pixels below the BM. The cropped region can be resized to a predetermined size (e.g., 512 pixels by 512 pixels) using (for example) resampling, interpolation, and / or bilinear interpolation.

[0061] The personalized GA prediction network 200 can include a lesion label detector 240 that can obtain one or more labels associated with each input data object in the training dataset. The labels may be initially identified based on information provided by an annotator, a medical expert, and / or a validated database. The labels can include the size of geographic atrophy lesions at subsequent times, the cumulative size of geographic atrophy lesions at subsequent times, the difference between the baseline time and subsequent times, the progression of geographic atrophy lesion size, and / or the progression of cumulative size of geographic atrophy lesions.

[0062] The model training controller 245 can execute code for training one or more machine learning models using one or more training datasets. A machine learning model can include one or more pre-processing functions, one or more neural networks, and / or one or more post-processing functions. It will be understood that for each function, algorithm, and / or network in a model, one or more variables can be fixed during training. For example, hyperparameters of a neural network (e.g., identifying the number of layers, the size of the input layer, the learning rate, etc.) can be predefined and non-tunable through the execution of code. It will also be understood that for each function, algorithm, and / or network in a model, one or more variables can be learned through training. For example, parameters of a neural network (e.g., identifying various inter-node weights) can be learned.

[0063] Each training data set can include a set of training data objects, each of which can include an input image (e.g., an image showing a portion of an eye or a flattened version thereof), collected at a baseline time.

[0064] Each of the data objects can be further associated with one or more labels (e.g., two or more labels). Each of the labels can be based at least in part on information collected at a time after the baseline time (e.g., at least two weeks, at least one month, at least two months, at least six months, at least one year, at least two years, at least five years, or at least ten years after the baseline time). In some cases, the labels of the one or more labels can include the size of a given geographic atrophy lesion at a subsequent time, the cumulative size of the geographic atrophy lesion at a subsequent time, the difference between the baseline time and the subsequent time, the progression of the size of a given geographic atrophy lesion, and / or the progression of the cumulative size of the lesion. Thus, the training dataset can include baseline images and label data indicating how and / or whether a subject's geographic atrophy has progressed over a period of time.

[0065] The model training controller 245 can use the training data to train a machine learning model. In addition to one or more conventional machine learning models (e.g., neural networks, bin-based classifiers, regression algorithms, etc.), the machine learning model can further include one or more pre-processing functions and / or one or more post-processing functions. For example, a pre-processing function can adjust the size, intensity distribution, and / or viewpoint of one or more input images, and / or a post-processing function can convert model output into values ​​along a predetermined scale (or category list), recommendations, stratification identifiers, etc.

[0066] The model training controller 245 can train various components of the machine learning model separately or jointly. For example, a training data set can be used to simultaneously train one or more pre-processing algorithms (e.g., to perform segmentation) and a neural network. As another example, a pre-processing algorithm may be trained separately from the neural network (e.g., using the same or different portions of the training data).

[0067] The GA prediction generator 250 can process non-training data using the architecture and learned parameters to generate results. The results can be generated in response to a request communication from the client device 210 (e.g., including or identifying input data). For example, the request can identify a database identifier, subject, eye, and access password to access a set of images of the subject's eye. The request communication can correspond to a request to generate predicted subsequent geographic atrophy characteristics (e.g., size or growth) of the subject's eye. The subjects to which the request relates can be different from each subject represented in the training data.

[0068] In some cases, the request includes a data object configured to be input to a machine learning model. In some cases, the request includes a data object configured to undergo pre-processing (e.g., decryption, normalization, etc.) configured to then be input to a machine learning model. In some cases, the request includes information that can be used to obtain the data object configured to be input to a machine learning model (e.g., one or more identifiers, a verification code, and / or a passcode).

[0069] In some cases, the post-processing function processes predictions using (for example) models, features, or rules. For example, a classifier can be used to assign a severity prediction outcome to a subject's eye input dataset based on preliminary results. As another example, a rule can be used to determine whether a treatment change is recommended for consideration based on preliminary results.

[0070] Thus, the GA prediction generator 250 can access input data objects and feed the data objects to a machine learning model. The results of the machine learning model can predict the current geographic atrophy lesion size (e.g., at the time the data object is generated) and / or subsequent geographic atrophy lesion status (e.g., subsequent geographic atrophy lesion growth or size). The GA progress controller 205 can output a prediction characterizing the subsequent geographic atrophy growth in the eye (e.g., growth of one or more geographic atrophy lesions in the eye) and / or subsequent geographic atrophy size in the eye (e.g., subsequent size of one or more geographic atrophy lesions in the eye). For example, the prediction can include the probability or likelihood of the subsequent growth or subsequent size of geographic atrophy in the subject's eye over a subsequent period of time, the predicted absolute or relative growth, and / or the predicted relative or absolute size of the geographic atrophy lesion. The GA progress controller 205 may further generate a prediction of the current size (e.g., area or volume) of one or more geographic atrophy lesions in the eye.

[0071] In some cases, the GA prediction generator 250 may implement a 3D convolutional neural network (e.g., trained by the model training controller 245). The 3D convolutional neural network may be configured to receive and process a 3D data object (e.g., corresponding to a set of flattened B-scans). The 3D convolutional neural network may include one or more 3D convolutional modules (e.g., an Inception convolutional module). Each 3D convolutional module may include a set of convolutional layers using at least two different sizes of convolutional filters (e.g., using a 1x1x1 convolutional filter, a first 3x3x3 convolutional filter, and a second 3x3x3 convolutional filter). Alternatively or additionally, the 3D convolutional network may include one or more 3D convolutional layers.

[0072] In some cases, the GA prediction generator 250 can implement two-dimensional and / or multi-channel neural networks. For example, the data object can include multi-channel image data. Each channel within the image channel can include two-dimensional data corresponding to (for example) one or more imaging techniques and / or one or more imaging perspectives. For example, the two-dimensional image data can include B-scans and / or one or more C-scans. Thus, in some cases, the multi-channel image data includes data corresponding to two or more different types of imaging acquisitions and / or two or more different types of perspectives. The multi-channel image data can be processed by one or more neural networks to generate a prediction of the growth rate and / or subsequent size of geographic atrophy lesions.

[0073] Each of the one or more channels of the data channels can include a two-dimensional image corresponding to at least a partial depiction of the subject's eye. In some examples, the three-channel data can include multiple en face C-scan OCT images. As another example, the three-channel data can include at least two en face C-scan OCT images and at least one B-scan image (e.g., a central B-scan or one or more B-scans corresponding to pseudo-randomly selected lateral positions). As another example, the three-channel data can include at least one en face C-scan OCT image and at least one image obtained using a type of imaging modality other than OCT (e.g., infrared and / or fundus autofluorescence imaging or cSLO). Alternatively, the three-channel data can include a full-depth en face OCT image, a sub-BM en face OCT image (e.g., 100 pixels deep below the BM), and an upper BM en face OCT image (e.g., 100 pixels deep above the BM). While some disclosures herein refer to "three-channel" data, object data, etc., it is understood that data including more than three channels can be used instead.

[0074] The neural network used by the GA prediction generator 250 may include a transfer learning neural network (Inception V3) and / or may be followed by a compact layer. For example, the neural network may be initialized with parameters for predicting the growth rate and / or subsequent size of different types of ocular lesions (e.g., groin lesions, conjunctival lesions, or choroidal melanoma). The neural network may then be further trained to predict the growth rate and / or subsequent size of geographic atrophy lesions in the eye.

[0075] The network may include batch normalization, one or more activation layers (e.g., using ReLU activation), and / or one or more pooling layers (e.g., using max pooling). For example, batch normalization, ReLU activation, and / or pooling layers may follow each of one or more convolutional layers. The neural network may include global average pooling and / or dense layers (e.g., in the output layer and / or a layer following the last convolutional layer). The neural network may include an activation function (e.g., linear activation). The neural network may include an attention unit, such as a standard squeeze-and-excite (SE) attention unit. The SE attention unit may include global average pooling of feature maps from the convolutional layers and applying one or more linear transformations to the pooling result. The attention unit may be configured to facilitate adaptively adjusting the weights of each feature map.

[0076] In some cases, the GA prediction generator 250 can process the data object using an ensemble model. The ensemble model can include multiple neural networks, such as one or more 3D convolutional neural networks and one or more 2D convolutional neural networks. The ensemble model can process the data object and aggregate the results from each neural network. For example, the ensemble model can average the predicted subsequent size of geographic atrophy lesions and / or the predicted growth rate of geographic atrophy lesions from each neural network. The ensemble model can output the average predicted subsequent size and / or subsequent growth of geographic atrophy in the subject's eye.

[0077] The GA analysis controller 255 can process the predictions and communicate the results (or a processed version thereof) to the client device 210 or other systems (e.g., associated with a laboratory technician or care provider). For example, the GA analysis controller 255 can generate a prediction output that identifies the subsequent size of geographic atrophy or subsequent growth of geographic atrophy in the subject's eye. In some cases, the GA analysis controller 255 can further post-process the results. For example, the GA analysis controller 255 may evaluate one or more rules to determine whether any condition of the one or more rules is met. For example, a rule can be configured to be met if the predicted lesion size and / or predicted progression is large enough to recommend a change in treatment. As another example, a rule can be configured to be met to recommend a particular type of treatment approach if the predicted lesion size and / or predicted progression is within a predetermined open or closed range. As another example, a rule can be configured to indicate whether a subject is eligible for a clinical trial based on the predicted lesion size and / or predicted progression.

[0078] The output may then be presented and / or transmitted, which may facilitate display of the output data on a display of a computing device (eg, client device 210), for example.

[0079] IV. Techniques for predicting geographic atrophy lesion area and growth rate FIG. 3 illustrates an exemplary process for using a convolutional neural network to predict geographic atrophy size and growth in a three-dimensional data object, according to some embodiments. In block 305, a three-dimensional data object corresponding to at least a partial depiction of a subject's eye is generated. The three-dimensional data object can be generated by segmenting each image in a set of two-dimensional images (e.g., B-scans) to identify segments within the image (e.g., corresponding to Bruch's membrane and / or internal limiting membrane). Alternatively, the three-dimensional data object can be generated by accessing two-dimensional B-scan OCT images of the subject's eye, generating feature maps using the two-dimensional B-scan OCT images, and generating the three-dimensional data object to include the set of feature maps. As another example, the three-dimensional data object can be generated by accessing a set of two-dimensional B-scan OCT images of the subject's eye. A set of pixels depicting a particular structure of the eye can be identified for each of the set of two-dimensional B-scan OCT images. For example, a set of pixels depicting Bruch's membrane and / or internal limiting membrane can be identified. Each of the two-dimensional B-scan OCT images can be flattened based on the set of pixels. A three-dimensional data object can then be generated to include at least a portion of each flattened B-scan OCT image.

[0080] At block 310, a three-dimensional data object is accessed. The three-dimensional data object can be accessed from local memory or storage. In some examples where the three-dimensional data object is generated at an imaging or remote system, the three-dimensional data object can be accessed from an imaging or remote system (e.g., imaging system 225 of FIG. 2). The three-dimensional data object can be accessed in response to a remote computing system receiving a request communication (e.g., from a client system) to generate a predicted subsequent geographic atrophy characteristic of the subject's eye.

[0081] In block 315, the three-dimensional data object is processed using a convolutional neural network to generate a prediction of the subsequent growth and / or subsequent size of the geographic atrophy lesion in the eye. The convolutional neural network can be a three-dimensional neural network having one or more convolutional modules (e.g., an inception module) and / or one or more pooling layers. Each convolutional module can include a three-dimensional convolutional module including a set of convolutional layers. At least two of the set of convolutional layers can include convolutional filters of different sizes. For example, a first layer can use a 1x1x1 convolutional filter, and each of one or more second layers can use a 3x3x3 convolutional filter. The convolutional neural network can further include an attention unit, such as a standard SE attention unit, that can add parameters to each channel of the convolutional block to facilitate adaptive adjustment of the weights of each feature map.

[0082] At block 320, the prediction is output. The prediction can be used to facilitate a determination of the subject's eligibility to participate in a particular clinical trial. For example, a prediction of geographic atrophy size above a threshold can indicate that the subject is eligible for a particular clinical trial. The prediction can additionally or alternatively be used to facilitate a stratification determination for a particular clinical trial in which the subject is or will be involved. For example, the prediction can be used to assign subjects to various treatment and / or control groups so that the groups have similar default predicted geographic atrophy scores if no treatment is administered, and / or to analyze or normalize results before making comparisons between various groups. Clinical trial results can be generated based on the stratification. The clinical trial results may also be output.

[0083] In some cases, predictions can be used to facilitate the selection of a subject's treatment and / or facilitate the decision of whether to change the subject's treatment.For example, a machine learning model can be trained using a training data set corresponding to a set of subjects who have received or have received a particular treatment.The baseline images collected for each subject can correspond to the period when the treatment has just begun or is about to begin.The trained model can then be used to predict the extent to which geographic atrophy will progress and / or the size of subsequent geographic atrophy lesions if the subject uses a particular treatment, and thus can inform caregivers whether to recommend a particular treatment (e.g., over another treatment).

[0084] FIG. 4 illustrates an exemplary process for using a convolutional neural network to predict geographic atrophy lesion size and growth in a data object, according to some embodiments. At block 405, a data object including at least three data channels is accessed. Each of the at least three data channels may include a two-dimensional image corresponding to at least a partial depiction of the subject's eye. The data object may include multiple different en face OCT-based maps of the eye. In some cases, the data object may include at least two en face OCT-based scans of the eye and at least one B-scan of the eye (e.g., a central B-scan or one or more B-scans corresponding to pseudo-randomly selected lateral positions). Alternatively, the data object may include at least one en face OCT-based scan of the eye and at least one image obtained using a type of imaging modality other than OCT.

[0085] In block 410, the data object is processed using a convolutional neural network to generate a prediction of the subsequent growth and / or subsequent size of the geographic atrophy lesion in the eye. The convolutional neural network can additionally or alternatively generate another prediction of the current size of the geographic atrophy lesion in the eye. The convolutional neural network can be a two-dimensional neural network trained to simultaneously predict the current geographic atrophy lesion size and the subsequent geographic atrophy lesion status, which can include the subsequent geographic atrophy lesion growth or the subsequent geographic atrophy lesion size. The convolutional neural network can be trained using transfer learning. For example, the convolutional neural network can be initialized with parameters for predicting the growth rate and / or subsequent size of different types of ocular lesions (e.g., groin lesions, conjunctival lesions, or choroidal melanoma). The neural network can then be further trained to predict the growth rate and / or subsequent size of the geographic atrophy lesion in the eye. The convolutional neural network can include one or more two-dimensional convolutional modules and / or pooling layers. The convolutional neural network may further include one or more optional units (e.g., SE attention units). The convolutional neural network may also include batch normalization, one or more activation layers (e.g., using ReLU activation), and / or one or more pooling layers (e.g., using max pooling).

[0086] At block 415, a prediction (e.g., subsequent growth, current size, and / or future size) is output. The prediction can be used to facilitate a determination of the subject's eligibility to participate in a particular clinical trial. For example, a prediction of geographic atrophy lesion size above a threshold may indicate that the subject is eligible for a particular clinical trial. The prediction can additionally or alternatively be used to facilitate a stratification determination for a particular clinical trial in which the subject is or will be involved. Clinical trial results can be generated based on the stratification. The clinical trial results may further be output.

[0087] In some cases, predictions can be used to facilitate the selection of a subject's treatment and / or facilitate the decision of whether to change the subject's treatment.For example, a machine learning model can be trained using a training data set corresponding to a set of subjects who have received or have received a particular treatment.The baseline images collected for each subject can correspond to the period when the treatment has just begun or is about to begin.The trained model can then be used to predict the extent to which geographic atrophy will progress and / or the size of subsequent geographic atrophy lesions if the subject uses a particular treatment, and thus can inform caregivers whether to recommend a particular treatment (e.g., over another treatment). [Example]

[0088] V. Working Examples VA Example 1: Prediction of geographic atrophy growth rate by processing 3-dimensional OCT images using neural networks VA1. Method The study was conducted retrospectively on the eyes of subjects with bilateral geographic atrophy who were enrolled in the lampalizumab phase 3 trial SPECTRI (NCT 02247531). 3 Macular Spectralis of 496 × 1024 × 49 voxels in the area SD-OCT volumes (Heidelberg Engineering, Inc., Heidelberg, Germany) were used to predict geographic atrophy lesion area and lesion growth. Geographic atrophy lesion area (mm 2 , measured from FAF images graded by two readers or arbitrators as appropriate) and geographic atrophy lesion growth rate (mm 2 The mean mean (m / year) was derived using all available visit measurements and fitted to a linear model. The image dataset from 522 subjects was split into five folds per subject. Five-fold cross-validation was performed with all folds balanced for baseline factors and treatment group. Each B-scan was flattened along the ILM and reduced to a size of 128 × 128 pixels. A multitask 3D convolutional neural network (CNN) model was then trained to predict geographic atrophy growth rate from baseline OCT volumes with initialized weights from the same model trained to predict geographic atrophy area. The within-sample coefficient of determination (R), defined as the square of the Pearson correlation coefficient (r) between the true and predicted geographic atrophy growth rates, was used. 2 The performance was evaluated by calculating

[0089] VA2.Result Geographic atrophy lesion area prediction had an average cross-validated R of 0.88 2 (range: 0.77 to 0.92), and the average cross-validated R 2 The results showed a mean mean (range: 0.21-0.43). Growth velocity prediction results are comparable to previous studies using baseline multimodal retinal images (Normand G et al., IOVS 2019;60:ARVO E-Abstract 1452) to predict geographic atrophy growth velocity (R2 = 0.35) using deep learning.

[0090] This example demonstrates the feasibility of using OCT images to predict individual geographic atrophy growth rate using a 3D deep learning approach. Predictions can be improved by combining them with other imaging modalities (e.g., infrared and fundus autofluorescence) and / or relevant clinical variables.

[0091] VB Example 2: Prediction of geographic atrophy growth rate by processing OCT images using neural networks VB1.3D model method The prediction is formulated as a multi-task regression problem. The training dataset is

number

[0092] An overview of this approach is shown in Figure 5. The 3D OCT volume was first preprocessed and then input into a multitask 3D Inception CNN-based model. Furthermore, a self-attention mechanism using squeeze and excitation (SE) units was added to enhance the feature map. The geographic atrophy lesion area and growth rate were simultaneously predicted. It has been shown that geographic atrophy growth rate is weakly correlated with geographic atrophy lesion area. The multitask model is expected to find a representation that incorporates information from both tasks, reducing the possibility of overfitting to the growth rate prediction task alone.

[0093] VB1.a.OCT Image Preprocessing The OCT volume contained 1,024 A-scans and 49 B-scans (496 × 1,024 × 49 voxels) with a depth of 496 pixels. The volume size was reduced to focus on the retina (region of interest). For each B-scan, the image was flattened along the internal limiting membrane (ILM). The ILM is the boundary between the retina and vitreous, which was pre-segmented by the OCT device software (Heidelberg Engineering, Inc., Heidelberg, Germany). Next, for each B-scan, a region 5 pixels above the ILM and 250 pixels below the ILM was cropped and defined as the region of interest. Each cropped region (256 pixels deep × 1,024 A-scans) was then resized to 128 × 128 pixels by bilinear interpolation (Figure 5). There was no reduction in the number of B-scans. Therefore, the original volume was reduced to 128 × 128 × 49 voxels as model input.

[0094] VB1.b. 3D Inception Convolution Module The details of the 3D Inception convolution module are shown in Figure 6. The network consisted of one regular convolutional layer and four 3D Inception convolutional blocks. Of these, the regular convolutional layer and the first three 3D Inception convolutional blocks were shared by area and growth rate prediction. After each convolutional layer, batch normalization, ReLU activation, and max pooling were performed. After the last convolutional block, a dense layer with global average pooling and linear activation was used in the output layer. The anisotropy of the 3D volume was taken into account in the selection of filter size, stride, and pooling size. Hyperparameters were optimized using random search.

[0095] VB1.c.SE Attention Unit and Joint Loss We generalized the 2D SE attention unit to 3D and added it to three 3D Inception modules. As in the 2D case, the feature maps from the Inception modules are reweighted by the output from the SE units to account for interdependencies between channels. More specifically, the output feature maps from the 3D Inception modules, with dimensions H × W × D × C, are transformed into a global tensor of dimensions 1 × 1 × 1 × C by 3D spatial averaging. This tensor is essentially a channel-oriented vector, with the same dimensions as the number of channels. A fully connected layer with ReLU activation is applied in the channel direction to reduce the output tensor to dimensions 1 × 1 × 1 × C / r, where r is called the reduction ratio and is a divisor of C, the number of channels. In this experiment, the reduction ratio was set to 4. A second fully connected layer followed by a sigmoid activation with output dimensions 1 × 1 × 1 × C provides smooth gating for each channel. Finally, each feature map in the convolutional block is weighted by multiplying it by the output from the SE unit. Finally, a joint loss of mean squared error of GA lesion area prediction and growth rate prediction was used for training, with each term weighted at 0.5.

[0096] VB1.d. Baseline Model To compare performance and gain insight into the models, three additional baseline models were tested. The first baseline model (i.e., the base model) was a simple 3D Inception convolutional neural network for predicting only geographic atrophy growth rate. The model was identical to the proposed model without the SE attention unit and multitasking setting. He initialization was employed for model parameter initialization. The second baseline model (i.e., the cascade model) used a cascade approach, first training the model to predict geographic atrophy lesion area. The weights learned during area prediction training were then used to initialize the model, which was then trained to predict geographic atrophy growth rate. The cascade models trained for each type of prediction shared the same 3D convolutional architecture as the first baseline model. The third baseline model (i.e., the multitask model) was a multitask model without the SE attention unit to see whether the SE attention block provided additional performance improvements. The second and third baseline models were trained to predict current and future geographic atrophy lesion area.

[0097] VB2.2D model method Figure 7 shows an overview of a multi-task 2D convolutional network 700. The model includes a transfer learning Inception V3 neural network 705. A dense layer 710 is included after the transfer learning Inception V3 network 705. The geographic atrophy lesion area 715 and growth rate 720 are predicted simultaneously. The 2D multi-task model is expected to find a representation that captures information from both tasks.

[0098] VB3.Result The model was run on the full training dataset of patients with bilateral geographic atrophy enrolled in the lampalizumab Phase 3 trial SPECTRI (NCT 02247531) and evaluated five times against a holdout dataset. The training dataset contained 1934 visits from 560 subjects, and the holdout dataset contained 114 baseline visits from 114 subjects. The mean performance and standard deviation for the holdout dataset are shown in Table 1. The results show that the proposed model performed best for predicting geographic atrophy growth rate. The multitasking framework contributed most of the improvement compared to the first baseline model. SE attention blocks did not appear to significantly improve performance. The cascade model also performed better than the first baseline model. In contrast, the performance of geographic atrophy lesion area prediction was comparable across all models.

[0099] Examples of regression plots of predicted geographic atrophy growth rate against true growth rate, and predicted geographic atrophy lesion area against true geographic atrophy lesion area, for all holdout cases from additional runs of the model are shown in Figures 8A and 8B. True vs. predicted lesion area predictions (R 2 The squared Pearson correlation coefficient between the FAF and infrared retinal images was 0.92. Performance when the geographic atrophy lesion area was relatively small was better than when the lesion area was larger. The performance of growth velocity prediction was comparable to that of deep learning prediction of geographic atrophy growth velocity (R ) using both baseline FAF and infrared retinal images from a much larger cohort (2000+ eyes). 2This was comparable to previous studies on the OCT en face image and central B-scan of an outlier in geographic atrophy growth rate prediction. The OCT volume can be seen to have motion artifacts and pixel intensity inconsistencies across B-scans. Furthermore, the OCT volume size in this case was 496 × 512 × 49 voxels, and only approximately 5% of the training data had this resolution. However, the prediction of geographic atrophy lesion area was consistent with the ground truth. Figure 8D shows the OCT en face image and central B-scan of another outlier in both geographic atrophy lesion area and growth rate prediction. In this case, image quality was again likely the underlying cause, as some B-scans had significantly lower image intensity, dark bands in the OCT en face image. [Table 1]

[0100] VB4.Interpretation Therefore, a deep learning model was successfully trained to process OCT volumes and predict geographic atrophy disease progression without any prior feature manipulation. The novel customized 3D multitask deep learning pipeline simultaneously predicted the current geographic atrophy lesion area and future geographic atrophy lesion growth rate with high accuracy based solely on baseline OCT volumes. The proposed model outperformed other baseline models.

[0101] VI. Further Considerations Some embodiments of the present disclosure include a system including one or more data processors. In some embodiments, the system includes a non-transitory computer-readable storage medium containing instructions that, when executed on the one or more data processors, cause the one or more data processors to perform some or all of one or more methods and / or some or all of one or more processes disclosed herein. Some embodiments of the present disclosure include a computer program product tangibly embodied in a non-transitory machine-readable storage medium containing instructions configured to cause one or more data processors to perform some or all of one or more methods and / or some or all of one or more processes disclosed herein.

[0102] The terms and expressions which have been employed are used as terms of description rather than of limitation, and there is no intention in the use of such terms and expressions to exclude equivalents of the features shown and described or portions thereof, but it is recognized that various modifications are possible within the scope of the invention as claimed. Thus, although the claimed invention has been specifically disclosed by embodiments and optional features, it will be understood that modifications and variations of the concepts disclosed herein may be resorted to by those skilled in the art, and that such modifications and variations are deemed to be within the scope of the invention as defined by the appended claims.

[0103] This description provides only preferred exemplary embodiments and is not intended to limit the scope, applicability, or configuration of the present disclosure. Rather, this description of preferred exemplary embodiments provides those skilled in the art with an enabling description for implementing various embodiments. It will be understood that various changes can be made in the function and arrangement of elements without departing from the spirit and scope of the appended claims.

[0104] In this description, specific details are set forth to provide a thorough understanding of the embodiments. However, it will be understood that the embodiments may be practiced without these specific details. For example, circuits, systems, networks, processes, and other components may be shown as components in block diagram form in order to avoid obscuring the embodiments in unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring the embodiments. In certain embodiments, for example, the following are provided: (Item 1) accessing a three-dimensional data object corresponding to at least a partial representation of an eye of the subject; processing the three-dimensional data object using a convolutional neural network to generate a prediction of subsequent growth of a geographic atrophy lesion in the eye or a subsequent size of the geographic atrophy lesion in the eye; outputting the prediction; and A method comprising: (Item 2) Item 10. The method of claim 1, wherein the convolutional neural network includes one or more three-dimensional convolutional modules. (Item 3) 3. The method of claim 1, wherein the convolutional neural network includes a pooling layer. (Item 4) 4. The method of any one of items 1 to 3, wherein the convolutional neural network is trained to simultaneously predict current geographic atrophy lesion size and subsequent geographic atrophy state, wherein the subsequent geographic atrophy state comprises subsequent geographic atrophy lesion growth or subsequent geographic atrophy lesion size. (Item 5) 5. The method of any one of items 1 to 4, wherein processing the three-dimensional data object with the convolutional neural network further generates another prediction of a current size of the geographic atrophy lesion in the eye. (Item 6) 6. The method of any one of items 1 to 5, wherein the subsequent growth of the geographic atrophy lesion comprises the growth of one or more geographic atrophy lesions in the eye and / or the subsequent size of the geographic atrophy lesion comprises the subsequent size of one or more geographic atrophy lesions in the eye. (Item 7) 7. The method of any one of items 1 to 6, further comprising generating the three-dimensional data object by performing a series of operations including segmenting each image of the set of two-dimensional images to identify segments bounded by predicted locations of Bruch's membrane and / or internal limiting membrane. (Item 8) accessing a set of two-dimensional B-scan OCT images of the eye of the subject; generating a set of feature maps using the set of two-dimensional B-scan OCT images; 7. The method of any one of items 1 to 6, further comprising generating the three-dimensional data object by performing a series of operations including: (Item 9) accessing a set of two-dimensional B-scan OCT images of the eye of the subject; identifying, for each of the set of two-dimensional B-scan OCT images, a set of pixels that depict a particular structure of the eye; for each of the set of 2D B-scan OCT images, flattening the 2D B-scan OCT image based on the set of pixels; generating the three-dimensional data object to include at least a portion of each flattened B-scan OCT image; 7. The method of any one of items 1 to 6, further comprising generating the three-dimensional data object by performing a series of operations including: (Item 10) training another convolutional neural network using the training data set to generate a first set of learned parameter values; setting a set of parameter values ​​of the convolutional neural network to the first set of learned parameter values; further training the convolutional neural network using another training data set to generate a second set of learned parameter values, the second set of learned parameter values ​​being constituted by the second set of learned parameter values ​​when the convolutional neural network processes the three-dimensional data object; 10. The method of any one of items 1 to 9, further comprising: training the convolutional neural network using transfer learning by: (Item 11) 11. The method of any one of items 1 to 10, wherein the three-dimensional data object comprises a three-dimensional representation of the eye volume of the subject. (Item 12) inputting, by a user, input data that includes or identifies said three-dimensional data object; receiving the prediction; and determining that the subject is eligible to participate in a particular clinical trial based on the prediction; 12. The method of any one of items 1 to 11, further comprising: (Item 13) inputting, by a user, input data that includes or identifies said three-dimensional data object; receiving the prediction; and determining a stratification for a particular clinical trial in which the subject is or will be involved based on said prediction; 12. The method of any one of items 1 to 11, further comprising: (Item 14) generating clinical trial results based on said stratification; outputting the clinical trial results; Item 14. The method of item 13, further comprising: (Item 15) inputting, by a user, input data that includes or identifies said three-dimensional data object; receiving the prediction; and determining adjustments for a particular clinical trial in which the subject is or will be involved based on said prediction; Facilitating the conduct of such specific clinical trials in a coordinated manner; and 12. The method of any one of items 1 to 11, further comprising: (Item 16) receiving input data including or identifying a three-dimensional data object corresponding to at least a partial representation of an eye of a subject; receiving a request communication corresponding to a request to generate a predicted subsequent geographic atrophy characteristic of the eye of the subject, the request communication including the input data; processing the three-dimensional data object using a convolutional neural network to generate a prediction of subsequent growth of a geographic atrophy lesion in the eye or a subsequent size of the geographic atrophy lesion in the eye; outputting the prediction; and A method comprising: (Item 17) acquiring a set of images of the eye of the subject, wherein the three-dimensional data object is generated using the set of images. Item 17. The method of item 16, further comprising: (Item 18) 1. Use of a geographic atrophy prediction in treating a subject, wherein the geographic atrophy prediction is provided by a computing device that implements a computational model based on subject data to provide the geographic atrophy prediction, the computational model comprising a convolutional neural network configured to process a three-dimensional data object corresponding to at least a partial representation of an eye of the subject. (Item 19) accessing a data object including at least three data channels, each of the at least three data channels including a two-dimensional image corresponding to at least a partial representation of an eye of the subject; processing the data object using a convolutional neural network to generate a prediction of subsequent growth of a geographic atrophy lesion in the eye or a subsequent size of the geographic atrophy lesion in the eye; outputting the prediction; and A method comprising: (Item 20) 20. The method of claim 19, wherein the data object comprises a plurality of different en face OCT-based maps of the eye. (Item 21) 21. The method of claim 19 or 20, wherein the data object comprises at least two en face OCT-based scans of the eye and at least one B-scan of the eye. (Item 22) The data object comprises at least one OCT-based en face scan of the eye and at least one OCT-based en face scan of the eye obtained using an imaging modality of a type different from OCT. 22. The method of any one of items 19 to 21, comprising: (Item 23) one or more data processors; a non-transitory computer-readable storage medium containing instructions that, when executed on the one or more data processors, cause the one or more data processors to perform some or all of one or more of the methods disclosed herein; A system comprising: (Item 24) A computer program product tangibly embodied in a non-transitory machine-readable storage medium comprising instructions configured to cause one or more data processors to perform some or all of one or more of the methods disclosed herein.

Claims

[Claim 1] The invention described in the specification.

Images