Neural network processing of OCT data to generate predictions of geographic atrophy growth rate

Abstract

Embodiments disclosed herein generally relate to predicting geographic atrophy lesion growth and / or geographic atrophy lesion size in an eye. The predictions can be generated by processing data objects using a neural network. The data objects can include a three-dimensional data object representing a depiction of at least a portion of the eye or a multi-channel data object representing one or more depictions of at least a portion of the eye. The neural network can include a convolutional multi-task neural network trained to learn features that predict both lesion growth and lesion size outputs.

Description

[0001] Cross-reference to related applications

[0002] This application claims the benefit and priority of U.S. Provisional Patent Application Nos. 62 / 944,201 (filed December 5, 2019) and 62 / 988,797 (filed March 12, 2020). Each of these applications is hereby incorporated herein by reference in its entirety for all purposes. Background Technology

[0003] Geographic atrophy (GA) is a type of advanced age-related macular degeneration (AMD) that causes degeneration of photoreceptors and supporting cells, leading to progressive vision loss. This condition affects millions of people worldwide. In developed countries, approximately 1 in 29 people aged 75 and older have geographic atrophy. Geographic atrophy is characterized by the progressive loss of structures in the retinal pigment epithelium (RPE), adjacent photoreceptors, and the choroidal capillary layer. The progression of geographic atrophy demonstrates significant patient variability. Currently, there are no approved treatments to prevent or slow the progression of geographic atrophy.

[0004] Geographic atrophy lesions can be imaged using various imaging modalities. Traditionally, two-dimensional fundus autofluorescence (FAF) images are used to quantify geographic atrophy lesion regions. The change in the lesion region derived from FAF over a period of time (geographic atrophy growth rate) serves as an anatomical outcome parameter indicating whether and / or the extent of progression of geographic atrophy in a subject. Nevertheless, the two-dimensional nature of FAF images may limit their ability to provide refined structural information about the lesion region. Therefore, in addition to quantifying GA lesion regions derived from FAF images, improved assessments of the lesion regions are needed to enhance the understanding of GA onset and progression. Summary of the Invention

[0005] In some embodiments, a method is provided. A three-dimensional data object corresponding to a depiction of at least a portion of a subject's eye is accessed. The three-dimensional data object may include a three-dimensional depiction of the volume of the subject's eye. The three-dimensional data object is processed using a convolutional neural network (e.g., an Inception neural network) to generate a prediction of the subsequent growth of map-like atrophy lesions in the eye or the subsequent size of map-like atrophy lesions in the eye. The subsequent growth of map-like atrophy lesions may include the growth of one or more map-like atrophy lesions in the eye, and / or the subsequent size of map-like atrophy lesions may include the subsequent size of one or more map-like atrophy lesions in the eye. The convolutional neural network may include one or more three-dimensional convolutional modules, pooling layers, and / or one or more optional units (e.g., attention units). The prediction is output.

[0006] In some embodiments, a convolutional neural network may be trained to simultaneously predict the current size of the map-like atrophy lesion and the subsequent map-like atrophy state. The subsequent map-like atrophy state may include subsequent map-like atrophy lesion growth or subsequent map-like atrophy lesion size.

[0007] In some embodiments, the processing of three-dimensional data objects using convolutional neural networks can further lead to another prediction of the current size of map-like atrophic lesions in the eye.

[0008] In some embodiments, the method may include generating a three-dimensional data object. For example, each image in a set of two-dimensional images may be segmented to identify segments defined by predicted locations of the Bruch's membrane and / or the internal limiting membrane. Generating the three-dimensional data object may also include accessing a set of two-dimensional B-scan OCT images of the subject's eye. A set of feature maps may be generated from the set of two-dimensional B-scan OCT images. A three-dimensional data object may be generated to include a set of feature maps.

[0009] In some embodiments, generating a three-dimensional data object includes accessing a set of two-dimensional B-scan OCT images of the subject's eye. For each image in the set of two-dimensional B-scan OCT images, a set of pixels depicting a specific structure of the eye (e.g., Bruch's membrane or internal limiting membrane) can be identified. Each image in the set of B-scan OCT images can be flattened based on the set of pixels. A three-dimensional data object can be generated to include at least a portion of each flattened B-scan OCT image.

[0010] In some embodiments, the method may include training another convolutional neural network using a training dataset to produce a first set of learned parameter values. Transfer learning can be used to train the convolutional neural network by setting a set of parameter values ​​for the convolutional neural network as the first set of learned parameter values, and further training the convolutional neural network using another training dataset to produce a second set of learned parameter values. When processing three-dimensional data objects, the convolutional neural network may be configured with the second set of learned parameter values.

[0011] In some embodiments, the method may further include determining aspects of a clinical study based on a prediction. For example, input data may be input by a user, including or identifying three-dimensional data objects. A prediction may be received, and based on this prediction, it may be determined whether a subject is eligible to participate in a specific clinical study. As another example, after receiving the prediction, stratification for the specific clinical study in which the subject has participated or will participate may be determined. For example, stratification can assign individual subjects to different treatments and / or control groups in a manner such that, without treatment, these groups have similar default predictions, map-like atrophy assessments, and / or resolved or standardized results before comparisons between groups. A clinical study may be generated based on the stratification. Clinical study results may be output. In another case, after receiving the prediction, adjustments to the specific clinical study in which the subject has participated or will participate may be determined. Adjustments may include changes to treatment, such as changes to medication, dosage, and / or time intervals between treatments. The implementation of the adjusted clinical study may be facilitated.

[0012] In some embodiments, a method is provided. The method includes detecting input data at a user device that includes or identifies a three-dimensional data object corresponding to at least a portion of a subject's eye. A request communication corresponding to a request to generate a prediction of subsequent map-like atrophy features of the subject's eye is transmitted to a remote computing system. The request communication includes the input data. In response to receiving the request communication, the remote computing system processes the three-dimensional data object using a convolutional neural network to generate a prediction of subsequent growth or subsequent size of map-like atrophy lesions in the eye. This prediction is transmitted to the user device, which receives the prediction from the remote computing system.

[0013] In some embodiments, the method may further include collecting a set of images of the subject's eyes. A three-dimensional data object can be generated using the set of images.

[0014] Some embodiments of this disclosure include the use of map-like atrophy prediction in the treatment of subjects. Map-like atrophy prediction is provided by a computational device that implements a computational model based on subject data to provide map-like atrophy prediction. The computational model includes a convolutional neural network configured to process three-dimensional data objects depicting at least a portion of the subject's eye.

[0015] In some embodiments, a method is provided. The method includes accessing a data object comprising at least three data channels. Each of the at least three data channels comprises a two-dimensional image depicting at least a portion of a subject's eye. The method further comprises processing the data object using a convolutional neural network to generate a prediction of the subsequent growth of geographic atrophy lesions in the eye or the subsequent size of geographic atrophy lesions in the eye, and outputting the prediction. The convolutional neural network may include one or more two-dimensional convolutional modules, pooling layers, and / or one or more optional units. The subsequent growth of geographic atrophy lesions may include the growth of one or more geographic atrophy lesions in the eye, and / or the subsequent size of geographic atrophy lesions may include the subsequent size of one or more geographic atrophy lesions in the eye.

[0016] In some embodiments, the data object may include multiple different OCT-based frontal images of the eye. For example, the data object may include at least two OCT-based frontal scans of the eye and at least one B-scan of the eye. As another example, the data object may include at least one OCT-based frontal scan of the eye and at least one image obtained using an imaging modality different from OCT (e.g., FAF).

[0017] In some embodiments, a convolutional neural network may be trained to simultaneously predict the current size of the map-like atrophy lesion and the subsequent map-like atrophy state. The subsequent map-like atrophy state may include subsequent map-like atrophy lesion growth or subsequent map-like atrophy lesion size.

[0018] In some embodiments, processing of data objects using a convolutional neural network can further generate another prediction of the current size of the map-like atrophic lesion in the eye.

[0019] In some embodiments, the method may further include determining aspects of a clinical study based on predictions. For example, input data may be provided by a user, including or identifying three-dimensional data objects. Predictions may be received, and based on these predictions, it may be determined whether a subject is eligible to participate in a specific clinical study. As another example, after receiving predictions, stratification for the specific clinical study in which a subject has participated or will participate may be determined. For example, stratification can assign individual subjects to different treatment and / or control groups in a manner such that, without treatment, these groups have similar default predictions for map-like atrophy assessment and / or resolution or standardization of results before comparisons between groups. Clinical studies may be generated based on stratification. Clinical study results may be output.

[0020] In some embodiments, a method is provided. The method includes detecting input data at a user device that includes or identifies a data object corresponding to at least a portion of a subject's eye. A request communication corresponding to a request to generate a prediction of subsequent map-like atrophy features of the subject's eye is transmitted to a remote computing system. The request communication includes the input data. In response to receiving the request communication, the remote computing system processes the data object using a convolutional neural network to generate a prediction of subsequent growth or subsequent size of map-like atrophy lesions in the eye. This prediction is transmitted to the user device, which receives the prediction from the remote computing system.

[0021] In some embodiments, the method may further include collecting a set of images of the subject's eyes. A data object can be generated using the set of images.

[0022] Some embodiments of this disclosure include the use of map-like atrophy prediction in the treatment of subjects. Map-like atrophy prediction is provided by a computational device that implements a computational model based on subject data to provide map-like atrophy prediction. The computational model includes a convolutional neural network configured to process data objects corresponding to at least a portion of a depiction of the subject's eye.

[0023] Some embodiments of this disclosure include a system comprising one or more data processors. The system may further include a non-transitory computer-readable storage medium containing instructions that, when executed on the one or more data processors, cause the one or more data processors to perform some or all of the methods disclosed herein.

[0024] In some embodiments, a computer program product tangibly embodied in a non-transitory machine-readable storage medium is provided. The computer program product may include instructions configured to cause one or more data processors to perform part or all of the methods disclosed herein. Attached Figure Description

[0025] This disclosure is described in conjunction with the following figures:

[0026] Figure 1A The central B-scan of the OCT volume is shown.

[0027] Figure 1B An exemplary FAF image used to quantify the grading of lesion areas is shown.

[0028] Figure 1C An example of identifying lesion areas is shown.

[0029] Figure 2 An exemplary computational network for generating predictions of the size and growth rate of map-like atrophic lesions, according to some embodiments, is shown.

[0030] Figure 3 An exemplary process is shown, according to some embodiments, of using a three-dimensional convolutional neural network to predict the size and growth of map-like atrophic lesions in a three-dimensional data object.

[0031] Figure 4 An exemplary process is shown, according to some embodiments, of using a two-dimensional convolutional neural network to predict the size and growth of map-like atrophic lesions for data objects.

[0032] Figure 5 An overview of a prediction framework with a multi-task 3D Inception convolution module is shown according to some embodiments.

[0033] Figure 6 An architecture for a 3D Inception convolution module is shown according to some embodiments.

[0034] Figure 7 An overview of a prediction framework with a multi-task 2D convolution module is shown according to some embodiments.

[0035] Figures 8A to 8D Performance details for all leaveout scenarios using the multi-task SE model are shown. (A) shows a regression plot of predicted map-like atrophic growth rate versus actual growth rate; (B) shows a regression plot of predicted map-like atrophic lesion area versus actual map-like atrophic lesion area; (C) shows OCT frontal images and central B-scans of outliers in the map-like atrophic growth rate prediction; and (D) shows OCT frontal images and central B-scans of map-like atrophic lesion area and outliers in the map-like atrophic growth rate prediction.

[0036] In the accompanying drawings, similar parts and / or features may have the same reference numerals. Furthermore, various parts of the same type can be distinguished by adding a dash after the reference numeral and a second reference numeral to differentiate similar parts. If only the first reference numeral is used in the description, the description applies to any similar parts having the same first reference numeral, regardless of the second reference numeral. Detailed Implementation

[0037] I. Overview

[0038] This disclosure describes a multi-task deep learning architecture capable of (e.g., simultaneously) predicting the current geographic atrophy (GA) lesion area and the growth rate of GA lesions. GA progression is unique to each individual in terms of both lesion area and growth rate. Currently, there are no approved treatments to prevent or slow the progression of GA.

[0039] Accurate and personalized prediction of geographic atrophy (GA) progression may help address important research and clinical questions. For example, GA progression prediction can be used to stratify participants in clinical studies with the endpoint of slowing GA progression, thereby allowing for targeted enrollment of clinical participants and improved assessment of treatment efficacy. Furthermore, personalized GA prognosis can be used to manage the disease more effectively and to understand its pathogenesis by correlating it with genotype or phenotypic traits.

[0040] Traditionally, fundus autofluorescence (FAF) images provide two-dimensional structural data about the lesion region of geographic atrophy (GA). Optical coherence tomography (OCT) images provide structural information beyond the lesion region. The additional structural information provided by high-resolution three-dimensional (3D) OCT images can offer insights into the onset and progression of geographic atrophy that cannot be discerned from FAF images alone. For example, reticular pseudodrusen, hyperreflective foci, multilayer thickness reduction, photoreceptor atrophy, and wedge-shaped subretinal hyporeflectivity detected on OCT images may be potential precursors or biomarkers associated with geographic atrophy transition. However, OCT images are difficult to process due to their 3D nature. Due to the complexity of processing 3D image volumes, current OCT-derived predictions rely on features extracted from the OCT volume rather than the complete OCT image. In short, current OCT predictions require additional feature extraction, feature engineering, or image processing steps, or some other abstraction of the baseline OCT image.

[0041] This embodiment provides a method for predicting GA directly from baseline OCT volumes without additional steps. Specifically, this embodiment includes a multi-task 3D neural network for GA lesion region detection and growth rate prediction that directly utilizes baseline 3D OCT images. In some embodiments, the deep learning architecture of the neural network may include a convolutional neural network (CNN). The deep learning architecture can be configured (as input) to receive 3D OCT image data and / or three-channel image data representing data captured at the baseline time. In some cases, the deep learning architecture is configured to generate predictions corresponding to times after the baseline time, even without receiving input collected at subsequent times. Furthermore, the network in the deep learning architecture can be trained end-to-end. This deep learning technique achieves improved statistical accuracy for map-like shrinkage compared to state-of-the-art baseline models.

[0042] One or more OCT images collected from a subject can be preprocessed. Preprocessing can be performed to, for example, flatten one or more images to compensate for the curvature of the eye. For example, the images can be flattened relative to structures such as the internal limiting membrane (ILM). Preprocessing may include aligning and / or combining multiple (e.g., flattened) images (e.g., to produce a three-dimensional image). For example, preprocessing may include converting a set of A scans (e.g., depicting multiple longitudinal scans) into B scans (e.g., to flatten B scans along a specific biological structure). In some cases, the flattened images can then be combined to produce a three-dimensional image (or flattening may be performed after combining two-dimensional images to form a three-dimensional image). Preprocessing may further include cropping and / or resizing one or more images to a predefined size and / or cropping and / or resizing by a predefined amount.

[0043] Preprocessing may include generating each of a plurality of data channels individually. The plurality of channels may include two or more channels, or three or more channels. At least one, at least two, or at least three channels may correspond to an OCT channel associated with a specific type of scan (e.g., a frontal C-scan or a B-scan). In some cases, at least two of the at least three channels may correspond to different types of scans and / or different imaging modalities. For example, at least one channel may include an image acquired using OCT, and at least another channel may include an image acquired using fundus photography, infrared imaging, or a laser scanning ophthalmoscopy (SLO).

[0044] The preprocessed image can be processed by a neural network, including, for example, a deep neural network, a two-dimensional neural network, a three-dimensional neural network, and / or a convolutional neural network. The neural network may include one or more convolutional layers (e.g., each of two or more convolutional layers, or each of all convolutional layers, has a convolutional filter whose size differs from that of another convolutional layer). The neural network may include activation layers, normalization layers, and / or pooling layers.

[0045] The neural network may have been trained and configured to predict one or more geographic atrophy lesion statistics, such as the current volume of the geographic atrophy lesion, the future volume of the geographic atrophy lesion, and / or the future growth rate. The neural network can be a multi-task model trained and configured to simultaneously predict both the growth rate and size of the geographic atrophy lesion. Compared to non-multi-task models, multi-task models are less likely to overfit on the growth rate prediction task.

[0046] Figure 1A An exemplary B-scan OCT image is shown that can be collected and processed in a deep learning workflow. Figure 1B An exemplary fundus autofluorescence (FAF) image is shown. Figure 1C Showing Figure 1B Lesion grading on FAF images. FAF images include map-like atrophic lesion areas 105.

[0047] The predicted growth rate or volume of geographic atrophy lesions can be output (e.g., presented or transmitted) along with the identification of the subject and / or eye. The predicted growth rate or volume can inform automated or manual recommendations for treatment and / or clinical study enrollment (e.g., subject stratification). For example, the predicted growth rate or volume can be used to determine (e.g., by a person or computing device) whether a subject is eligible for a particular clinical study. This determination can be made through assessment rules indicating that eligibility requirements are a predicted growth rate or volume exceeding a predetermined lower threshold and / or falling below a predetermined upper threshold. As another example, the predicted growth rate or volume can be used for clinical study stratification (e.g., assigning individual subjects to different treatment and / or control groups such that, if no treatment is administered, these groups have similar default predicted geographic atrophy assessments and / or resolving or standardizing results before comparisons between groups).

[0048] II. Definition

[0049] As used herein, the term "multi-task model" refers to a model that includes one or more neural networks trained to simultaneously predict multiple statistical variables (e.g., current geographic atrophy lesion size and future geographic atrophy lesion growth rate). Multi-task models may include one or more neural networks. Multi-task models avoid or reduce the likelihood of overfitting because they are trained to fit two sets of information simultaneously.

[0050] As used herein, the term "A-scan" refers to a one-dimensional image of a depth profile along the direction of incident light from the eye. Near-infrared light can be guided in a linear direction to generate data at multiple A-scan locations.

[0051] As used herein, the term "B-scan" refers to a cross-sectional view of the eye. In some cases, a B-scan is generated in response to a specific type of measurement. For example, near-infrared light can be transmitted into the eye, and low-coherence interference signals can generate a two-dimensional B-scan image. In some cases, a B-scan is a frame composed of an array of A-scans. B-scans can have a lateral, longitudinal, or axial orientation, depending on how the probe is oriented.

[0052] As used herein, the term "frontal C-scan" refers to a lateral image of a layer of the eye at a specified depth. For example, a frontal C-scan could be a lateral image of the retina or choroid of the eye. A C-scan can be obtained by rapid lateral scanning and slow axial scanning at a constant depth. In some cases, a frontal C-scan is generated from a set of B-scans.

[0053] As used herein, the term "growth rate" refers to the change in size of a geographic atrophic lesion over time. This change in size can be absolute or relative. For example, the growth rate can be the change in volume or area of ​​geographic atrophy in the eye. In two dimensions (e.g., area), the growth rate can be expressed as a unit area (e.g., mm) per unit time. 2 The growth rate can be measured per unit volume per unit time (e.g., per year) or as a ratio or percentage (e.g., area at a later time point relative to area at the baseline time point). In three dimensions (e.g., volume), the growth rate can be expressed as a unit volume per unit time (e.g., mm²). 3 The volume is measured as a percentage or ratio (e.g., the volume at a later time point relative to the volume at the baseline time point) or per year.

[0054] As used herein, the term "map-like atrophy size" refers to the measurable size of one or more map-like atrophy lesions in the eye. For example, in two dimensions, map-like atrophy size can be the area of ​​the map-like atrophy lesion (e.g., in mm). 2 (Unit: )

[0055] As used in this article, the term "current" refers to the baseline time. For example, the current size of geographic atrophy lesions could be the size of geographic atrophy lesions at the time when the image data of the eye was generated.

[0056] As used herein, the term "progression" refers to a measurable difference from baseline time to a future time. For example, map-like atrophy progression can be the growth of map-like atrophy from baseline time to a future time. In some cases, map-like atrophy progression can be determined from the growth rate.

[0057] III. Example Computational Network

[0058] Figure 2 An exemplary personalized GA prediction network 200 for generating predictions of the growth rate of map-like atrophy lesions, according to some embodiments, is illustrated. A GA progression controller 205 may be configured to train and execute a machine learning model to generate predictions related to map-like atrophy status. More specifically, the GA progression controller 205 may be configured to receive one or more images of the eye and output a predicted growth rate or volume of the map-like atrophy lesion.

[0059] Predictions can be generated using one or more machine learning models, such as one or more CNNs. CNNs may include 3D CNNs and / or 2D CNNs. In some cases, machine learning models may include ensemble machine learning models that process the aggregated results from 3D networks and 2D neural networks.

[0060] The GA progress controller 205 may include an OCT input controller 220 that receives and / or retrieves input data comprising a set of input data objects. The input data objects may include data objects used in a training dataset to train a machine learning model and / or data objects processed by the trained machine learning model.

[0061] Each input data object may correspond to a specific object, a specific eye, and / or specific imaging data. An input data object may include, for example, an image of at least a portion of the eye. An input data object may include images generated using imaging techniques disclosed herein, such as OCT and / or fundus photography. For example, an input data object may include a set of A-scan images (e.g., each image depicting a different longitudinal scan of a specific eye), a set of B-scan images (e.g., each image depicting a flattened image), or a set of C-scan images (e.g., each image corresponding to a specific depth).

[0062] Input data objects can be received and / or retrieved from one or more imaging systems 225, which may include an OCT device and / or a fundus camera. In some cases, the imaging system 225 sends one or more images to an image data storage 230, which may be partially or fully accessed by the GA progression controller 205.

[0063] The input data object may be preprocessed by the preprocessing controller 235. Preprocessing may include performing normalization and / or standardization. For example, preprocessing may include histogram matching of scans or channels across a data object (e.g., a set of B scans) to unify the intensity of B scans or multiple data objects in the dataset. Based on the average intensity level of the B scans, the preprocessing controller 235 may select a B scan from a set of B scans as a reference B scan. The preprocessing controller 235 may perform histogram matching on the remaining B scans in the B scan set to normalize the intensity.

[0064] The preprocessing controller 235 may also perform segmentation (e.g., BM segmentation or ILM segmentation) on each B-scan during preprocessing. After segmentation, each B-scan may be flattened along a description of a specific biological structure (e.g., Bruch's membrane (BM) boundary or internal limiting membrane (ILM) boundary). This segmentation facilitates volumetric truncation (e.g., removing upper and lower background portions to reduce neural network constraints) and generates a frontal image. After flattening, the preprocessing controller 235 may crop each B-scan to a pixel region around the boundary. For example, a flattened B-scan may be cropped to a region 5 pixels above and 250 pixels below the ILM, or a region 120 pixels above and 135 pixels below the BM. The cropped region may be adjusted to a predefined size (e.g., 512 pixels by 512 pixels) using, for example, resampling, interpolation, and / or bilinear interpolation.

[0065] The personalized GA prediction network 200 may include a lesion label detector 240 that retrieves one or more labels associated with each input data object in the training dataset. The labels may have been initially identified based on information provided by annotators, medical professionals, and / or validated databases. Labels may include the size of the geographic atrophic lesion at subsequent times, the cumulative size of the geographic atrophic lesion at subsequent times, the difference between baseline and subsequent times, the progression of the size of the geographic atrophic lesion, and / or the progression of the cumulative size of the geographic atrophic lesion.

[0066] The model training controller 245 executes code to train one or more machine learning models using one or more training datasets. The machine learning models may include one or more preprocessing functions, one or more neural networks, and / or one or more post-processing functions. It should be understood that one or more variables may be fixed during training with respect to each function, algorithm, and / or network in the model. For example, the hyperparameters of the neural network (e.g., the number of recognition layers, the size of the input layer, the learning rate, etc.) may be predefined and not adjustable during code execution. It should be further understood that one or more variables may be learned through training with respect to each function, algorithm, and / or network in the model. For example, the parameters of the neural network (e.g., parameters that recognize weights between various nodes) may be learned.

[0067] Each training dataset may include a set of training data objects. Each data object may include an input image (e.g., an image depicting a portion of an eye or its flattened version). The input image may include an image collected at baseline time.

[0068] Each data object in the object data object may be further associated with one or more labels (e.g., two or more labels). Each label may be based on information collected at least in part over a period of time after the baseline time (e.g., at least two weeks, at least one month, at least two months, at least six months, at least one year, at least two years, at least five years, or at least ten years from the baseline time). In some cases, one of the labels may include the size of a given geographic atrophic lesion at a subsequent time, the cumulative size of the geographic atrophic lesion at a subsequent time, the difference between the baseline time and the subsequent time, the progression of the size of a given geographic atrophic lesion, and / or the progression of the cumulative size of the lesion. Thus, the training dataset may include images at the baseline time and labeled data indicating how and / or whether the subject's geographic atrophy progressed over a period of time.

[0069] The model training controller 245 can use training data to train a machine learning model. In addition to one or more traditional machine learning models (e.g., neural networks, group-based classifiers, regression algorithms, etc.), the machine learning model may further include one or more preprocessing functions and / or more than one post-processing function. For example, preprocessing functions may adjust the size, intensity distribution, and / or perspective of one or more input images; and / or post-processing functions may convert the model output into values ​​along a predefined scale (or classification list), suggestions, hierarchical identifiers, etc.

[0070] The model training controller 245 can train various parts of a machine learning model individually or together. For example, the training dataset can be used to train one or more preprocessing algorithms (e.g., algorithms that perform segmentation) and neural networks simultaneously. As another example, the preprocessing algorithm can be trained separately from the neural network (e.g., using the same or different parts of the training data).

[0071] The GA prediction generator 250 can use architecture and learned parameters to process non-training data and generate results. Results can be generated in response to request communication from client device 210 (e.g., a device that includes or identifies the input data). For example, the request may identify an identifier of the database, the subject, the eyes, and an access password in order to utilize a set of images of the subject's eyes. The request communication may correspond to a request to generate a prediction of the subject's eyes followed by map-like atrophy features (e.g., size or growth). The topic to which the request pertains may differ from every topic represented in the training data.

[0072] In some cases, the request includes a data object configured to be input into a machine learning model. In some cases, the request includes a data object configured to undergo preliminary processing (e.g., decryption, normalization, etc.) before being configured to be input into a machine learning model. In some cases, the request includes information (e.g., one or more identifiers, CAPTCHAs, and / or passwords) that can be used to retrieve the data object configured to be input into a machine learning model.

[0073] In some cases, post-processing functions handle predictions when used with, for example, models, functions, or rules. For instance, a classifier might be used to assign severity predictions to a subject-eye input dataset based on preliminary results. As another example, rules could be used to determine whether to recommend treatment changes based on preliminary results.

[0074] The GA prediction generator 250 thus has access to the input data object and feeds the data object into a machine learning model. The results of the machine learning model can predict the current size of the geographic atrophy lesion (e.g., at the time the data object is generated) and / or the subsequent status of the geographic atrophy lesion (e.g., subsequent growth or size of the geographic atrophy lesion). The GA progression controller 205 can output predictions characterizing the subsequent growth of geographic atrophy in the eye (e.g., growth of one or more geographic atrophy lesions in the eye) and / or the subsequent size of geographic atrophy in the eye (e.g., the subsequent size of one or more geographic atrophy lesions in the eye). For example, the prediction may include the probability or likelihood of subsequent growth or subsequent size of geographic atrophy in the subject's eye in a subsequent time period; the predicted absolute or relative growth; and / or the predicted relative or absolute size of the geographic atrophy lesion. The GA progression controller 205 can additionally generate a prediction of the current size (e.g., area or volume) of one or more geographic atrophy lesions in the eye.

[0075] In some cases, the GA prediction generator 250 may execute a three-dimensional convolutional neural network (e.g., a three-dimensional convolutional neural network trained by the model training controller 245). The three-dimensional convolutional neural network may be configured to receive and process three-dimensional data objects (e.g., three-dimensional data objects corresponding to a set of flattened B-scans). The three-dimensional convolutional neural network may include one or more three-dimensional convolutional modules (e.g., an Inception convolutional module). Each three-dimensional convolutional module may include a set of convolutional layers that use at least two convolutional filters of different sizes (e.g., using a 1x1x1 convolutional filter, a first 3x3x3 convolutional filter, and a second 3x3x3 convolutional filter). The three-dimensional convolutional network may alternatively or additionally include one or more three-dimensional convolutional layers.

[0076] In some cases, the GA prediction generator 250 can execute two-dimensional and / or multi-channel neural networks. For example, the data object may include multi-channel image data. Each of the image channels may include, for example, two-dimensional data corresponding to one or more imaging techniques and / or one or more imaging perspectives. For example, the two-dimensional image data may include B-scans and / or one or more C-scans. Thus, in some cases, the multi-channel image data includes data corresponding to two or more different types of image collections and / or two or more different types of perspective. Multi-channel image data can be processed by one or more neural networks to generate predictions of the growth rate and / or subsequent size of the map-like atrophic lesion.

[0077] Each of one or more channels in the data channel may include a two-dimensional image corresponding to at least a partial depiction of the subject's eye. In some examples, three-channel data may include multiple frontal C-scan OCT images. As another example, three-channel data may include at least two frontal C-scan OCT images and at least one B-scan image (e.g., a central B-scan or one or more B-scans corresponding to a pseudo-randomly selected lateral position). As yet another example, three-channel data may include at least one frontal C-scan OCT image and at least one image obtained using an imaging modality other than OCT (e.g., infrared and / or fundus autofluorescence imaging or cSLO). Three-channel data may alternatively include a full-depth frontal OCT image, a sub-BM frontal OCT image (e.g., 100 pixels below the BM), and a BM-above frontal OCT image (e.g., 100 pixels above the BM). It should be understood that while some disclosures herein refer to "three-channel" data, object data, etc., data comprising more than three channels is contemplated as an alternative.

[0078] The neural network used by the GA prediction generator 250 may include a transfer learning neural network (Inception V3) and / or subsequently a dense layer. For example, the neural network may be initialized with parameters for predicting the growth rate and / or subsequent size of different types of lesions in the eye (e.g., pineal gland, conjunctival lesions, or choroidal melanoma). The neural network can then be further trained to predict the growth rate and / or subsequent size of map-like atrophic lesions in the eye.

[0079] The network may include batch normalization, one or more activation layers (e.g., using ReLU activation), and / or one or more pooling layers (e.g., using max pooling). For example, the batch normalization, ReLU activation, and / or pooling layers may follow each of one or more convolutional layers. The neural network may include (e.g., in the output layer and / or in the layer after the last convolutional layer) global average pooling and / or dense layers. The neural network may include activation functions (e.g., with linear activation). The neural network may include an attention unit, such as the standard Squeeze and Excitation (SE) attention unit. The SE attention unit may include global average pooling of feature maps from the convolutional layers and apply one or more linear transformations to the pooling result. The attention unit may be configured to adaptively adjust the weights of each feature map.

[0080] In some cases, the GA predictor generator 250 may use an ensemble model to process the data object. The ensemble model may include multiple neural networks, such as one or more three-dimensional convolutional neural networks and one or more two-dimensional convolutional neural networks. The ensemble model processes the data object and aggregates the results from each neural network. For example, the ensemble model may average the predicted subsequent size and / or predicted growth rate of the map atrophy lesion from each neural network. The ensemble model may output the average of the predicted subsequent size and / or the subsequent growth of the map atrophy in the subject's eye.

[0081] The GA analysis controller 255 can process predictions and transmit the results (or a processed version thereof) to client device 210 or other systems (e.g., associated with laboratory technicians or care providers). For example, the GA analysis controller 255 can generate outputs identifying the subsequent size of map-like atrophy or the subsequent growth of map-like atrophy in a subject's eye. In some cases, the GA analysis controller 255 can further post-process the results. For example, the GA analysis controller 255 can evaluate one or more rules to determine whether any of the conditions for one or more rules are met. For example, a rule can be configured to be met when the predicted lesion size and / or predicted progression is large enough to recommend a change in treatment. As another example, a rule can be configured to be met when the predicted lesion size and / or predicted progression is within a predefined open or closed range to recommend a specific type of treatment. As yet another example, rules can be configured based on the predicted lesion size and / or predicted progression to indicate whether a subject is eligible for a clinical trial.

[0082] The output can then be presented and / or transmitted, which may facilitate the display of the output data, such as on a computing device (e.g., client device 210).

[0083] IV. Map-like atrophic lesion area and growth rate prediction technology

[0084] Figure 3An exemplary process is illustrated using a convolutional neural network, according to some embodiments, to predict the map-like shrinkage size and growth of a three-dimensional data object. At block 305, a three-dimensional data object corresponding to at least a portion of a subject's eye is generated. The three-dimensional data object can be generated by segmenting each image in a set of two-dimensional images (e.g., B-scans) to identify segments within the images (e.g., corresponding to Bruch's membrane and / or the internal limiting membrane). Alternatively, the three-dimensional data object can be generated by accessing two-dimensional B-scan OCT images of the subject's eye, using feature maps generated from the two-dimensional B-scan OCT images, and generating the three-dimensional data object to include a set of feature maps. As another example, the three-dimensional data object can be generated by accessing a set of two-dimensional B-scan OCT images of the subject's eye. A set of pixels depicting specific structures of the eye can be used to identify each image in the set of two-dimensional B-scan OCT images. For example, a set of pixels depicting Bruch's membrane and / or the internal limiting membrane can be identified. Each image in the two-dimensional B-scan OCT images can be flattened based on the pixel set. The three-dimensional data object can then be generated to include at least a portion of each flattened B-scan OCT image.

[0085] In block 310, the 3D data object is accessed. The 3D data object can be accessed from local memory or storage. In some cases where the 3D data object is generated at an imaging or remote system, it can be accessed from the imaging or remote system (e.g., Figure 2 The imaging system 225 in the image accesses the three-dimensional data object. A remote computing system (e.g., from a client system) that receives a request communication accesses the three-dimensional data object to generate a predicted map-like atrophy feature of the subject's eye.

[0086] In block 315, a convolutional neural network is used to process 3D data objects to generate predictions of subsequent growth and / or subsequent size of map-like atrophic lesions in the eye. The convolutional neural network can be a 3D neural network having one or more convolutional modules (e.g., an Inception module) and / or one or more pooling layers. Each convolutional module may include a 3D convolutional module comprising a set of convolutional layers. At least two of these convolutional layers may include convolutional filters of different sizes. For example, the first layer may use a 1x1x1 convolutional filter, and each of one or more second layers may use a 3x3x3 convolutional filter. The convolutional neural network may further include an attention unit, such as a standard SE attention unit, which adds parameters to each channel of the convolutional block to facilitate adaptive adjustment of the weights of each feature map.

[0087] In block 320, the output prediction is provided. This prediction can be used to help determine a subject's eligibility to participate in a specific clinical study. For example, a prediction of the size of geographic atrophy above a threshold could indicate a subject's eligibility for a specific clinical study. The prediction can additionally or alternatively stratify the specific clinical studies in which a subject has participated or will participate. For example, the prediction can be used to assign subjects to different treatment and / or control groups such that, if no treatment is administered, these groups have similar default predictions for geographic atrophy assessment and / or to resolve or standardize results before comparisons between groups. Clinical study results can be generated based on stratification. Additional clinical study results can be output.

[0088] In some cases, predictions can be used to help select treatment for a subject and / or help determine whether to change a subject's treatment. For example, a machine learning model can be trained using a training dataset corresponding to a group of subjects receiving or having received a specific treatment. Baseline images collected for each subject may correspond to a time period when treatment has just begun or is about to begin. If a subject uses a specific treatment, the trained model can then be used to predict the extent to which geographic atrophy will progress and / or the subsequent size of the geographic atrophy lesions, thereby informing the care provider whether to recommend the specific treatment (e.g., compared to another treatment).

[0089] Figure 4 An exemplary process is shown, according to some embodiments, of using a convolutional neural network to predict the size and growth of map-like atrophic lesions for a data object. At block 405, access is made to a data object comprising at least three data channels. Each of the at least three data channels may include a two-dimensional image depicting at least a portion of the subject's eye. The data object may include multiple different OCT-based frontal images of the eye. In some cases, the data object may include at least two OCT-based frontal scans of the eye and at least one B-scan of the eye (e.g., a central B-scan or one or more B-scans corresponding to a pseudo-randomly selected lateral location). Alternatively, the data object may include at least one OCT-based frontal scan of the eye and at least one image obtained using an imaging modality different from OCT.

[0090] In block 410, a convolutional neural network (CNN) is used to process the data object to generate predictions of the subsequent growth and / or subsequent size of a geographic atrophic lesion in the eye. The CNN may further or alternatively generate another prediction of the current size of the geographic atrophic lesion in the eye. The CNN may be a two-dimensional neural network trained to simultaneously predict the current size and subsequent state of the geographic atrophic lesion, which may include subsequent geographic atrophic lesion growth or subsequent size. Transfer learning may be used to train the CNN. For example, the CNN may be initialized with parameters for predicting the growth rate and / or subsequent size of different types of lesions in the eye (e.g., conjunctival macular degeneration, conjunctival lesions, or choroidal melanoma). The neural network can then be further trained to predict the growth rate and / or subsequent size of a geographic atrophic lesion in the eye. The CNN may include one or more two-dimensional convolutional modules and / or pooling layers. The CNN may further include one or more optional units (e.g., SE attention units). The CNN may also include batch normalization, one or more activation layers (e.g., using ReLU activation), and / or one or more pooling layers (e.g., using max pooling).

[0091] In block 415, output predictions (e.g., predictions of subsequent growth, current size, and / or subsequent size). These predictions can be used to help determine a subject's eligibility for a specific clinical study. For example, a size prediction of a geographic atrophic lesion exceeding a threshold could indicate a subject's eligibility for a specific clinical study. Predictions may additionally or alternatively stratify the specific clinical studies a subject has participated in or will participate in. Clinical study results can be generated based on stratification. Additional clinical study results may be output.

[0092] In some cases, predictions can be used to help select treatment for a subject and / or help determine whether to change a subject's treatment. For example, a machine learning model can be trained using a training dataset corresponding to a group of subjects receiving or having received a specific treatment. Baseline images collected for each subject may correspond to a time period when treatment has just begun or is about to begin. If a subject uses a specific treatment, the trained model can then be used to predict the extent to which geographic atrophy will progress and / or the subsequent size of the geographic atrophy lesions, thereby informing the care provider whether to recommend the specific treatment (e.g., compared to another treatment).

[0093] V. Example

[0094] VA Example 1: Predicting the growth rate of map-like atrophy by processing 3D OCT images using neural networks

[0095] VA1. Method

[0096] This study retrospectively examined the eyes of subjects with bilateral geographic atrophy enrolled in the phase 3 trial of lanparibizumab SPECTRI (NCT02247531). The eyes measured 1.92 x 6 x 6 mm. 3 A 496x1024x49 voxel macular Spectralis SD-OCT volume (Heidelberg Engineering, Inc., Heidelberg, Germany) was used to predict the area and growth of map-like atrophic lesions. The map-like atrophic lesion area (mm) was derived using all available access measurements. 2 The growth rate of map-like atrophic lesions (measured in mm) from FAF images scored by two readers or, if necessary, by a judge. 2 (Per year) and fitted a linear model. The image dataset from 522 subjects was divided into 5 groups per subject. Five-fold cross-validation was performed, and all folds were balanced against baseline factors and the treatment group. Each B-scan was flattened along the ILM and reduced to a size of 128x128 pixels. A multi-task 3D convolutional neural network (CNN) model was then trained to predict the growth rate of map-like atrophy from a baseline OCT volume with initialized weights from the same model trained to predict map-like atrophy regions. The in-sample coefficient of determination (R²) was calculated. 2 The coefficient is used to evaluate performance, and is defined as the square of the Pearson correlation coefficient (r) between the actual and predicted map-like shrinkage growth rates.

[0097] VA2. Results

[0098] Map-like atrophic lesion area prediction showed a mean cross-validation R of 0.88 (range: 0.77 to 0.92). 2 Map-like shrinkage growth rate prediction has a mean cross-validation R-value of 0.30 (range: 0.21 to 0.43). 2 The growth rate prediction results are comparable to previous work that predicted map-like atrophy growth rate using deep learning with baseline multimodal retinal images (R2 = 0.35) (Normand G, et al. IOVS 2019; 60: ARVO E-Abstract 1452).

[0099] This example demonstrates the feasibility of using OCT images with a 3D deep learning approach to predict the growth rate of individual map-like atrophy. Predictions can be improved by incorporating other imaging modalities (e.g., infrared and fundus autofluorescence) and / or relevant clinical variables.

[0100] VB Example 2: Predicting Geographic Shrinkage Growth Rate by Processing OCT Images Using Neural Networks

[0101] VB1. Three-dimensional modeling method

[0102] The prediction is formulated as a multi-task regression problem. Let... The training dataset consists of Xi,yi, which represents the i-th OCT volume and its corresponding map-like atrophic lesion region (y1) and growth rate (y2) labels; N represents the total number of samples. The model takes the baseline OCT volume as input and predicts the current map-like atrophic lesion region and lesion growth rate.

[0103] Figure 5 The paper presents an overview of the method. First, the 3D OCT volume is preprocessed and then fed into a model based on a multi-task 3D Inception CNN. Furthermore, a self-attention mechanism using Squeeze and Excitation (SE) units is added to enhance the feature map. Simultaneous prediction of map-like atrophic lesion regions and growth rates is achieved. Map-like atrophic lesion growth rates have been shown to be weakly correlated with map-like atrophic lesion regions. The multi-task model promises to find representations that capture information from both tasks and exhibits a lower likelihood of overfitting only on the growth rate prediction task.

[0104] VB1.a.OCT Image Preprocessing

[0105] The OCT volume comprises 496 depth pixels, 1024 A scans, and 49 B scans (496 x 1024 x 49 voxels). The model input volume is reduced in size and focused on the retina (target region). For each B scan, the image is flattened along the internal limiting membrane (ILM). The ILM is the boundary between the retina and the vitreous body, pre-segmented by the OCT machine software (Heidelberg Engineering, Inc., Heidelberg, Germany). Then, for each B scan, a region 5 pixels above and 250 pixels below the ILM is cropped and defined as the target region. Each cropped region (256 depth pixels × 1024 A scans) is then adjusted to 128 x 128 pixels using bilinear interpolation. Figure 5 The number of B scans was not reduced. Therefore, the original volume was reduced to 128×128×49 voxels as the model input.

[0106] VB1.b. 3D Inception Convolution Module

[0107] Figure 6The details of the 3D Inception convolutional module are shown. The network consists of one regular convolutional layer and four 3D Inception convolutional blocks. The regular convolutional layer and the first three 3D Inception convolutional blocks are shared by region and growth rate predictions. Each convolutional layer is followed by batch normalization, ReLU activation, and max pooling. After the last convolutional block, global average pooling and a dense layer with linear activation are used for the output layer. The anisotropic properties of the 3D volume are considered when choosing the filter size, stride, and pooling size. Hyperparameters are optimized using a random search.

[0108] VB1.c.SE Attention Unit and Joint Loss

[0109] The two-dimensional SE attention units are generalized to three dimensions and added to three three-dimensional Inception modules. Similar to the two-dimensional case, feature maps from the Inception modules are reweighted by the outputs of the SE units to account for inter-channel dependencies. More specifically, the output feature maps from the three-dimensional Inception modules (H×W×D×C) are transformed into a global tensor of dimension 1×1×1×C by 3D spatial averaging. This tensor is essentially a vector in the channel direction and has the same dimension as the number of channels. Fully connected layers with ReLU activations are applied in the channel directions to reduce the output tensor to a dimension of 1×1×1×C / r, where r is called the reduction ratio and is the divisor of C and the number of channels. In this study, the reduction ratio is set to 4. A second fully connected layer is followed by a sigmoid activation with an output size of 1×1×1×C, providing smooth gating for each channel. Finally, each feature map of the convolutional block is weighted by multiplying by the output from the SE units. Finally, training is performed using a joint loss of the mean squared error of map-like atrophic lesion region prediction and growth rate prediction. Each term has a weight of 0.5.

[0110] VB1.d. Baseline Model

[0111] Three additional baseline models were tested to compare performance and gain deeper insights into the models. The first baseline model (i.e., the base model) is a simple 3D Inception convolutional neural network to predict only the growth rate of map-like atrophy. This model is identical to the proposed model without the SE attention unit and in the multi-task setting. Model parameters were initialized using He initialization. The second baseline model (i.e., the cascaded model) uses a cascaded approach, where the model is first trained to predict map-like atrophy lesion regions. The model is then initialized with weights learned during region prediction training and trained again to predict map-like atrophy growth rates. The cascaded model trained for each type of prediction shares the same 3D convolutional architecture as the first baseline model. The third baseline model (i.e., the multi-task model) is a multi-task model without the SE attention unit to see if the SE attention block provides any additional performance gain. The second and third baseline models were trained to predict both current and future map-like atrophy lesion regions.

[0112] VB2. Two-dimensional model method

[0113] Figure 7 The image shows an overview of a multi-task 2D convolutional network 700. This model includes a transfer learning Inception V3 neural network 705. A dense layer 710 follows the transfer learning Inception V3 network 705. It simultaneously predicts map-like atrophic lesion regions 715 and growth rates 720. This 2D multi-task model aims to find representations that capture information for both tasks.

[0114] VB3. Results

[0115] The model was run on the complete training dataset of patients with bilateral geographic atrophy enrolled in the Lampalizumab phase 3 SPECTRI (NCT02247531) trial and evaluated five times on the holdout dataset. 560 subjects performed 1934 visits on the training dataset, and 114 subjects performed 114 baseline visits on the holdout dataset. The mean performance and standard deviation on the holdout dataset are shown in Table 1. The results indicate that the proposed model has the best performance in predicting the growth rate of geographic atrophy. The multi-task framework contributed most of the improvement compared to the first baseline model. The SE attention block did not appear to significantly improve performance. The cascaded model also performed better than the first baseline model. In contrast, the performance in predicting the lesion area of ​​geographic atrophy was comparable among all models.

[0116] Figure 8A and Figure 8BExample regression plots are shown, illustrating the relationship between predicted map-like atrophy growth rates and actual growth rates across all hold-out scenarios from additional runs of the model, as well as the relationship between predicted map-like atrophy lesion areas and actual map-like atrophy lesion areas. The squared Pearson correlation coefficient (R²) between the predicted and actual lesion areas is also presented. 2 The growth rate prediction performance was 0.92. It outperformed larger lesion areas in cases with relatively small geographic atrophy lesions. The growth rate prediction performance was compared with deep learning prediction of geographic atrophy growth rate (Rg) using baseline FAF and infrared retinal images from a larger population (over 2000 eyes). 2 =0.43) is comparable to previous work. Figure 8C The image shows an OCT frontal image and a central B-scan of the abnormal values ​​in the map-like atrophic growth rate prediction. It can be seen that the OCT volume exhibits motion artifacts and pixel intensity inconsistent with the B-scan. Furthermore, the OCT volume size in this case is 496 × 512 × 49 voxels; only about 5% of the training data has this resolution. However, the predicted map-like atrophic lesion area is consistent with the true value. Figure 8D The image shows an OCT frontal image and a central B-scan, revealing another outlier in the map-like atrophic lesion area and growth rate prediction. In this case, image quality may again be the root cause, as multiple B-scans show rather low image intensity, i.e., dark bands in the OCT frontal image.

[0117] Table 1. Performance comparison of the four models on the holdout dataset

[0118]

[0119] VB4. Explanation

[0120] Therefore, the deep learning model was successfully trained to process OCT volumes to predict the progression of geographic atrophy disease without any prior feature engineering. A novel custom-designed 3D multi-task deep learning pipeline simultaneously and accurately predicts both the current geographic atrophy lesion region and the future growth rate of geographic atrophy lesions based solely on baseline OCT volumes. The proposed model outperforms other baseline models.

[0121] VI. Additional Considerations

[0122] Some embodiments of this disclosure include a system comprising one or more data processors. In some embodiments, the system includes a non-transitory computer-readable storage medium containing instructions that, when executed on the one or more data processors, cause the one or more data processors to perform part or all of one or more methods and / or part or all of one or more processes disclosed herein. Some embodiments of this disclosure include a computer program product tangibly embodied in a non-transitory machine-readable storage medium, comprising instructions configured to cause one or more data processors to perform part or all of one or more methods and / or part or all of one or more processes disclosed herein.

[0123] The terms and expressions used are descriptive rather than restrictive, and their use is not intended to exclude any equivalents of the features or portions thereof shown and described, but it should be recognized that various modifications can be made within the scope of the claimed invention. Therefore, it should be understood that although the claimed invention has been specifically disclosed by way of examples and optional features, modifications and variations of the concepts disclosed herein can be made by those skilled in the art, and such modifications and variations are considered to be within the scope of the invention as defined in the appended claims.

[0124] This description provides only preferred exemplary embodiments and is not intended to limit the scope, applicability, or configuration of this disclosure. Rather, this description of preferred exemplary embodiments will provide those skilled in the art with a feasible description for implementing various embodiments. It should be understood that various changes can be made to the function and arrangement of the elements without departing from the spirit and scope set forth in the appended claims.

[0125] Specific details are set forth in this description to provide a thorough understanding of the embodiments. However, it should be understood that the embodiments may be practiced without these specific details. For example, circuits, systems, networks, processes, and other components may be shown as parts in block diagram form to avoid obscuring the embodiments with unnecessary details. In other instances, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary details to avoid obscuring the embodiments.

Claims

1. A method for predicting the growth or size of geographic atrophy lesions in the eye of a subject, comprising: Access a three-dimensional data object that corresponds to a depiction of at least a portion of the subject's eye; The three-dimensional data object is processed using a convolutional neural network trained to generate predictions of the growth of geographic atrophy lesions in the eye or the size of geographic atrophy lesions in the eye, wherein the convolutional neural network is trained in a multi-task setting to simultaneously predict the current size of the geographic atrophy lesion and the subsequent geographic atrophy state, the subsequent geographic atrophy state including subsequent geographic atrophy lesion growth or subsequent geographic atrophy lesion size; and The convolutional neural network outputs a prediction of the growth or size of the map-like atrophic lesion.

2. The method of claim 1, wherein the convolutional neural network comprises one or more three-dimensional convolutional modules.

3. The method according to claim 1 or 2, wherein the convolutional neural network includes a pooling layer.

4. The method according to any one of claims 1 to 2, wherein the convolutional neural network is used to process the three-dimensional data object to further generate another prediction of the current size of the map-like atrophic lesion in the eye.

5. The method according to any one of claims 1 to 2, wherein the growth of the geographic atrophic lesion includes the growth of one or more geographic atrophic lesions in the eye, and wherein the size of the geographic atrophic lesion includes the size of one or more geographic atrophic lesions in the eye.

6. The method according to any one of claims 1 to 2, further comprising generating the three-dimensional data object by performing a set of operations, the set of operations including segmenting each of a set of two-dimensional images to identify segments defined by predicted locations of the Bruch membrane and / or the internal limiting membrane.

7. The method according to any one of claims 1 to 2, further comprising generating the three-dimensional data object by performing a set of operations, said set of operations including: Access a set of two-dimensional B-scan OCT images of the subject's eyes; A set of feature maps is generated using the aforementioned set of two-dimensional B-scan OCT images; as well as The three-dimensional data object is generated to include the set of feature maps.

8. The method according to any one of claims 1 to 2, further comprising generating the three-dimensional data object by performing a set of operations, said set of operations including: Access a set of two-dimensional B-scan OCT images of the subject's eyes; For each image in the set of two-dimensional B-scan OCT images, identify a set of pixels that depict a specific structure of the eye; For each image in the set of two-dimensional B-scan OCT images, the two-dimensional B-scan OCT image is flattened based on the set of pixels; as well as The three-dimensional data object is generated to include at least a portion of each flattened B-scan OCT image.

9. The method according to any one of claims 1 to 2, further comprising: Use the training dataset to train another convolutional neural network to produce the first set of learned parameter values; and Transfer learning is used to train the convolutional neural network as follows: Set a set of parameters used in the convolutional neural network to the parameter values ​​learned in the first set; and The convolutional neural network is further trained using another training dataset to generate a second set of learned parameter values, wherein the convolutional neural network is configured with the second set of learned parameter values ​​when processing the 3D data object.

10. The method according to any one of claims 1 to 2, wherein the three-dimensional data object comprises a three-dimensional depiction of the volume of the subject's eye.

11. The method according to any one of claims 1 to 2, further comprising: Input data provided by the user that includes or identifies the three-dimensional data object; Receive the prediction; as well as Based on the predictions, the subjects were determined to be eligible to participate in specific clinical studies.

12. The method according to any one of claims 1 to 2, further comprising: Input data provided by the user that includes or identifies the three-dimensional data object; Receive the prediction; as well as Based on the predictions, stratification is determined for specific clinical studies in which the subjects have participated or will participate.

13. The method of claim 12, further comprising: Clinical research results were generated based on the aforementioned stratification; as well as Output the results of the aforementioned clinical study.

14. The method according to any one of claims 1 to 2, further comprising: Input data provided by the user that includes or identifies the three-dimensional data object; Receive the prediction; Based on the predictions, adjustments are determined for specific clinical studies in which the subjects have participated or will participate; and Facilitate the implementation of adjusted specific clinical studies.

15. A system comprising: One or more data processors; as well as A non-transitory computer-readable storage medium comprising instructions that, when executed on the one or more data processors, cause the one or more data processors to perform the method of any one of claims 1-14.

16. A computer program product tangibly embodied in a non-transitory machine-readable storage medium, comprising instructions configured to cause one or more data processors to perform the method of any one of claims 1-14.

Images