A deep neural network framework for processing OCT images to predict treatment intensity

Abstract

Systems and methods relate to processing optical coherence tomography (OCT) images to predict characteristics of a treatment to be administered to effectively treat age-related macular degeneration. The processing may include preprocessing the images by flattening and / or cropping the images and processing the preprocessed images using a neural network. The neural network may include a deep convolutional neural network. The output of the neural network may indicate a predicted frequency and / or interval at which a treatment (e.g., anti-vascular endothelial growth factor therapy) will be administered to prevent leakage of the vascular system in the eye.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of and priority to U.S. Provisional Patent Application No. 62 / 944,815, filed December 6, 2019, and U.S. Provisional Patent Application No. 63 / 017,898, filed April 30, 2020, each of which is incorporated herein by reference in its entirety for all purposes. [Background technology]

[0002] background Age-related macular degeneration (AMD) is the leading cause of vision loss in people over the age of 60. For most individuals, AMD first manifests as dry AMD and progresses to wet AMD. In the dry form, small deposits (drusen) form on the retina under the macula, causing retinal deterioration over time. In the wet form, abnormal blood vessels grow toward the macula. These blood vessels frequently break down and leak fluid, which can cause the macula to separate from its base, resulting in severe and rapid vision loss.

[0003] Anti-vascular endothelial growth factor (aVEGF) agents are frequently used to treat wet AMD. Specifically, aVEGF agents can dry the subject's retina, thereby better controlling the subject's wet AMD and reducing or preventing permanent vision loss. However, aVEGF agents are administered via intravitreal injection, which is undesirable for the subject and carries the potential for side effects (e.g., eye redness, eye pain, infection). Therefore, protocols exist that attempt to identify the minimum effective frequency of aVEGF injections. Many of these techniques resemble guesswork and check-and-check approaches.

[0004] One such technique is a treatment extension protocol, in which the inter-injection interval is slowly extended unless a new leak is observed after the previous inter-injection period. A drawback of this approach is that some subjects will experience a new leak before the injection frequency has increased to a sufficient level.

[0005] It would be advantageous to identify an objective, subject-specific approach to determining an aVEGF injection schedule sufficient to effectively keep the eye dry while avoiding over-injection. Summary of the Invention [Means for solving the problem]

[0006] overview An optical coherence tomography (OCT) image corresponding to a subject's eye having age-related macular degeneration (e.g., wet age-related macular degeneration) is accessed. Sets of pixels corresponding to retinal layers are identified within the OCT image. The OCT image is flattened based on the sets of pixels. One or more cropping processes are performed using the flattened OCT image to generate one or more cropped images. Labels corresponding to characteristics of a proposed treatment schedule for the subject's eye are generated using the one or more cropped images. The labels are output.

[0007] The one or more cropped images may include a plurality of cropped images, each of the plurality of cropped images including a different patch within the flattened OCT image. Generating the labels may include, for each cropped image of the one or more cropped images, generating a patch-specific result using a patch-specific neural network. The patch-specific neural network may be trained with other images of a size corresponding to the cropped image. Processing the one or more cropped images may further include processing the patch-specific result using an ensemble model.

[0008] The label may indicate the frequency of treatment administration (e.g., predicted to be sufficiently effective to prevent fluid leakage from the eye's blood vessels between successive treatment administrations) and / or the interval between successive treatment administrations (e.g., predicted to be sufficiently effective to prevent fluid leakage from the eye's blood vessels between successive treatment administrations). Characteristics of the proposed treatment schedule may include the frequency of treatment administration (e.g., predicted to be sufficiently effective to prevent fluid leakage from the eye's blood vessels between successive treatment administrations) and / or the interval between successive treatment administrations (e.g., predicted to be sufficiently effective to prevent fluid leakage from the eye's blood vessels between successive treatment administrations).

[0009] The neural network may include a deep convolutional neural network having at least five convolutional blocks and less than 10,000 trainable parameters. The retinal layers within the retina (e.g., the representation of which is used for flattening) may include the retinal pigment epithelium layer. The proposed treatment schedule may include a proposed schedule for administering an anti-vascular endothelial growth factor. The labels may be generated by inputting one or more cropped images into a neural network (e.g., including one or more convolutional neural networks and / or ensemble neural networks).

[0010] In some embodiments, a system is provided that includes one or more data processors and a non-transitory computer-readable storage medium that includes instructions that, when executed on the one or more data processors, cause the one or more data processors to perform some or all of one or more of the methods disclosed herein.

[0011] In some embodiments, a computer program product is provided that is tangibly embodied in a non-transitory machine-readable storage medium and includes instructions configured to cause one or more data processors to perform some or all of one or more of the methods disclosed herein.

[0012] In some embodiments, a method is provided for treating an eye of a subject with age-related macular degeneration. An OCT image depicting at least a portion of the eye of the subject with age-related macular degeneration (e.g., wet age-related macular degeneration) is accessed. Processing of the OCT image begins with a machine learning model. The processing includes flattening the OCT image and processing at least a portion of the flattened OCT image using a neural network. Results of the processing of the OCT image are accessed. The results are indicative of characteristics of a proposed treatment schedule for the eye of the subject. The eye of the subject is treated according to the proposed treatment schedule.

[0013] Treating the subject's eye according to the proposed treatment schedule can include administering an anti-vascular endothelial growth factor to the eye according to the proposed treatment schedule. Some embodiments of the present disclosure include a system including one or more data processors. In some embodiments, the system includes a non-transitory computer-readable storage medium including instructions that, when executed on the one or more data processors, cause the one or more data processors to perform some or all of one or more methods and / or some or all of one or more processes disclosed herein. Some embodiments of the present disclosure include a computer program product tangibly embodied in a non-transitory machine-readable storage medium including instructions configured to cause one or more data processors to perform some or all of one or more methods and / or some or all of one or more processes disclosed herein.

[0014] The terms and expressions which have been employed are used as terms of description rather than of limitation, and there is no intention in the use of such terms and expressions to exclude equivalents of the features shown and described or portions thereof, but it is recognized that various modifications are possible within the scope of the invention as claimed. Thus, although the claimed invention has been specifically disclosed by embodiments and optional features, it will be understood that modifications and variations of the concepts disclosed herein may be resorted to by those skilled in the art, and that such modifications and variations are deemed to be within the scope of the invention as defined by the appended claims.

[0015] The present disclosure is described in conjunction with the accompanying drawings, in which: [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 1 shows a block diagram of a network for collecting and analyzing optical coherence tomography (OCT) images to predict an effective treatment schedule for treating an ocular disorder according to some embodiments.

[0017] [Figure 2] FIG. 2 shows a flowchart of an exemplary process for processing OCT images using a deep neural network to generate labels corresponding to treatment administration schedules.

[0018] [Figure 3A] FIG. 3A shows multiple two-dimensional OCT images associated with different depths of a subject's eye. [Figure 3B] FIG. 3B shows multiple two-dimensional OCT images associated with different depths of the subject's eye. [Figure 3C] FIG. 3C shows multiple two-dimensional OCT images associated with different depths of the subject's eye.

[0019] [Figure 4A] FIG. 4A shows the unflattened and flattened OCT images, respectively. [Figure 4B] FIG. 4B shows the unflattened and flattened OCT images, respectively.

[0020] [Figure 5] Figure 5 shows a deep learning pipeline for training and using a deep learning network to process data.

[0021] [Figure 6A] Figure 6A shows aggregated receiver operator curves (ROCs) based on the labels (associated with the first labeling framework) generated by a dense neural network that predicts whether individual OCT images correspond to cases in which the associated eye will receive high-intensity aVEGF treatment or low-intensity aVEGF treatment, respectively. [Figure 6B] Figure 6B shows the aggregated receiver operator curves (ROC) based on the labels (associated with the first labeling framework) generated by the dense neural network that predicts whether an individual OCT image corresponds to a case in which the associated eye will receive high-intensity aVEGF treatment or low-intensity aVEGF, respectively.

[0022] [Figure 6C] Figure 6C shows the ROC characterizing the accuracy of treatment intensity predictions (associated with the first labeling framework) generated by using a random forest model to process OCT images.

[0023] [Figure 7A] Figure 7A shows the label-based ROC (associated with the second labeling framework) generated by a dense neural network predicting whether various OCT images correspond to cases in which the associated eye will receive high-intensity aVEGF treatment or low-intensity aVEGF treatment, respectively. [Figure 7B]Figure 7B shows the label-based ROC (associated with the second labeling framework) generated by a dense neural network predicting whether various OCT images correspond to cases in which the associated eye will receive high-intensity aVEGF treatment or low-intensity aVEGF treatment, respectively.

[0024] [Figure 7C] Figure 7C shows the ROC characterizing the accuracy of treatment intensity predictions (associated with the second labeling framework) generated by using a random forest model to process OCT images. DETAILED DESCRIPTION OF THE INVENTION

[0025] In the accompanying drawings, similar components and / or features may have the same reference label. Furthermore, various components of the same type may be distinguished by following the reference label with a dash and a second label that distinguishes between the similar components. When only a first reference label is used in this specification, the description is applicable to any of the similar components having the same first reference label, regardless of the second reference label.

[0026] Detailed Description I. Overview The present disclosure relates to predicting treatment schedule characteristics for a given subject and a given eye based on processing of optical coherence tomography (OCT) images. The treatment schedule may indicate when multiple doses of anti-vascular endothelial growth factor (aVEGF) should be administered. The aVEGF agent may include (for example) ranibizumab or bevacizumab. The aVEGF treatment characteristics may indicate (for example) the frequency of treatment administration, one or more periods between successive treatment administrations, or the number of times treatment should be administered within a given period. In some examples, the aVEGF treatment characteristics may alternatively or additionally specify the dosage of the active ingredient to be administered.

[0027] A given eye may be determined to have (e.g., may have been diagnosed with) macular degeneration, such as age-related macular degeneration and / or wet age-related macular degeneration.

[0028] In one embodiment, the OCT image is preprocessed to flatten the image based on the representation of a particular biological structure, such as the retinal pigment epithelium, and the preprocessing further includes cropping the flattened image to exclude portions of the image that are relatively far from the straightened retinal pigment epithelium representation.

[0029] The preprocessed images are then input into a trained neural network (e.g., a deep neural network and / or a convolutional neural network) to generate a label corresponding to a treatment schedule predicted to be effective in the eye (e.g., for treating age-related macular degeneration, e.g., wet age-related macular degeneration, of the eye). More specifically, the label may indicate characteristics of a treatment schedule predicted to be effective in preventing vascular leakage between successive administrations of a therapeutic agent (e.g., successive injections of aVEGF).

[0030] The label may include one or more numbers (e.g., identifying a proposed frequency of treatment administration, a proposed number of treatment administrations for a given time period, or a proposed interval between successive treatment administrations), one or more categories (e.g., a "low," "medium," or "high" frequency identifier), and / or one or more binary indicators (a "low" or "not low" frequency identifier). For example, a "low" label may predict that a treatment administration number or frequency equal to (or potentially below) a predetermined threshold or value will be effective in preventing leakage between treatment administrations for a given subject's eye. As another example, a "high" label may indicate that leakage between treatment administrations is predicted to occur for a given subject's eye unless a high-intensity treatment (e.g., defined using a treatment administration number or frequency threshold or value) is used. Thresholds can be defined as absolute values ​​and durations (e.g., 5 or fewer treatments administered over a 20-month period is labeled low, 15 or more treatments administered over a 20-month period is labeled high, and 6-14 treatments administered over a 20-month period is labeled medium). In some examples, labels are defined using thresholds for frequency of treatment administration (e.g., if, on average, treatment is administered less than once every 3 months, it is labeled "low," and if, on average, treatment is administered at least once every month, it is labeled "high."

[0031] Each label may be generated via an activation layer in a neural network.

[0032] In some examples, a set of treatment-administration schedules is defined.For example, the first schedule can indicate that dosage is delivered at 1 month, 2 months, 4 months, 6 months, 9 months, and 12 months from baseline time.The second schedule can indicate that dosage is delivered at 1 month, 2 months, 3 months, 4.5 months, 6 months, 8 months, 10 months, and 12 months.Then, the label can specify a specific one of the set of treatment-administration schedules.

[0033] The label may indicate the characteristics of the maintenance therapy schedule after the administration of the initial (or onboarding) treatment. For example, the initial treatment may be defined as consisting of a specific number of treatment doses administered according to a specific schedule (e.g., monthly administration for three months). The initial treatment may, but does not necessarily, use the same type of therapeutic agent and the same specific schedule across subjects. The maintenance treatment may, but does not have to, use the same or different therapeutic agent as the initial therapeutic agent used.

[0034] The neural network may include a deep network that may include multiple convolutional blocks (e.g., 10 convolutional blocks), which may exponentially increase the expressive power of the network. The neural network may also or alternatively be lean (e.g., have fewer than 6,000 learnable parameters), which allows the network to be trained quickly (e.g., less than 1 minute per epoch) with relatively few computational resources. The neural network may include a fully convolutional neural network that is invariant to the input space size.

[0035] The neural network may be trained using a set of patches from preprocessed training OCT images. Data augmentation may be performed by extracting multiple patches (e.g., of various sizes and / or relative positions) from a single OCT image. In some examples, a single neural network (e.g., a flexible CNN) is trained using patches of various sizes. In some examples, multiple neural networks are each associated with a given patch size and / or relative patch position and trained using images of the given patch size and / or relative patch position. The neural network may include an ensemble model constructed and trained to aggregate and process results from multiple patch-size-specific neural networks. Cross-validation techniques may be used while training the neural network. For example, cross-validation techniques may include 5-fold cross-validation or Monte Carlo cross-validation, which may have the advantages of being scalable, fault-tolerant, poolable, and supporting distributed model training, selection, and / or evaluation.

[0036] A label identifying characteristics of the treatment schedule may be output and provided to a healthcare provider. In some examples, the output may indicate (or indicate) that the identified treatment schedule characteristics are potentially associated with a maintenance-therapy period. The healthcare provider may use the label to inform the selection of a treatment approach. For example, the healthcare provider may recommend and / or prescribe a treatment (e.g., aVEGF) according to a schedule having the treatment schedule characteristics indicated by the label.

[0037] II. Definition As used herein, an "effective" treatment for AMD can include a scenario in which a treatment (e.g., administered according to a particular treatment schedule) is observed in which no new blood vessel leakage or visual acuity loss is observed between doses administered. In some instances, an effective treatment includes a minimally effective treatment. For example, it will be understood that multiple treatment schedules may be effective for treating AMD, and in such cases, a minimally effective treatment may use a treatment schedule that includes (for example) the smallest amount of treatment administered over a period of time, the least frequent treatment administered over a period of time, the longest average time between successive treatment administrations, or the lowest total dose administered over a period of time.

[0038] As used herein, a treatment "schedule" indicates when each of multiple treatments of a particular treatment is administered. A treatment schedule may specify (for example) a set of dates, one or more dosing periods, or dosing frequencies. For example, a treatment schedule may indicate that a particular treatment is to be administered once a month. In some instances, a treatment schedule includes one or more ranges. For example, a treatment schedule may specify a dosing interval indicating that each of one or more treatment doses is to be administered anytime between 7 and 9 weeks after the previous treatment administration.

[0039] III. Network for generating treatment schedule labels using OCT images 1 shows a block diagram of a network 100 for collecting and analyzing optical coherence tomography (OCT) images to predict effective treatment administration schedules for treating ocular disorders, according to some embodiments of the present invention. Network 100 includes one or more imaging systems 105 configured to collect one or more images, each depicting at least a portion of a subject's eye.

[0040] The imaging system 105 may be configured to collect optical coherence tomography (OCT) images using OCT imaging technology. OCT is a non-invasive technique that uses light waves to construct cross-sectional images of the eye. The imaging system 105 may include an interferometer (e.g., a light source, a beam splitter, and a reference mirror). For example, a light source may generate a light beam (e.g., a low-coherence near-infrared light beam) that may be split by a beam splitter. A first portion of the split light beam may be directed toward the subject's eye, and a second portion of the split light beam may be directed toward the reference mirror. Backscattered light from the eye and the reference mirror may be combined, and the combined light may be analyzed to measure interference. Areas of the target tissue that reflect more light may result in more interference.

[0041] Each scan can be generated by laterally steering a light beam to generate interference information associated with multiple locations. Each set of A-scans can be defined to correspond to a specific scan-related depth. Each A-scan can be a one-dimensional scan. Sets of A-scans (e.g., 128, 256, or 512 A-scans) can then be aligned with each other to generate a two-dimensional B-scan.

[0042] The B-scan may depict (for example) at least a portion of the retina, macula, and / or optic nerve. The thickness between particular layers of the eye and / or the shape between various layers may indicate whether blood vessels have ruptured and leaked into the eye (e.g., into the retinal, subretinal, or subpigmented epithelial spaces).

[0043] The imaging system 105 may further include one or more processors and / or one or more memories to facilitate computational operations. For example, computational operations may include using multiple A-scans to generate a B-scan, normalizing intensity values, changing resolution, and / or applying one or more filters. It will be appreciated that multiple B-scans may be generated for a given eye (e.g., based on a set of multiple A-scans). Each of the multiple B-scans may be associated with a different depth.

[0044] The imaging system 105 may transmit and / or otherwise serve images to the OCT image processing controller 110. For example, the imaging system 105 may upload images in association with one or more identifiers to a remote data store, which may be partially or fully accessible to the OCT image processing controller 110. As another example, the imaging system 105 may transmit one or more images of the eye to the client system 115, which may then transmit or utilize the images to the OCT image processing controller 110. The client system 115 may be a computing system associated with a healthcare provider (e.g., a doctor, hospital, ophthalmologist, medical technician, etc.) providing care to a subject whose eye was imaged by the imaging system 105.

[0045] The OCT image processing controller 110 may include a preprocessing controller 120 configured to preprocess the image (e.g., one or more B-scans). Preprocessing may be performed to mitigate image variations caused by different types of machines and / or environments. For example, preprocessing may include changing the image resolution, changing the image zoom, and / or changing the intensity distribution of the image (e.g., by applying normalization or standardization techniques).

[0046] Due to the natural curvature of the eye, B-scans may depict curved layers. Therefore, preprocessing may include flattening the image. Flattening the image may include first estimating the location of the depiction of a given structure. The location may be defined as a set of pixels. The structure may include the retinal pigment epithelium.

[0047] Detecting structures may include performing segmentation. Detecting structures may alternatively or additionally include applying a filter (e.g., a Gaussian filter) to remove noise from the image. Next, within each column, one or more pixels associated with the highest intensity across the column may be pre-identified as corresponding to the structure. A smoothing function may be used to facilitate the selection of relatively contiguous pixels across the column. The smoothing may then be performed by shifting the columns relative to each other so that the selected pixels (e.g., associated with the highest intensity) are aligned with the rows.

[0048] In some cases, smoothing may involve segmenting the biological structure (e.g., by applying a filter and then thresholding the filtered image) and then fitting a function to the segmented pixels. The function may include a spline function. Columns of the image may be shifted relative to each other to smooth the spline function.

[0049] Preprocessing may include cropping a portion of the flattened image. Cropping may be performed to generate an image of a target size. Cropping may be performed to remove the top of the image and / or the bottom of the image. Cropping may be performed to remove all pixels that are above a first threshold above the flattened anatomical structure and / or above a second threshold above the flattened anatomical structure. In some examples, pixel intensity modification (e.g., normalization or standardization) is performed following flattening.

[0050] At least some of the images, along with corresponding labels, may be used to train a neural network. The corresponding labels may indicate one or more characteristics of treatments administered over a period of time following collection of the images. That is, for each image in the training dataset, the date on which it was collected may be defined as a baseline time. The period during which treatment is characterized may begin (for example) at the baseline time, one month after the baseline time, two months after the baseline time, or three months after the baseline time. In some examples, initial treatment begins immediately after or just before the baseline time, and the monitoring period is defined to begin after completion of the initial treatment.

[0051] The treatment data store 125 may contain information used to determine the labels, or may contain the labels themselves. For example, the treatment data store 125 may contain, for each of a set of subjects, a record that includes an identifier associated with the subject or image and also includes observed treatment information. The observed treatment information may identify the type of treatment administered, the date the treatment was administered, the interval between treatment administrations, and / or the amount of treatment administered over a period of time.

[0052] The OCT image processing controller 110 may include an OCT image processing training controller 130 that may access training data including baseline images and corresponding treatment label data. In some examples, the OCT image processing training controller 130 may generate a label to associate with each training database line image. The label may be generated based on treatment data (e.g., from the treatment data store 125) corresponding to an identifier associated with the baseline image. For example, the label may identify the amount of treatment (e.g., of a particular type) administered over a monitored period (e.g., associated with an identifier corresponding to the baseline image by referencing the treatment administration date within the corresponding period). As another example, the label may identify the interval between the last two treatment administrations or the average (or median) interval between multiple consecutive treatment administration pairs. It will be understood that in some examples, the processing data store 125 stores the labels themselves.

[0053] The OCT image processing training controller 130 may train one or more neural networks using the preprocessed images and labels associated with the training dataset. The neural networks may include one or more deep neural networks and / or one or more convolutional neural networks. Each of the neural networks may be configured to include at least 1, at least 2, at least 5, at least 10, at least 15, or at least 20 convolutional blocks. Each of the one or more deep convolutional neural networks may be a thin neural network having fewer than 20,000 trainable parameters, fewer than 10,000 trainable parameters, fewer than 6,000 trainable parameters, or fewer than 3,000 trainable parameters.

[0054] In some examples, the neural network may include multiple neural networks. Each neural network may be trained to process images of different sizes and / or each neural network may be trained to process patches corresponding to different locations. Thus, in some examples, initial preprocessing is performed at the image level (e.g., to flatten and crop the image). Subsequent preprocessing may be performed to prepare input for a particular neural network by extracting specific patches from the flattened and cropped images, where the size and location of the patches are determined based on metadata associated with the particular neural network. For example, the multiple neural networks may include a first set of neural networks trained to process patches having 128x128 pixels, a second set of neural networks trained to process patches having 256x256 pixels, a third set of neural networks trained to process patches having 512x512 pixels, a fourth set of neural networks trained to process patches having 1024x1024 pixels, and another neural network trained to process the entire image (a low-level image-based network). Each of a given set of neural networks (e.g., a first set, a second set, etc.) can be associated with a given location of an image region, and the regions associated with the first set can be arranged in an overlapping or non-overlapping manner across the image regions.

[0055] Outputs from multiple neural networks may be input to yet another neural network (e.g., a committee machine) that can integrate the results, so that the neural networks collectively function as an ensemble model. For example, other neural networks may be configured to learn weights to apply to the outputs of the various neural networks. In some examples, the integrated network learns a single weight to apply to each output of a given low-level neural network (e.g., associated with a particular patch). In some examples, the integrated network learns more complex relationships, and the weights applied to the output of a given neural network may depend (for example) on the output from the given neural network and / or on the output from each of one or more other neural networks. In some examples, each patch-specific neural network is configured to generate both an output and a confidence metric. The integrated network may then (additionally or alternatively) determine the weight to apply to a given output from a given low-level neural network based at least in part on the confidence metric from the given low-level neural network and / or on the confidence metric from one or more other low-level neural networks.

[0056] The OCT image processing training controller 130 may be configured to train all neural networks (e.g., all patch-specific neural networks and other integrated neural networks) together. Alternatively, the OCT image processing training controller 130 may train each patch-specific network, the low-level image-based network, and the other integrated networks individually. In some examples, independent training is performed to initialize the parameters of each model, and then collective training is performed.

[0057] The OCT image processing controller 110 may include a treatment intensity generator 135 that uses a trained neural network to generate results corresponding to OCT images not included in the training data. The OCT images may correspond to subjects and / or eyes not represented in the training data. The imaged eyes may include eyes diagnosed with age-related macular degeneration and / or wet age-related macular degeneration.

[0058] The OCT images may include pre-processing, which may include remote pre-processing (e.g., in imaging system 105) and / or pre-processing performed in pre-processing controller 120. Pre-processing may include one or more pre-processing techniques disclosed herein, such as using multiple A-scans to generate a B-scan, flattening the B-scan image, cropping the flattened image, and / or adjusting the intensity (e.g., normalizing or standardizing).

[0059] The treatment intensity generator 135 may then feed the preprocessed OCT images to a trained neural network to generate an output corresponding to characteristics of a treatment schedule predicted to be effective in treating the eye (e.g., sufficient to prevent blood vessels from leaking between treatment administrations). In some examples, the output corresponds to characteristics of a treatment schedule predicted to include a minimum amount of treatment administration over a time interval (and / or a longest period between treatment administrations) that effectively treats the eye. The output may identify the amount of treatment (e.g., a particular treatment, e.g., a particular aVEGF treatment) to be administered over a given period of time, the frequency at which the treatment (e.g., a particular type of treatment) is administered, and the interval (e.g., expressed as a particular number or range with units) separating successive administrations of the treatment. In some examples, the treatment intensity generator 135 applies one or more post-processing techniques to convert the output from the neural network into a result. For example, the output may identify a target frequency of treatment administration, and post-processing may convert the output into a set of dates (or date range) at which the treatment is administered.

[0060] The results may be returned to the client device 115. The client device 115 may be associated with (e.g., owned, used, controlled, and / or operated by) an entity that provides medical care to a subject whose eye was at least partially depicted in the analyzed OCT image. For example, the entity may include a doctor, a clinic, or an ophthalmologist. In some examples, the client device 115 initially provided the OCT image to the OCT image processing controller 110. In some examples, the client device 115 initiated and / or completed a request to the imaging system 105 to collect an OCT image.

[0061] Results indicating characteristics of treatment schedules predicted to provide effective treatment to the eye may be used by a care delivery entity to inform the selection and / or definition of a treatment schedule prescribed and / or recommended for a subject.

[0062] IV. Machine Learning Process for Generating Treatment Schedule Labels Using OCT Images FIG. 2 shows a flowchart of a process 200 for processing OCT images using a deep neural network to generate labels corresponding to treatment administration schedules. In block 205, a set of OCT training images is accessed. Each OCT training image may be collected from an eye and subject diagnosed with age-related macular degeneration (e.g., wet age-related macular degeneration). Each OCT training image may be further associated with historical data indicating a selection of a treatment schedule (e.g., frequency of aVEGF treatment administration) and / or treatment outcomes (e.g., indicating the effectiveness of treatment administered according to a particular treatment schedule). Each OCT training image may be associated with data indicating the subject's subsequent observed condition in response to a treatment regimen associated with a particular treatment (e.g., a particular therapeutic agent).

[0063] The training data set may include multiple two-dimensional images corresponding to a single subject's eye, for example, the multiple two-dimensional images may correspond to different depths.

[0064] At block 210, each of the OCT training images is flattened. Flattening may be performed to transform the image so that depictions of particular biological structures (e.g., retinal layers such as the retinal pigment epithelium) are substantially flat in the flattened image. Other portions (e.g., pixels) in the image may also be morphed to adjust their position in response to the flattening.

[0065] Specific biological structures may be identified based on metadata associated with the image (e.g., identifying pixels associated with a layer) and / or by performing computer vision techniques (e.g., to detect and / or characterize edges). In some examples, biological structures may be identified during image acquisition and detected within the OCT scanner. Figure 4A shows a portion of an unflattened OCT training image, and Figure 4B shows a portion of a flattened OCT training image. It will be understood that each individual OCT training image (and each flattened OCT training image) may comprise a two-dimensional image. The bottom white layer in each image is the retinal pigment epithelium, which served as the basis for flattening. Therefore, the approach is to define a target range of intensity values ​​(e.g., an open or closed range), detect pixels with intensity values ​​within that range, and then define a line (e.g., a curve) based on those detected pixels. The flattened OCT image was further cropped to generate the flattened and cropped OCT image shown in Figure 4B.

[0066] At block 215, each flattened OCT training image is cropped. Cropping may be performed to remove the top and / or bottom of the flattened image. The removed portions may lack depiction of a portion of the retina and / or have intensity statistics (e.g., mean intensity, median intensity, or intensity variability) below a predetermined threshold. In some examples, the same or subsequent cropping is performed to generate patches of flattened images (e.g., having widths smaller than the minimum or maximum widths of the flattened, potentially initially cropped, OCT training images). Cropping may be performed to generate images of a predetermined size. In some examples, multiple biological landmarks are detected and cropping is performed to scale the training images to a default scale.

[0067] For example, each image shown in Figures 3A-3C was collected by imaging a single eye using a Zeiss Cirrus machine. Each of Figures 3A-3C corresponds to a central B-scan at a different depth. Additionally, the images shown include a flattened image (with flattening performed toward the retinal pigment epithelium layer) and a cropped image. Cropping was performed to include a region extending from 128 pixels below the flattened RPE layer to 384 pixels above the flattened RPE layer.

[0068] At block 220, a training label characterizing a treatment administration event (e.g., an intravitreal injection event) is identified for each OCT training image. The training label may be based on the treatment selected to treat the subject's eye (e.g., used following collection of the OCT training images and / or at a particular pre-specified time point following collection of the OCT training images).

[0069] The training label may include characteristics of the selected treatment or the results of the selected treatment. For example, the training label may identify the medication being used in the treatment, the dosage of the particular therapeutic agent being administered, or the temporal characteristics of the treatment administration (e.g., the frequency or duration of drug administration or the period between doses). As one example, the medication may include any aVEGF medication or a particular aVEGF medication. As another example, the training label may identify whether a particular treatment or type of treatment (and / or a particular dosage of a particular treatment or type of treatment) was sufficiently effective, such as whether a particular event was not observed (e.g., whether new vascular leakage was not observed), whether a particular event was observed (e.g., whether macular fluid was removed secondary to underlying wet age-related macular degeneration), or whether a particular event was not observed (e.g., whether leakage occurred between successive treatment doses). As an additional or alternative example, the training label may indicate whether a particular treatment schedule was sufficiently effective (e.g., indicating the relative times at which multiple treatments were administered and potentially the dosages of multiple treatments), such as whether a particular event was not observed (e.g., whether no new vascular leakage was observed) or whether a particular event was identified (e.g., whether macular fluid was cleared secondary to potential wet age-related macular degeneration).

[0070] The treatment label may indicate the characteristics of the treatment administered using an approach in which the interval between successive treatment administrations decreases each time leakage is observed, or in which treatment is administered after each leakage observation. In some examples, the label may indicate the amount of treatment administered over a period of time (e.g., potentially normalized by the duration of the time period to indicate frequency). The period may include (for example) a period from a baseline date and / or a date after an initial treatment period. The period may have a defined duration (e.g., with respect to training data).

[0071] In some examples, the label may be based in part or in whole on one or more treatment modifications (or lack thereof). For example, a default treatment regimen may be selected, and the training label may reflect whether and / or which modifications to the default treatment regimen were used to treat the eye. In some examples, the label may identify the number or frequency (e.g., and dosage) of a given type of treatment administered within a given period of time.

[0072] In block 225, deep neural networks are trained using the cropped and flattened training images and corresponding training labels. In some examples, block 225 includes training each of one or more deep neural networks using at least some of the cropped and flattened training images and corresponding training labels. For example, a set of deep neural networks may be used, each trained using patches from cropped and flattened training images of a particular size (e.g., different patch sizes are processed by different networks). Each of the set of deep neural networks may have similar or the same architecture, but may be trained using different portions of the training data so that different parameter values ​​are learned by the different networks. An ensemble model may be used to aggregate and process results from multiple low-level, patch-size-specific neural networks.

[0073] Each of the one or more deep convolutional neural networks may include at least 1, at least 2, at least 5, at least 10, at least 15, or at least 20 convolutional blocks. Each of the one or more deep convolutional neural networks may be a thin neural network having fewer than 20,000 learnable parameters, fewer than 10,000 learnable parameters, fewer than 6,000 learnable parameters, or fewer than 3,000 learnable parameters.

[0074] It will be appreciated that in some examples, cross-validation techniques (e.g., nested cross-validation techniques) may be used to select hyperparameters and / or estimate errors. More specifically, to estimate generalization ability, cross-validation techniques may be used instead of dividing a training dataset into training, validation, and test subsets. This latter technique may include developing a model using training data, selecting an optimal model using validation data, and evaluating the performance of the selected optimal model using test data. On the other hand, as shown in FIG. 5, a nested cross-validation approach may use multiple levels of cross-validation iterations (i.e., outer loop 505 and inner loop 510) to perform data division and evaluation of the model's generalization ability.

[0075] In the outer loop 505, the training data may be split into test data and a combination of training data and validation data (e.g., using a 5-fold split). At each iteration, the training and validation data may be further separated in the inner loop 510. The training data may be used to train one or more fully convolutional neural networks. In some examples, a single fully convolutional neural network is trained using the training data. Each of the fully convolutional neural networks may include a relatively small number of parameters (e.g., fewer than 10,000 parameters, fewer than 8,000 parameters, or fewer than 5,000 parameters). The validation data may be used to select hyperparameters for the model (e.g., to determine optimal hyperparameters). In the illustrated example, the validation data is used to determine a particular stopping epoch (e.g., an optimal stopping epoch). With a given split of the training data and validation data, the particular stopping epoch may be determined by selecting the epoch (across the multiple epochs used during training) with the lowest validation loss.

[0076] The training and validation splits may be performed iteratively, with a different portion of the data assigned to the training data at each iteration. The final specific stopping period (t* in the figure) may be a statistic generated based on the specific stopping periods identified for each split (e.g., the average of the specific stopping periods across the splits). Multiple models may be trained using a combination of training and validation data with the final specific stopping period. Each of the multiple models may have the same architecture but may include different learning parameters as a result of being trained using different training data elements. These multiple models may form a model ensemble or ensemble model 515. The ensemble model may be evaluated with held-out test data.

[0077] In block 230, the input image may be flattened and cropped. The input image may include OCT images associated with an eye and a subject with age-related macular degeneration (e.g., wet age-related macular degeneration). In some examples, the input image includes images not represented in the training data. The flattening may be performed using the same or similar techniques as those applied in block 210 to flatten the training images. The cropping may be performed using the same or similar techniques as those applied in block 215. In some examples, the cropping is performed to potentially remove top and / or bottom portions that do not depict a region of interest (e.g., a portion of the retina). The cropping may, but need not, extend to generate one or more patches of data (e.g., regardless of whether the patches were used to train the model).

[0078] In block 235, the flattened and cropped input image is processed using a trained deep neural network to generate labels. The labels may correspond to characteristics of a treatment administration schedule. The labels may identify one or more characteristics of a treatment administration schedule (e.g., related to the administration of an aVEGF agent) that is predicted to be effective for treating the eye depicted in the input image (e.g., removing macular fluid secondary to underlying wet age-related macular degeneration) and / or preventing progression of a condition (e.g., preventing progression to wet age-related macular degeneration).

[0079] The label may alternatively or additionally correspond to a prediction as to whether a treatment administration schedule (e.g., relating to the administration of an aVEGF schedule) will be effective in treating the indicated eye and / or preventing a particular type of progression.

[0080] A label corresponding to the treatment-administration schedule is output in block 240. For example, the label may be presented or transmitted (at or to a device associated with a healthcare provider). The label may be output along with other information about the subject, such as the subject's name and / or date of diagnosis. [Example]

[0081] V. Example 1 OCT images collected by imaging retinas diagnosed with age-related macular degeneration were analyzed using a deep learning model to generate eye- and subject-specific predictions regarding what type of aVEGF treatment should subsequently be provided to each subject's indicated eye.

[0082] VA method 1042 OCT images were captured during the loading period from the HARBOR PRN arm of nAMD. Specifically, OCT from study eyes from 352 subjects was evaluated and correlated with anti-VEGF treatment load over 23 months. The need for low-intensity anti-VEGF treatment was defined as having five or fewer injections over 21 visits between the loading period and the completion of three consecutive monthly ranibizumab treatment doses and the 23-month visit. (See Bogunovic H et al., Prediction of anti-VEGF treatment requirements in neovascular AMD using a machine learning approach. Invest Ophthalmol Vis Sci. 2017;58(7):3240-3248, incorporated herein by reference in its entirety for all purposes.) A stratified 5-fold fold was created at the subject level for nested cross-validation.

[0083] OCT images (1024 × 512 × 128 resolution) from a Zeiss Cirrus instrument were flattened (or rebased) to the flattened retinal pigment epithelium (RPE) layer and cropped 384 pixels above and 128 pixels below the RPE. The central 15 B-scans were selected. Stochastic cropping (e.g., the location to be cropped, and potentially the size used for cropping, was determined using a stochastic process) was applied to sample training and validation patches of random or pre-specified size.

[0084] We designed a deep learning network containing 10 convolutional blocks to exponentially increase expressive power with fewer than 5600 weight parameters. The deep learning network contained fully convolutional layers. This design facilitated fast and computationally inexpensive training. We applied an F-CNN (fully convolutional neural network) architecture to enable patches of any size as input and predict across slices. We used a committee machine as the ensemble model. Each committee member model generated a patch-level prediction, and the committee machine aggregated the patch-level predictions (e.g., by determining the average of the patch-level predictions) to generate the image-level result. The image-level result corresponded to a binary prediction regarding whether a particular treatment intensity (e.g., high anti-VEGF treatment or low anti-VEGF treatment would be administered in other cases) was used. Low-intensity anti-VEGF treatment was defined as having five or fewer anti-VEGF treatment doses over the 20-month period between the onboarding 3-month visit and the 23-month visit, when ranibizumab was administered monthly.

[0085] VB results Of the 547 PRN study eyes, 352 unique study eyes were eligible for analysis. Of these, the model predicted that 79 eyes (22.4%) would be classified as having disease requiring only low-intensity anti-VEGF treatment for effective treatment.

[0086] 1042 OCT scans from the loading period were used for modeling. For each scan, the observed treatment was classified as low-, medium-, or high-intensity anti-VEGF treatment, and the predicted treatment was similarly classified. The model achieved an area under the receiver operating curve (AUROC) of 78.6% (range, 72.7-84.4%).

[0087] VC conclusion The deep learning model performed well in predicting which type of anti-VEGF treatment (e.g., low, medium, or high) an individual nAMD subject would receive after the ranibizumab loading period. These types of results can help physicians / subjects understand future treatment loads. The results can also or alternatively be used to stratify subjects in future clinical trials aimed at evaluating anti-VEGF treatment tolerance.

[0088] VI. Example 2 Another analysis was performed using various sets of OCT images collected by imaging retinas diagnosed with age-related macular degeneration. A deep learning model was used to generate eye- and subject-specific predictions regarding which type of aVEGF treatment would subsequently be provided to each subject's depicted eye.

[0089] VI.A. Method The deep learning model was trained using 1,069 OCT images collected from 362 subjects. This training data was repeatedly split into a first subset used to train the model and a second subset to validate the model. During each of these data splits, the accuracy of the model's predictions on the validation data was determined. When evaluating predictions generated using the validation subset, average and aggregate statistics (e.g., mean AUROC statistics and aggregate AUROC statistics) were generated by assessing the accuracy of the predictions across the data splits. The holdout set (not used for training) included 183 OCT images from 62 subjects. Anti-VEGF treatment dose burden was evaluated using the following two metric frameworks, respectively: First, the "MUV" framework: Low-intensity anti-VEGF treatment was defined as having 5 or fewer treatment doses between the completion of 3 consecutive monthly ranibizumab injections and the 23-month visit. High-intensity anti-VEGF treatment was defined as having 16 or more aVEGF treatment doses between the completion of 3 consecutive monthly ranibizumab treatment doses and the 23-month visit. Second "Turing" framework: The need for low-intensity anti-VEGF treatment was defined as having two or fewer aVEGF treatment doses between the completion of three consecutive monthly ranibizumab treatment doses and the 12-month visit. High-intensity anti-VEGF treatment was defined as having nine or more aVEGF treatment doses between the completion of three consecutive monthly ranibizumab treatment doses and the 12-month visit.

[0090] The OCT images were preprocessed using the same type of flattening and cropping protocol as described in Example 1. A deep learning dense network was designed to be configured as described in Example 1.

[0091] VI.B. Results Figures 6A-6B show aggregated cross-validated receiver operator curves (ROCs) characterizing the accuracy of predictions of the first MUV framework labels generated by the high-density network. The data represent accuracy characterizations from validation data assessment. Figure 6A characterizes the accuracy of predictions regarding whether a given eye-subject pair would receive high-intensity anti-VEGF treatment. To generate the ROCs, the threshold was varied over a range. For each image (e.g., each image in the split or holdout dataset) and each threshold, a prediction was generated regarding whether the subject received a particular treatment schedule, considered a "positive" case, based on whether the results generated by the deep learning model exceeded the threshold. For each dataset (e.g., split dataset of the holdout dataset), a sensitivity metric was calculated as the number of true positives divided by the sum of true positives and false negatives. Additionally, a specificity metric was calculated for each dataset as the number of true negatives divided by the sum of true negatives and false positives.

[0092] Ideally, the sensitivity and specificity metrics are both close to (or equal to) 1.0 for at least one threshold. In this example, the receiver operator curve (ROC), which plots sensitivity against specificity across thresholds, includes one or more points near or at (100,100), and the curve does not follow a unit line. One technique for measuring the extent to which an ROC has these properties is to measure the area under the curve. An area under the curve near or at 100% indicates that, for at least one threshold, the model can successfully predict positive cases to be used and can successfully predict negative cases.

[0093] Figure 6A defines "positive" cases as cases in which the subject received high-intensity anti-VEGF treatment, and the positive cases correspond to analyses in which high-intensity anti-VEGF treatment was defined according to the MUV label definition above. ROC curves were calculated for each of the 10 data partitions. Figure 6A shows the mean ROC and ROC distribution. The mean AUROC (calculated by averaging the AUROC calculated for each of the 10 data partitions) was 0.77, and the aggregate AUROC (calculated by first averaging the ROCs for the data partitions and then calculating the AUROC using the mean ROC) was 0.76.

[0094] Figure 6B corresponds to an analysis in which "positive" cases were defined as cases in which the subject received low-intensity anti-VEGF treatment (as defined by the MUV framework). The plot shown characterizes the accuracy of prediction regarding whether a given eye-subject pair received low-intensity anti-VEGF treatment. The mean AUROC was 0.80, and the aggregate AUROC was 0.79.

[0095] Figure 6C shows the receiver operator curves for high- and low-intensity treatment prediction using a random forest model of extracted features (e.g., using 10-fold cross-validation to generate a binary prediction for whether a particular treatment intensity was administered). As shown, the AUROCs for predicting high- and low-intensity treatment were 0.77 and 0.70, respectively, both lower than those corresponding to predictions from the dense neural network technique (for the data in Figure 6A-B).

[0096] Figures 7A-7B show receiver operating characteristics (ROCs) characterizing the accuracy of predictions using labels from the second Turing framework. Figure 7A characterizes the accuracy of predictions regarding whether a given eye-subject pair will receive high-intensity anti-VEGF treatment (as defined using the second Turing framework). The mean AUROC was 0.73, and the aggregate AUROC was 0.75 (calculated using 10-fold cross-validation). Figure 7B characterizes the accuracy of predictions regarding whether a given eye-subject pair will receive low-intensity anti-VEGF treatment (as defined using the second Turing framework). The mean AUROC was 0.69, and the aggregate AUROC was 0.69 (calculated using 10-fold cross-validation). Figure 7C shows receiver operator curves for high-intensity and low-intensity treatment predictions using a random forest model on extracted features. As shown, the AUROCs for predicting high-intensity and low-intensity treatments were 0.76 and 0.66, respectively, both lower than those corresponding to predictions from the dense neural network technique (for data in Figures 7A-B).

[0097] A summary comparison of end-to-end performance using various metrics and prediction techniques is shown in Table 1. [Table 1]

[0098] VI.C. Conclusion The deep learning model continued to demonstrate strong performance when evaluated for predicting anti-VEGF treatment after a loading period (predicting the frequency of treatment administration based on processing of baseline OCT images) using validation or holdout data. Performance metrics remained strong even when different types of performance metrics were used. Furthermore, performance exceeded the output from other techniques for predicting anti-VEGF treatment.

[0099] VII. Further Considerations Some embodiments of the present disclosure include a system including one or more data processors. In some embodiments, the system includes a non-transitory computer-readable storage medium including instructions that, when executed on the one or more data processors, cause the one or more data processors to perform some or all of one or more methods and / or some or all of one or more processes disclosed herein. Some embodiments of the present disclosure include a computer program product tangibly embodied in a non-transitory machine-readable storage medium including instructions configured to cause one or more data processors to perform some or all of one or more methods and / or some or all of one or more processes disclosed herein.

[0100] The terms and expressions which have been employed are used as terms of description rather than of limitation, and there is no intention in the use of such terms and expressions to exclude equivalents of the features shown and described or portions thereof, but it is recognized that various modifications are possible within the scope of the invention as claimed. Thus, although the claimed invention has been specifically disclosed by embodiments and optional features, it will be understood that modifications and variations of the concepts disclosed herein may be resorted to by those skilled in the art, and that such modifications and variations are deemed to be within the scope of the invention as defined by the appended claims.

[0101] The description provides only preferred exemplary embodiments and is not intended to limit the scope, applicability, or configuration of the present disclosure. Rather, the description of preferred exemplary embodiments provides those skilled in the art with an enabling description for implementing various embodiments. It will be understood that various changes can be made in the function and arrangement of elements without departing from the spirit and scope of the appended claims.

[0102] Specific details are provided herein to provide a thorough understanding of the embodiments. However, it will be understood that the embodiments may be practiced without these specific details. For example, circuits, systems, networks, processes, and other components may be shown as components in block diagram form in order to avoid obscuring the embodiments in unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring the embodiments. The present invention provides, for example, the following items. (Item 1) 1. A computer-implemented method comprising: accessing an optical coherence tomography (OCT) image corresponding to an eye of a subject having age-related macular degeneration; identifying a set of pixels within the OCT image corresponding to a retinal layer; flattening the OCT image based on the set of pixels; performing one or more cropping processes using the flattened OCT image to generate one or more cropped images; generating a label corresponding to a characteristic of a proposed treatment schedule for the eye of the subject, the label being generated using the one or more cropped images; and and outputting the label. (Item 2) wherein the one or more cropped images include a plurality of cropped images, each of the plurality of cropped images including a different patch in the flattened OCT image, and generating the label; For each cropped image of the one or more cropped images, generating a patch-specific result using a patch-specific neural network; and and processing the patch-specific results using an ensemble model. (Item 3) 3. The computer-implemented method of claim 1 or 2, wherein the label indicates a frequency of treatment administration predicted to be sufficiently effective to prevent fluid from leaking from the eye's blood vessels between successive treatment administrations. (Item 4) 3. The computer-implemented method of claim 1 or 2, wherein the characteristic of the proposed treatment schedule indicates an interval between successive administrations of treatment. (Item 5) 5. The computer-implemented method according to any one of items 2 to 4, wherein the patch-specific neural network is trained using an image of a size corresponding to the cropped image. (Item 6) 6. The computer-implemented method according to any one of items 1 to 5, wherein the retinal layer comprises a retinal pigment epithelium layer. (Item 7) 7. The computer-implemented method of any one of items 1 to 6, wherein the proposed treatment schedule includes a proposed schedule for administering an anti-vascular endothelial growth factor. (Item 8) 8. The computer-implemented method of any one of items 1 to 7, wherein the labels are generated by inputting the one or more cropped images into a neural network. (Item 9) 1. A system comprising: one or more data processors; a non-transitory computer-readable storage medium containing instructions that, when executed on the one or more data processors, cause the one or more data processors to perform a set of operations, the set of operations including: accessing an optical coherence tomography (OCT) image corresponding to an eye of a subject having age-related macular degeneration; identifying a set of pixels within the OCT image corresponding to a retinal layer; flattening the OCT image based on the set of pixels; performing one or more cropping processes using the flattened OCT image to generate one or more cropped images; generating a label corresponding to a characteristic of a proposed treatment schedule for the eye of the subject, the label being generated using the one or more cropped images; and and outputting the label. (Item 10) wherein the one or more cropped images include a plurality of cropped images, each of the plurality of cropped images including a different patch in the flattened OCT image, and generating the label; For each cropped image of the one or more cropped images, generating a patch-specific result using a patch-specific neural network; and and processing the patch-specific results using an ensemble model. (Item 11) 11. The system of claim 9 or 10, wherein the label indicates a frequency of treatment administration predicted to be sufficiently effective to prevent fluid from leaking from the eye's blood vessels between successive treatment administrations. (Item 12) 11. The system of claim 9 or 10, wherein the characteristic of the proposed treatment schedule indicates an interval between successive administrations of treatment. (Item 13) 13. The system according to any one of items 10 to 12, wherein the patch-specific neural network is trained using an image of a size corresponding to the cropped image. (Item 14) 14. The system according to any one of items 9 to 13, wherein the retinal layer comprises a retinal pigment epithelium layer. (Item 15) 15. The system of any one of items 9 to 14, wherein the proposed treatment schedule includes a proposed schedule for administering an anti-vascular endothelial growth factor. (Item 16) A computer program product tangibly embodied in a non-transitory machine-readable storage medium comprising instructions configured to cause one or more data processors to perform a set of operations, said set of operations comprising: accessing an optical coherence tomography (OCT) image corresponding to an eye of a subject having age-related macular degeneration; identifying a set of pixels within the OCT image corresponding to a retinal layer; flattening the OCT image based on the set of pixels; performing one or more cropping processes using the flattened OCT image to generate one or more cropped images; generating a label corresponding to a characteristic of a proposed treatment schedule for the eye of the subject, the label being generated using the one or more cropped images; and and outputting the label. (Item 17) wherein the one or more cropped images include a plurality of cropped images, each of the plurality of cropped images including a different patch in the flattened OCT image, and generating the label; For each cropped image of the one or more cropped images, generating a patch-specific result using a patch-specific neural network; and and processing the patch-specific results using an ensemble model. (Item 18) 18. The computer program product of item 16 or 17, wherein the label indicates a frequency of treatment administration or a time interval between successive treatment administrations that is predicted to be sufficiently effective to prevent fluid from leaking from the eye's blood vessels between successive treatment administrations. (Item 19) 19. The computer program product of item 17 or 18, wherein the patch-specific neural network is trained using an image of a size corresponding to the cropped image. (Item 19) 1. A method of treating an eye of a subject with age-related macular degeneration, comprising: accessing an OCT image depicting at least a portion of the eye of the subject with age-related macular degeneration; beginning processing the OCT image using a machine learning model, the processing including flattening the OCT image and processing at least a portion of the flattened OCT image using a neural network; accessing results of the processing of the OCT images, the results indicating a proposed treatment schedule for the eye of the subject; and and treating the eye of the subject according to the proposed treatment schedule. (Item 20) 20. The method of claim 19, wherein treating the eye of the subject according to the proposed treatment schedule comprises administering an anti-vascular endothelial growth factor to the eye according to the proposed treatment schedule.

Claims

1. 1. A computer-implemented method comprising: accessing an optical coherence tomography (OCT) image corresponding to an eye of a subject with age-related macular degeneration; identifying a set of pixels within the OCT image corresponding to a retinal layer; flattening the OCT image based on the set of pixels; performing one or more cropping processes using the flattened OCT image to generate one or more cropped images; inputting the one or more cropped images into one or more neural networks trained to generate predicted class labels corresponding to a frequency of administration of a treatment to the eye of the subject; and and outputting, by the one or more neural networks, the predicted class label corresponding to the administration frequency of the treatment.

2. wherein the one or more cropped images include a plurality of cropped images, each of the plurality of cropped images including a different patch in the flattened OCT image, and generating the predicted class label; for each cropped image of the one or more cropped images, generating a patch-specific result using a patch-specific neural network; and and processing the patch-specific results using multiple neural networks.

3. 3. The computer-implemented method of claim 1 or 2, wherein the frequency of administration of the treatment is predicted to be sufficiently effective to prevent fluid from leaking from the eye's blood vessels between successive treatment administrations.

4. 3. The computer-implemented method of claim 1 or 2, further comprising inputting the one or more cropped images into one or more neural networks trained to generate predicted class labels corresponding to intervals between successive administrations of the treatment to the eye of the subject.

5. A computer-implemented method according to claim 2 and any one of claims 3-4 when dependent on claim 2, wherein an image of a size corresponding to the cropped image is used to train the patch-specific neural network.

6. The computer-implemented method of claim 1 , wherein the retinal layer comprises a retinal pigment epithelium layer.

7. The computer-implemented method of any one of claims 1 to 6, wherein administering the treatment comprises administering an anti-vascular endothelial growth factor to the eye of the subject.

8. 1. A system comprising: one or more data processors; a non-transitory computer-readable storage medium containing instructions that, when executed on the one or more data processors, cause the one or more data processors to perform a set of operations, the set of operations including: accessing an optical coherence tomography (OCT) image corresponding to an eye of a subject with age-related macular degeneration; identifying a set of pixels within the OCT image corresponding to a retinal layer; flattening the OCT image based on the set of pixels; performing one or more cropping processes using the flattened OCT image to generate one or more cropped images; inputting the one or more cropped images into one or more neural networks trained to generate predicted class labels corresponding to a frequency of administration of a treatment to the eye of the subject; and and outputting, by the one or more neural networks, the predicted class label corresponding to an administration frequency of the treatment.

9. wherein the one or more cropped images include a plurality of cropped images, each of the plurality of cropped images including a different patch in the flattened OCT image, and generating the predicted class labels; For each cropped image of the one or more cropped images, generating a patch-specific result using a patch-specific neural network; and and processing the patch-specific results using multiple neural networks.

10. 10. The system of claim 8 or 9, wherein the frequency of administration of the treatment is predicted to be sufficiently effective to prevent fluid from leaking from the eye's blood vessels between successive treatment administrations.

11. 10. The system of claim 8 or 9, further comprising inputting the one or more cropped images into one or more neural networks trained to generate predicted class labels corresponding to intervals between successive administrations of the treatment to the eye of the subject.

12. The system of claim 9 and any one of claims 10-11 when dependent on claim 9, wherein an image of a size corresponding to the cropped image is used to train the patch-specific neural network.

13. The system of any one of claims 8 to 12, wherein the retinal layer comprises a retinal pigment epithelium layer.

14. The system of any one of claims 8 to 13, wherein the administering of the treatment comprises administering an anti-vascular endothelial growth factor to the eye of the subject.

15. A computer program product tangibly embodied in a non-transitory machine-readable storage medium comprising instructions configured to cause one or more data processors to perform a set of operations, said set of operations comprising: accessing an optical coherence tomography (OCT) image corresponding to an eye of a subject with age-related macular degeneration; identifying a set of pixels within the OCT image corresponding to a retinal layer; flattening the OCT image based on the set of pixels; performing one or more cropping processes using the flattened OCT image to generate one or more cropped images; inputting the one or more cropped images into one or more neural networks trained to generate predicted class labels corresponding to a frequency of administration of a treatment to the eye of the subject; and and outputting, by the one or more neural networks, the predicted class label corresponding to the administration frequency of the treatment.

16. wherein the one or more cropped images include a plurality of cropped images, each of the plurality of cropped images including a different patch in the flattened OCT image, and generating the predicted class label; For each cropped image of the one or more cropped images, generating a patch-specific result using a patch-specific neural network; and and processing the patch-specific results using multiple neural networks.

17. 17. The computer program product of claim 15 or 16, wherein the administration frequency of the treatment is predicted to be sufficiently effective to prevent fluid from leaking from the eye's blood vessels between successive treatment administrations.

18. 18. A computer program product according to claim 16 and claim 17 when dependent on claim 16, wherein the patch-specific neural network is trained using an image of a size corresponding to the cropped image.

19. 1. A system comprising: one or more data processors; a non-transitory computer-readable storage medium containing instructions that, when executed on the one or more data processors, cause the one or more data processors to perform a method of treating an eye of a subject with age-related macular degeneration, the method comprising: accessing an OCT image depicting at least a portion of the eye of the subject with age-related macular degeneration; beginning processing the OCT image using a machine learning model, the processing including flattening the OCT image and inputting at least a portion of the flattened OCT image into one or more neural networks trained to generate predicted class labels corresponding to a frequency of administration of a treatment to the eye of the subject; outputting, by the one or more neural networks, the predicted class label corresponding to the administration frequency of the treatment to the eye of the subject; and treating the eye of the subject according to the administration frequency of the treatment.

20. 20. The system of claim 19, wherein treating the eye of the subject according to the administration frequency of the treatment comprises administering an anti-vascular endothelial growth factor to the eye according to the administration frequency of the treatment.

Images