Prognostic model for predicting fibrosis development
Through optical coherence tomography (OCT) image data and machine learning models, retinal characteristics are automatically analyzed, and the accuracy and efficiency of retinal fibrosis prediction in the prior art are solved, achieving more efficient and accurate predictions, supporting personalized treatment and clinical trials.
Patent Information
- Application Number
- JP2024536115
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2022-04-13
- Filing Date
- 2022-12-16
- Publication Date
- 2025-02-04
AI Technical Summary
The prior art is difficult to accurately and rapidly predict the occurrence of retinal fibrosis, especially in neovascular macular degenerative lesions (NAMD), resulting in vision loss, and existing methods such as fluorescein angiography (FFA) are highly invasive and rely on manual evaluation and are inefficient.
Using optical coherence tomography (OCT) image data, combined with machine learning models and deep learning models, automated analysis of retinal features and predict the risk of retinal fibrosis, including through the application of segmentation models and feature-based models.
Improve the accuracy and efficiency of fibrosis prediction, reduce the need for computing resources, reduce the dependence on expert knowledge, and support the screening of personalized treatment strategies and clinical trials.
Smart Images

Figure 2025503447000001_ABST
Abstract
Description
[Technical field]
[0001] Inventor: Julio Hernandez Sanchez, Andreas Maunz, Siqing Yu, Beatriz Garcia Garcia CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 63 / 330,756, filed April 13, 2022, entitled "Prognostic Models for Predicting Fibrosis Development," and to U.S. Provisional Patent Application No. 63 / 290,628, filed December 16, 2021, entitled "Prognostic Models for Predicting Fibrosis Development," each of which is incorporated by reference in its entirety herein.
[0002] Field The present disclosure relates generally to predicting the onset of fibrosis, and more particularly, to methods and systems for automating the prediction of the onset of fibrosis using machine learning. [Background technology]
[0003] background Age-related macular degeneration (AMD) remains the most frequent cause of irreversible blindness in people over 50 years of age in developed countries. Neovascular AMD (nAMD) is an advanced form of AMD. The introduction of anti-vascular endothelial growth factor (anti-VEGF) therapy has significantly improved the prognosis of nAMD. However, the majority of patients suffer from irreversible vision loss despite treatment. In many cases, this vision loss is due to irreversible changes, such as the development of fibrosis.
[0004] Fibrosis is believed to be the result of an abnormal wound healing process that may be characterized by the deposition of collagen fibers, which dramatically changes the structure and function of different retinal layers.However, the pathophysiology of retinal fibrosis is complex and not fully understood, making it difficult to develop specific treatments and identify reliable biomarkers.Currently available methods for detecting biomarkers that predict the development of fibrosis include manual evaluation of images by human evaluators, making detection less accurate, less efficient, and slower than desired. Summary of the Invention
[0005] overview In one or more embodiments, a method for predicting the onset of fibrosis is provided. Optical coherence tomography (OCT) image data of a subject's retina with neovascular age-related macular degeneration (nAMD) can be received. The OCT image data is processed using a model system including a machine learning model to generate a prediction output. A final output is generated based on the prediction output, and the final output indicates the risk of developing fibrosis in the retina.
[0006] In one or more embodiments, a method for predicting the onset of fibrosis is provided. Optical coherence tomography (OCT) image data of a retina of a subject having neovascular age-related macular degeneration (nAMD) is received. The OCT image data is segmented using a segmentation model to generate segmented image data. The segmented image data is processed using a deep learning model to generate a prediction output. Based on the prediction output, a final output is generated that indicates a risk of the onset of fibrosis in the retina.
[0007] In one or more embodiments, a method for predicting the onset of fibrosis is provided. At least one of clinical data or retinal characteristic data is received for a retina of a subject with neovascular age-related macular degeneration (nAMD). At least one of the clinical data or retinal characteristic data is processed using a regression model to generate a prediction output. A final output is generated that indicates the risk of the onset of fibrosis in the retina based on the prediction output.
[0008] In some embodiments, a system is provided that includes one or more data processors and a non-transitory computer-readable storage medium that includes instructions that, when executed on the one or more data processors, cause the one or more data processors to perform some or all of the methods disclosed herein.
[0009] In some embodiments, a computer program product is provided that is tangibly embodied in a non-transitory machine-readable storage medium and includes instructions configured to cause one or more data processors to perform some or all of the methods disclosed herein. [Brief description of the drawings]
[0010] The present disclosure is described in conjunction with the accompanying drawings, in which:
[0011] [Figure 1] FIG. 1 is a block diagram of a prediction system 100 according to one or more embodiments.
[0012] [Diagram 2] 2 is a block diagram of an example implementation of the model system of FIG. 1 according to one or more embodiments.
[0013] [Diagram 3] 2 is a block diagram of an example implementation of the model system of FIG. 1 according to one or more embodiments.
[0014] [Figure 4] 1 is a flow chart of a process for predicting the onset of fibrosis according to one or more embodiments.
[0015] [Diagram 5] 1 is a flowchart of a process for predicting the onset of fibrosis using OCT image data, according to one or more embodiments.
[0016] [Figure 6] 1 is a flow chart of a process for predicting the onset of fibrosis according to one or more embodiments.
[0017] [Figure 7] 1 is a flow chart of a process for predicting the onset of fibrosis according to one or more embodiments.
[0018] [Figure 8] 1 is an OCT image according to one or more embodiments.
[0019] [Figure 9] 1 is a segmented image according to one or more embodiments.
[0020] [Figure 10] 1 is a table comparing statistical results of feature-based models using clinical data according to one or more embodiments.
[0021] [Figure 11] 1 is a table comparing statistical results of feature-based models using retinal features derived from OCT image data, in accordance with one or more embodiments.
[0022] [Figure 12] 1 is a table comparing statistical results of deep learning models using OCT image data and segmented image data, according to one or more embodiments.
[0023] [Figure 13] 1 is a table comparing statistical results of deep learning models using OCT image data and segmented image data in combination with clinical data, according to one or more embodiments.
[0024] [Figure 14]FIG. 1 is a block diagram of a computer system according to one or more embodiments.
[0025] In the accompanying drawings, similar components and / or features may have the same reference label. Furthermore, various components of the same type may be distinguished by following the reference label with a dash and a second label that distinguishes the similar components. If only a first reference label is used in this specification, the description is applicable to any one of the similar components having the same first reference label, regardless of the second reference label. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0026] Detailed Description I. Overview The embodiments described herein recognize that it may be desirable to have a method and system for predicting the onset of fibrosis in neovascular age-related macular degeneration (nAMD) subjects that is less invasive, more efficient, and / or faster than currently available methods and systems. Onset of fibrosis may include the onset of fibrosis and may include the continued progression of fibrosis. Since fibrosis can lead to irreversible vision loss, and there is currently no treatment that specifically targets fibrosis once it has developed, it may be important to predict whether and when a subject undergoing or receiving treatment for nAMD will develop fibrosis.
[0027] Typically, classical choroidal neovascularization (CNV) has been used as a prognostic biomarker for the development of fibrosis. The type and size of CNV are traditionally detected by manual observation of dye leakage in images generated via fluorescein fundus angiography (FA), which may also be called fluorescein fundus angiography (FFA). However, FA (or FFA) imaging is invasive, and using images from such imaging modalities may be more burdensome than desired. For example, interpreting FA images to detect fibrosis currently relies on human evaluators with the necessary expertise or training.
[0028] Optical coherence tomography (OCT) imaging may be used to improve the diagnosis and follow-up of nAMD patients at risk for fibrosis, as OCT imaging is minimally invasive. In addition to being minimally invasive, it is easier to obtain OCT images, as technician training that may be required is reduced. Furthermore, OCT imaging may allow for both qualitative and quantitative information to be obtained. Thus, the embodiments recognize that it may be desirable to have a method and system for automating the prediction of fibrosis onset via OCT images. Various morphological features seen in OCT images are associated with an increased risk of fibrosis onset, including, but not limited to, subretinal hyperluminal material (SHRM), subfoveal retinal exudate (SRF), pigment epithelial detachment (PED), and foveal retinal thickness.
[0029] Thus, the embodiments described herein provide methods and systems for automating prediction of fibrosis onset using OCT images and machine learning. The OCT images may be, for example, baseline OCT images. In one or more embodiments, deep learning models are used to process OCT images or segmented images (e.g., segmentation masks) developed from OCT images to predict fibrosis. These segmented images may be generated using trained deep learning models. These deep learning models may provide similar or improved accuracy of fibrosis prediction compared to using manual assessment of CNV type and size via FA images by human raters. Furthermore, using these deep learning models to predict fibrosis may be easier, faster, and more efficient than using FA images or manual grading. Still further, the use of the deep learning models described herein may improve fibrosis prediction in a manner that reduces the amount of computing resources required.
[0030] In one or more embodiments, feature-based modeling is used to process retinal feature data extracted from segmented images to predict fibrosis. These segmented images may be generated from the same trained deep learning model as the segmented images described above with respect to the segmented images for the deep learning model approach. These feature-based models may provide similar or improved accuracy for fibrosis prediction compared to using manual assessment of CNV type and size via FA images by human raters. Furthermore, using these feature-based models to predict fibrosis may be easier, faster, and more efficient than using FA images or manual grading. Still further, using the feature-based models described herein may improve fibrosis prediction in a manner that reduces the amount of computing resources required.
[0031] In some embodiments, in addition to the OCT image data, segmented image data, and / or retinal feature data described above, clinical data may be used, which may be, for example, baseline clinical data including values of various clinical variables such as age, visual acuity (e.g., visual acuity measurement such as best corrected visual acuity (BCVA)), or CNV type determined from the FA images, but is not limited to this.
[0032] In various embodiments, machine learning models may process the OCT images, segmented images, and / or retinal feature data to detect the presence of CNV and classify the CNV by its type. These machine learning models may detect the type of CNV with improved accuracy compared to manual evaluation of FA images by human raters. Furthermore, using machine learning models to detect the type of CNV may reduce the amount of time and computing resources required to detect the type of CNV.
[0033] Automated fibrosis detection using the machine learning-based methods and systems described herein can guide prognosis and aid in the development of new treatment strategies for nAMD and / or fibrosis. Furthermore, automated fibrosis prediction can enable better stratification and selection of subjects for clinical trials to ensure richer and / or more accurate population selection for clinical trials. Furthermore, automated fibrosis prediction can enable more accurate assessment of treatment response. For example, using machine learning models (e.g., deep learning and feature-based) as described herein to predict the onset of fibrosis can help optimize the use of available medical resources, improve treatment efficacy, and thereby improve overall subject (e.g., patient) health care.
[0034] Recognizing and taking into account the importance and usefulness of the methodologies and systems that can provide the above improvements, the embodiments described herein provide machine learning models for improving the accuracy, speed, efficiency, and ease of predicting fibrosis onset in subjects diagnosed with and / or undergoing treatment for nAMD.Furthermore, the methods and systems described herein may enable less invasive methods of predicting fibrosis onset, reducing the level of expertise or expert training required to perform the prediction.
[0035] II. Exemplary Systems for Predicting the Development of Fibrosis in nAMD II.A. System Overview Referring now to the drawings, FIG. 1 is a block diagram of a prediction system 100 according to one or more embodiments. The prediction system 100 may be used to predict the onset of ocular fibrosis in subjects diagnosed with neovascular age-related macular degeneration (nAMD). In one or more embodiments, the prediction system 100 includes a computing platform 102, a data storage 104, and a display system 106. The computing platform 102 may take a variety of forms. In one or more embodiments, the computing platform 102 includes a single computer (or computer system) or multiple computers that communicate with each other. In other examples, the computing platform 102 takes the form of a cloud computing platform, a mobile computing platform (e.g., a smartphone, a tablet, etc.), or a combination thereof.
[0036] The data storage 104 and the display system 106 each communicate with the computing platform 102. In some examples, the data storage 104, the display system 106, or both may be considered part of the computing platform 102 or may be otherwise integrated with the computing platform 102. Thus, in some examples, the computing platform 102, the data storage 104, and the display system 106 may be separate components that communicate with each other, while in other examples some combination of these components may be integrated together.
[0037] The prediction system 100 includes a fibrosis predictor 110, which may be implemented using hardware, software, firmware, or a combination thereof. In one or more embodiments, the fibrosis predictor 110 is implemented on the computing platform 102.
[0038] The fibrosis predictor 110 receives and processes input data 112 to generate a final output 114. The final output 114 may be, for example, a binary classification indicating whether or not fibrosis is predicted to develop. The indication may be in terms of risk of developing fibrosis. For example, the binary classification may be a positive or negative prediction of fibrosis development, and may be a prediction of high or low risk. The prediction may be made for a future time point (e.g., 1 month, 2 months, 3 months, 4 months, 6 months, 8 months, 12 months, 15 months, 24 months, etc. after the first or more recent administration of treatment) or for an indefinite period of time. In other examples, the final output 114 may be a score indicating whether or not fibrosis is predicted to develop. For example, a score equal to or above a selected threshold (e.g., a threshold between 0.4 and 0.9) may indicate a positive prediction of fibrosis development, and a score below the selected threshold may indicate a negative prediction. In some cases, the score may be a probability value or a likelihood value of fibrosis developing.
[0039] The input data 112 may be data of a subject diagnosed with nAMD. The subject may have been previously treated with an nAMD treatment (e.g., an anti-VEGF therapy such as ranibizumab; an antibody therapy, e.g., faricimab, or some other type of treatment). In other embodiments, the subject may be treatment naive.
[0040] The input data 112 may include, for example, but is not limited to, at least one of optical coherence tomography (OCT) image data 116, segmented image data 118, retinal feature data 120, clinical data 122, or a combination thereof. In one or more embodiments, the input data 112 includes at least one of optical coherence tomography (OCT) image data 116, segmented image data 118, or retinal feature data 120, and optionally includes clinical data 122.
[0041] The OCT image data 116 may include, for example, one or more raw OCT images that have not been preprocessed, or one or more OCT images that have been preprocessed using one or more standardization or normalization procedures. The OCT images may take the form of, but are not limited to, a time-domain optical coherence tomography (TD-OCT) image, a spectral-domain optical coherence tomography (SD-OCT) image, a two-dimensional OCT image, a three-dimensional OCT image, an OCT angiography (OCT-A) image, or a combination thereof. SD-OCT, also known as Fourier-domain OCT, may be referenced with respect to the embodiments described herein, although other types of OCT images are contemplated for use with the methodologies and systems described herein. Thus, the description of the embodiments with respect to images, image types, and techniques provides only non-limiting examples of such images, image types, and techniques.
[0042] The segmented image data 118 may include one or more segmented images generated through retinal segmentation. Retinal segmentation includes detection and identification of one or more retinal (e.g., retina-related) elements of the retinal image. The segmented image uses one or more graphical indicators to identify one or more retinal (e.g., retina-related) elements of the segmented image. The segmented image may be a representation of an OCT image that identifies one or more retinal elements, or may be an OCT image in which one or more retinal elements have been identified.
[0043] For example, one or more color indicators, shape indicators, pattern indicators, shading indicators, lines, curves, markers, labels, tags, text features, other types of graphical indicators, or combinations thereof may be used to identify portions of the image (e.g., pixel by pixel) that are identified as retinal elements. As one specific example, a group of pixels may be identified as acquiring a particular retinal fluid (e.g., intraretinal or subretinal exudate). The segmented image may identify this group of pixels using a color indicator. For example, each pixel of the group of pixels may be assigned a color that is unique to a particular retinal fluid, thereby assigning each pixel to a particular retinal fluid. As another example, the segmented image may identify the group of pixels by applying a patterned region or shape (continuous or discontinuous) over the group of pixels.
[0044] The retinal element may be composed of at least one of a retinal layer element or a retinal pathology element. The detection and identification of one or more retinal layer elements may be referred to as segmenting a layer element (or a retinal layer element). The detection and identification of one or more retinal pathology elements may be referred to as segmenting a pathology element (or a retinal pathology element).
[0045] The retinal layer element may be, for example, a retinal layer or a boundary associated with a retinal layer. Examples of retinal layers include, but are not limited to, an inner limiting membrane (ILM) layer, a retinal nerve fiber layer, a ganglion cell layer, an inner plexiform layer, an inner nuclear layer, an outer plexiform layer, an outer nuclear layer, an outer limiting membrane (ELM) layer, a photoreceptor layer, a retinal pigment epithelium (RPE) layer, an RPE detachment, a Bruch's membrane (BM) layer, a choriocapillaris layer, a choroidal stromal layer, an ellipsoid zone (EZ), and other types of retinal layers. In some cases, a retinal layer may be composed of one or more layers. As an example, a retinal layer may be the interface between the outer plexiform layer and the fiber layer of Henle (OPL-HFL). A boundary associated with a retinal layer may be, for example, an inner boundary of a retinal layer, an outer boundary of a retinal layer, a boundary associated with a pathological feature of a retinal layer (e.g., an inner or outer boundary of a retinal layer detachment), or some other type of boundary. For example, the boundary can be the inner boundary of a peeling layer of the RPE (IB-RPE), the outer boundary of a peeling layer of the RPE (OB-RPE), or another type of boundary.
[0046] Retinal pathology elements may include, for example, fluid (e.g., fluid pockets), cells, solid material, or combinations thereof, indicative of retinal pathology (e.g., a disease or condition such as AMD or diabetic macular edema). For example, the presence of certain retinal fluid may be a sign of nAMD. Examples of retinal pathology elements include, but are not limited to, intraretinal exudate (IRF), subretinal exudate (SRF), fluid associated with pigment epithelial detachment (PED), hyperreflective material (HRM), subretinal hyperluminal material (SHRM), intraretinal hyperreflective material (IHRM), hyperreflective foci (HRF), retinal fluid pockets, drusen, and fibrosis. In some cases, a retinal pathology element may be a disruption (e.g., discontinuity, delamination, loss, etc.) of a retinal layer or zone. For example, the disruption may be a disruption of the ellipsoid region, ELM, RPE, or another layer or region. The disruption may represent damage or loss of cells (e.g., photoreceptors) in the area of the disruption. In some examples, the retinal pathology element may be clear IRF, cloudy IRF, clear SRF, cloudy SRF, any other type of clear retinal fluid, any other type of cloudy retinal fluid, or a combination thereof.
[0047] In one or more embodiments, the segmented image data 118 may be generated via a deep learning model. The deep learning model may be comprised of a convolutional neural network system comprised of one or more neural networks. Each or at least one of these one or more neural networks may itself be a convolutional neural network.
[0048] The retinal feature data 120 may include, for example, but not limited to, feature data extracted from the segmented image data 118. For example, feature data may be extracted for one or more retinal elements identified in the segmented image data 118. This feature data may include any number or combination of feature (e.g., quantitative feature) values. These features may include features related to pathology, volumetric features related to layers, thickness features related to layers, or combinations thereof. Examples of features include, but are not limited to, maximum retinal layer thickness, minimum retinal layer thickness, average retinal layer thickness, maximum height of a boundary related to a retinal layer, volume of a retinal fluid pocket, length of a fluid pocket, width of a fluid pocket, number of retinal fluid pockets, and multiple hyperreflective foci. Thus, at least some of the features may be volumetric features. For example, feature data may be derived for each selected OCT image (e.g., a single OCT B-scan) and then combined to form a value for the entire volume. In one or more embodiments, between 1 and 200 features may be included in the retinal feature data 120.
[0049] The clinical data 122 may include, for example, but are not limited to, age, visual acuity measurements, choroidal neovascularization (CNV) type, or a combination thereof. The visual acuity measurements may be, for example, measurements of best-corrected visual acuity (BCVA). The CNV type may be a type identification based on evaluation of fluorescein angiography (FA) image data. The CNV type may be, for example, occult CNV, predominantly classical CNV, minimal classical CNV, or retinal angiomatous proliferation (RAP). In some cases, "classical CNV" may be used as a CNV type that captures both predominantly classical CNV or minimal classical CNV. In some cases, the CNV type is identified based on a numbering scheme (e.g., type 1 refers to occult CNV, type 2 refers to classical CNV, and type 3 refers to RAP). In one or more embodiments, at least some of the clinical data 122 may be from a baseline time point. For example, the CNV type and / or BCVA may be obtained for the baseline time point. The baseline time point can be a time point after nAMD diagnosis but immediately prior to treatment (e.g., before the first dose), a period after the first dose of treatment (e.g., 6 months, 9 months, 12 months, 15 months, etc. after the first dose), or another type of baseline time point.
[0050] The fibrosis predictor 110 uses a model system 124 to process input data 112, which may include any one or more of the different types of data described above, to generate a final output 114. The model system 124 may be implemented using different types of architectures. The model system 124 may include a set of machine learning models 126. One or more of the set of machine learning models 126 may receive the input data 112 (e.g., some or all of the input data 112) for processing. The data included in the input data 112 may vary based on the type of architecture used for the model system 124. Examples of various types of architectures that may be used for the model system 124 and various types of data that may be included in the input data 112 are described in more detail below in Sections II.B. and II.C.
[0051] In one or more embodiments, the final output 114 may include other types of information. For example, in some cases, the final output 114 may include a clinical trial recommendation, a treatment recommendation, or both. A clinical trial recommendation may be a recommendation to include or exclude a subject from a clinical trial. A treatment recommendation may be a recommendation to change the type of treatment, adjust the treatment regimen (e.g., frequency of injections, dosage, etc.), or both.
[0052] At least a portion of the final output 114, or a graphical representation of at least a portion of the final output 114, may be displayed on a display system 106. In some embodiments, at least a portion of the final output 114, or a graphical representation of at least a portion of the final output 114, is transmitted to a remote device 128 (e.g., a mobile device, a laptop, a server, the cloud, etc.).
[0053] II.B. Fibrosis Predictor Using Deep Learning Models Figure 2 is a block diagram of an example implementation of the model system 124 of Figure 1, in accordance with one or more embodiments. With continued reference to Figure 1, the model system 124 of Figure 2 will be described. The model system 124 includes a deep learning model 200, which may be an example implementation of a machine learning model in the set of machine learning models 126. The deep learning model 200 may receive a model input 202 and generate a predicted output 204.
[0054] In one or more embodiments, the model input 202 is formed using at least a portion of the input data 112 described above with respect to Figure 1. In some embodiments, the model input 202 includes OCT image data 116. In other embodiments, the model input 202 includes OCT image data 116 and at least a portion of the clinical data 122 (e.g., baseline CNV type, baseline visual acuity measurements, age, or a combination thereof).
[0055] In some embodiments, the model input 202 includes the segmented image data 118. In other embodiments, the model input 202 includes the segmented image data 118 and at least some of the clinical data 122 (e.g., baseline CNV type, baseline visual acuity measurements, age, or a combination thereof).
[0056] The deep learning model 200 may be implemented using a binary classification model. In one or more embodiments, the deep learning model 200 is implemented using a convolutional neural network system, which may be composed of one or more neural networks. Each or at least one of these one or more neural networks may itself be a convolutional neural network. In some embodiments, the deep learning model 200 is implemented using the ResNet-50 model, which is a 50-layer deep convolutional neural network, or a modified form of ResNet-50.
[0057] If the model input 202 includes at least some clinical data 122 in addition to either the OCT image data 116 or the segmented image data 118, the deep learning model 200 may use a modified form of a convolutional neural network to concatenate a vector of clinical data (clinical variables) to the OCT image data 116 or the segmented image data 118, respectively. As an example, if the deep learning model 200 is implemented using ResNet-50, a first portion of the deep learning model 200 includes ResNet-50 without its top layer. This first portion of the deep learning model 200 is used to generate a first intermediate output based on the OCT image data 116 or the segmented image data 118. A second portion of the deep learning model (e.g., replacing the top layer of the ResNet-50) may include a custom dense layer portion (e.g., one or more dense layers). The set of vectors of clinical variables (e.g., baseline CNV type, baseline visual acuity, and / or baseline age) is concatenated with the first intermediate output generated by the first portion of the deep learning model 200 to form a second intermediate output. The second intermediate output is sent to the custom dense layer portion of the deep learning model 200. In some cases, the output of the ResNet-50 in the first portion of the deep learning model 200 may be passed through an average pooling layer to form the first intermediate output.
[0058] The deep learning model 200 outputs a predicted output 204 based on the model input 202. The fibrosis predictor 110 may use the predicted output 204 to form the final output 114. For example, the predicted output 204 may be the likelihood that an eye of a subject diagnosed with nAMD will develop fibrosis. In one or more embodiments, the predicted output 204 is a binary classification indicating whether or not the development of fibrosis is predicted. In such an example, the final output 114 may include the predicted output 204. In other embodiments, the predicted output 204 takes the form of a score (e.g., a probability distribution value or a likelihood value) indicating whether or not the development of fibrosis is predicted. In such an example, the final output 114 may include a binary classification formed based on the predicted output 204 and / or the score. For example, the fibrosis predictor 110 may generate the final output 114 as a binary classification or indication based on whether or not the score generated by the deep learning model is above a selected threshold (e.g., a threshold between 0.4 and 0.9).
[0059] In some embodiments, the model system 124 may further include a segmentation model 206. The segmentation model 206 may receive the OCT image data 116 as an input and may generate segmented image data, such as segmented image data 118. The segmentation model 206 is used to automate the segmentation of the OCT image data 116. The segmentation model 206 may include, for example, but is not limited to, a deep learning model. The segmentation model 206 may include, for example, one or more neural networks. In one or more embodiments, the segmentation model 206 takes the form of a U-net.
[0060] The deep learning model 200 may be trained using training data 208 of subjects who have been diagnosed and treated for nAMD. The training data 208 may include training clinical data 210 and training image data 212. The training image data 212 may include or be generated from OCT images at future time points after treatment initiation. For example, the OCT images may be generated at 6 month intervals, 9 month intervals, 12 month intervals, 24 month intervals, or some other time interval after treatment initiation. The development of fibrosis at this future time point may be evaluated by a human assessor.
[0061] II.C. Fibrosis predictor using feature-based models Figure 3 is a block diagram of an example implementation of model system 124 of Figure 1, in accordance with one or more embodiments. With continued reference to Figures 1 and 2, model system 124 of Figure 3 will be described. Model system 124 includes a feature-based model 300, which may be an example implementation of a machine learning model in set of machine learning models 126. Feature-based model 300 may receive model input 302 and generate predicted output 304.
[0062] In one or more embodiments, the model input 302 is formed using some of the input data 112 described above with respect to Figure 1. For example, the model input 302 includes retinal feature data 120. In other embodiments, the model input 302 includes retinal feature data 120 and at least some of the clinical data 122 (e.g., baseline CNV type, baseline visual acuity measurement, age, or a combination thereof). In yet other embodiments, the model input 302 includes at least some of the clinical data 122 including baseline CNV type, and baseline visual acuity measurement, age, or both.
[0063] The feature-based model 300 may be a regression model (or algorithm). For example, the feature-based model 300 may be a logistic regression model, a linear regression model, or some other type of regression model. The feature-based model 300 may generate a prediction output 304 in the form of a score (e.g., a probability or likelihood value). A score above a selected threshold (e.g., 0.5, 0.6, 0.7, or some other value between 0.4 and 0.9) may be a score that is positively indicative of the onset of fibrosis. A score below the selected threshold may indicate that fibrosis is not predicted to develop.
[0064] In one or more embodiments, the feature-based model 300 may be a regression model trained using one or more regularization techniques to reduce overfitting. These regularization techniques may include Ridge regularization, Lasso regularization, Elastic Net regularization, or a combination thereof. For example, the number of features used in the feature-based model may be reduced to those that have importance above a threshold to the prediction output 304. In some cases, this type of training may simplify the feature-based model 300 to allow for shorter run times. For example, the Lasso regularization technique may be used to reduce the number of features used in the regression model and / or identify important features (e.g., features that have the highest importance to the predictions generated by the regression model). The Elastic Net regularization technique depends on both the amount of overall regularization (lambda) and the mix of Lasso and Ridge regularization (alpha). The cross-validation strategy may include a 5-fold or 10-fold cross-validation strategy. The parameters alpha and lambda may be selected to minimize the cross-validated deviance.
[0065] In one or more embodiments, the model input 302 includes three baseline clinical variables from the clinical data 122, including CNV type, BCVA, and age. In one or more embodiments, the model input 302 includes SHRM grade (e.g., graded according to a centralized grading protocol), PED grade (e.g., graded according to a centralized grading protocol), and maximum SRF height for each of the 1 mm and 3 mm foveal regions. In one or more embodiments, the model input 302 includes maximum thickness between the OPL-HFL layer and the RPE, total neuroretinal thickness from the ILM layer to the RPE layer, or both. In one or more embodiments, the model input 302 includes baseline CNV type, baseline age, and baseline BCVA from the clinical data 122, as well as central retinal thickness (CRT), subfoveal choroidal thickness (SFCT), grade of PED, maximum SRF height, and grade of SHRM from the retinal feature data 120. In other embodiments, the model input 302 includes CRT, SFCT, PED, SRF, and SHRM.
[0066] In some embodiments, the model system 124 includes a segmentation model 206, a feature extraction model 306, or both. The segmentation model 206 may be the same pre-trained model described in FIG. 2. The segmentation model 206 may be used to generate segmented image data 118 from the OCT image data 116 provided in the model input 302. The feature extraction model 306 may be an example of an implementation of a machine learning model in the set of machine learning models 126 and may be used to generate retinal feature data 120 based on the segmented image data 118 included in the model input 302 or generated by the segmentation model 206.
[0067] In one or more embodiments, the CNV type may be a type of feature included in the retinal feature data 120. For example, the CNV type may be determined by the feature extraction model 306. In other embodiments, the model system 124 includes a CNV classifier 308. The CNV classifier 308 may be an example of an implementation of a machine learning model in the set of machine learning models 126. For example, the CNV classifier 308 may include a machine learning model (e.g., a deep learning model including one or more neural networks) capable of detecting the CNV type using the OCT image data 116 instead of the FA image. This CNV type may be referred to as a model-generated CNV or an OCT-based CNV type. In some cases, this CNV type is sent directly from the CNV classifier 308 to the feature-based model 300 for processing.
[0068] The feature-based model 300 outputs a predicted output 304 based on the model input 302. The fibrosis predictor 110 may use the predicted output 304 to form the final output 114. For example, the predicted output 304 may be the likelihood that an eye of a subject diagnosed with nAMD will develop fibrosis. In one or more embodiments, the predicted output 304 is a binary classification indicating whether or not the onset of fibrosis is predicted. In such an example, the final output 114 may include the predicted output 304. In other embodiments, the predicted output 304 takes the form of a score (e.g., a probability distribution value or a likelihood value) indicating whether or not the onset of fibrosis is predicted. In such an example, the final output 114 may include a binary classification formed based on the predicted output 304 and / or the score. For example, the fibrosis predictor 110 may generate the final output 114 as a binary classification or indication based on whether or not the score generated by the deep learning model is above a selected threshold (e.g., a threshold between 0.4 and 0.9).
[0069] The feature-based model 300 may be trained using training data 208 of subjects who have been diagnosed with and treated for nAMD. The training data 208 may include the same training data described with respect to FIG.
[0070] III. Exemplary Methodologies for Predicting Fibrosis Onset 4 is a flow chart of a process 400 for predicting the onset of fibrosis according to one or more embodiments. In one or more embodiments, the process 400 may be implemented using the prediction system 100 described in FIG. 1 and / or the fibrosis predictor 110 described in FIGS. 1-3. The process 400 includes various steps and may be described with continued reference to FIGS. 1-3. One or more steps not explicitly shown in FIG. 4 may be included before, after, between, or as part of a step of the process 400. In some embodiments, the process 400 may begin with step 402.
[0071] Step 402 includes receiving optical coherence tomography (OCT) image data for a retina of a subject having neovascular age-related macular degeneration (nAMD). The OCT image data may be, for example, OCT image data 116 described with reference to FIGS.
[0072] Step 404 includes processing the OCT image data using a model system including a machine learning model to generate a predicted output. The model system may be, for example, the model system 124 described with respect to FIGS. 1-3. The machine learning model may be, for example, the deep learning model 200 of FIG. 2 or the feature-based model 300 of FIG. 3. In some cases, the model system includes a segmentation model (e.g., the segmentation model 206 of FIGS. 2-3). In some cases, the model system includes a feature extraction model (e.g., the feature extraction model 306 of FIG. 3). In some cases, the model system includes a CNV classifier (e.g., the CNV classifier 308 of FIG. 3).
[0073] The prediction output generated in step 404 may be, for example, prediction output 204 of FIG. 2 or prediction output 304 of FIG. 3. The prediction output may be the likelihood that a retina of a subject diagnosed with nAMD will develop fibrosis. In one or more embodiments, the prediction output is a binary classification indicating whether or not fibrosis is predicted to develop. For example, the binary classification may indicate a low or high risk of developing fibrosis. The prediction output may take the form of a score (e.g., a probability distribution value or a likelihood value) indicating whether or not fibrosis is predicted to develop.
[0074] The processing of step 404 may be performed in a variety of ways. In one or more embodiments, the machine learning includes a deep learning model (e.g., at least one neural network, such as a convolutional neural network). The deep learning model may process the OCT image data and generate a predictive output. The deep learning model may be, for example, a binary classification model. The OCT image data may be raw OCT image data generated by an OCT imager, or may be a pre-processed form of the raw OCT image data (e.g., pre-processed by any number of standardization or normalization procedures).
[0075] In other embodiments, step 404 includes segmenting the OCT image data via a segmentation model (e.g., segmentation model 206 of FIGS. 2-3) to form segmented image data. The OCT image data may be raw OCT image data generated by an OCT imager, or may be a pre-processed form of the raw OCT image data (e.g., pre-processed by any number of standardization or normalization procedures). The segmented image may then be processed by a deep learning model to generate a predicted output.
[0076] In yet other embodiments, step 404 includes the machine learning model including a feature-based model (e.g., feature-based model 300 of FIG. 3 ), and the model system may further include a feature extraction model (e.g., feature extraction model 306 of FIG. 3 ), a CNV classifier 308, or both. The feature extraction model may receive the segmented image data from the segmentation model and may use the segmented image data to extract retinal feature data (e.g., retinal feature data 120 of FIGS. 1 and 3 ) from the segmented image data. The retinal feature data may include at least one first feature value associated with at least one retinal layer element or a second feature value associated with at least one retinal pathology element. The retinal feature data may include, for example, 1 to 200 retinal features (or retinal feature values).
[0077] The machine learning model of step 404 can also be used to process clinical data (e.g., clinical data 122 of FIGS. 1-3) in addition to the OCT image data, segmented image data, or retinal feature data. The clinical data may include baseline clinical data. For example, the clinical data may include a baseline choroidal neovascularization (CNV) type, a baseline visual acuity measurement, or a baseline age. The baseline visual acuity measurement may be a baseline BCVA or some other type of visual acuity measurement.
[0078] When the machine learning model includes a deep learning model for processing either the OCT image data or the segmented image data, the deep learning model may include, for example, a convolutional neural network (CNN) system that may include ResNet-50 or a modified form of ResNet-50. In one or more embodiments, a first portion of the deep learning system (e.g., ResNet-50 without one or more top layers) is used to process the OCT image data or the segmented image data to generate a first intermediate output. A second portion of the deep learning model (e.g., a replacement of one or more top layers of ResNet-50) may include a custom dense portion (e.g., one or more dense layers). A set of vectors of one or more clinical variables included in the clinical data may be concatenated to the first intermediate output to form a second intermediate output. The second intermediate output may be processed using a second portion of the deep learning model, the custom dense layer portion, to generate a predicted output.
[0079] Step 406 includes generating a final output indicating a risk of developing fibrosis in the retina based on the prediction output. For example, the final output, which may be the final output 114 of FIGS. 1-3, may include the prediction output and / or a binary classification formed based on the prediction output. In some cases, the final output, which may be the final output 114 of FIGS. 1-3, may be a report that includes other information in addition to the prediction output and / or the binary classification. For example, this other information may include a clinical trial recommendation to include or exclude the subject in a clinical trial based on either the prediction output or the binary classification. The information may include a treatment recommendation to change the type of treatment, adjust the subject's treatment regimen, or both, based on either the prediction output or the binary classification. In some embodiments, the final output includes at least some of the inputs used to generate the prediction output.
[0080] FIG. 5 is a flow chart of a process 500 for predicting the onset of fibrosis using OCT image data, according to one or more embodiments. In one or more embodiments, the process 500 may be implemented using the prediction system 100 described in FIG. 1 and / or the fibrosis predictor 110 described in FIGS. 1-2. The process 500 includes various steps and may be described with continued reference to FIGS. 1-2. One or more steps not explicitly shown in FIG. 5 may be included before, after, between, or as part of the steps of the process 500. In some embodiments, the process 500 may begin with step 502. The process 500 of FIG. 5 may be a more detailed version of the process 400 of FIG. 4 specific to the generation of a final output based on the OCT image data.
[0081] Step 502 includes receiving optical coherence tomography (OCT) image data for the retina of a subject having neovascular age-related macular degeneration (nAMD). The OCT image data may be, for example, the OCT image data 116 described with reference to FIGS. 1-2. The OCT image data may be raw OCT image data generated by an OCT imaging device or may be a pre-processed form of the raw OCT image data (e.g., pre-processed by any number of standardization or normalization procedures).
[0082] Step 504 includes processing the OCT image data using a deep learning model of the model system to generate a predicted output. The deep learning model may be, for example, the deep learning model 200 of FIG. 2. In one or more embodiments, the deep learning model includes a binary classification model. The deep learning model may include a convolutional neural network.
[0083] The prediction output generated in step 504 may be, for example, prediction output 204 of FIG. 2. The prediction output may be the likelihood that a retina of a subject diagnosed with nAMD will develop fibrosis. In one or more embodiments, the prediction output is a binary classification indicating whether or not fibrosis is predicted to develop. For example, the binary classification may indicate a low or high risk for developing fibrosis, a positive or negative prediction for developing fibrosis, or other types of binary classifications. The prediction output may take the form of a score (e.g., a probability distribution value or a likelihood value) indicating whether or not fibrosis is predicted to develop.
[0084] Step 506 includes generating a final output indicative of a risk of developing fibrosis in the retina based on the prediction output. The final output may be, for example, the final output 114 described with respect to Figures 1-2. The final output may be similar to the final output described with respect to step 406 of Figure 4.
[0085] In some embodiments, step 502 includes receiving clinical data (e.g., clinical data 122 of FIGS. 1 and 2) for processing. The clinical data may include at least one of a baseline CNV type, a baseline visual acuity measurement, or a baseline age. In these embodiments, once the clinical data is received in step 502, step 504 may include processing both the OCT image data and the clinical data using a model system to generate a predictive output.
[0086] FIG. 6 is a flow chart of a process 600 for predicting the onset of fibrosis according to one or more embodiments. In one or more embodiments, the process 600 may be implemented using the prediction system 100 described in FIG. 1 and / or the fibrosis predictor 110 described in FIGS. 1-2. The process 600 includes various steps and may be described with continued reference to FIGS. 1-2. One or more steps not explicitly shown in FIG. 6 may be included before, after, between, or as part of the steps of the process 600. In some embodiments, the process 600 may begin with step 602. The process 600 of FIG. 6 may be a more detailed version of the process 400 of FIG. 4 specific to the generation of a final output based on the segmented image data.
[0087] Step 602 may optionally include receiving an optical coherence tomography (OCT) image of the retina of a subject having neovascular age-related macular degeneration (nAMD). The OCT image data may be, for example, OCT image data 116 described with reference to FIGS. 1-2. The OCT image data may be raw OCT image data generated by an OCT imager, or may be a pre-processed form of the raw OCT image data (e.g., pre-processed by any number of standardization or normalization procedures).
[0088] Step 604 may optionally include segmenting the OCT image data using a segmentation model to generate segmented image data. The segmentation model may be, for example, segmentation model 206 of FIG. 2. In one or more embodiments, the segmentation model includes a U-Net-based architecture that is pre-trained by training OCT image data including OCT images annotated via a human rater (e.g., a certified rater). The segmentation model may be trained to automatically segment one or more retinal pathology elements (e.g., SHRM, SRF, PED, IRF, etc.), one or more retinal layer elements (e.g., ILM, OPL-HFL, RPE, BM, etc.), or both.
[0089] Step 606 may include receiving the segmented image data at a deep learning model. The deep learning model may be, for example, deep learning model 200 of FIG. 2.
[0090] Step 608 may include processing the segmented image data using a deep learning model to generate a prediction output (e.g., prediction output 204 of FIG. 2). The prediction output may be the likelihood that a retina of a subject diagnosed with nAMD will develop fibrosis. In one or more embodiments, the prediction output is a binary classification indicating whether or not fibrosis is predicted to develop. For example, the binary classification may indicate a low or high risk for developing fibrosis, a positive or negative prediction for developing fibrosis, or other types of binary classifications. The prediction output may take the form of a score (e.g., a probability distribution value or a likelihood value) indicating whether or not fibrosis is predicted to develop.
[0091] Step 610 may include generating a final output indicative of a risk of developing fibrosis in the retina based on the prediction output. The final output may be, for example, the final output 114 described with respect to Figures 1-2. The final output may be similar to the final output described with respect to step 406 of Figure 4.
[0092] In some embodiments, step 602 includes receiving clinical data (e.g., clinical data 122 of FIGS. 1 and 2) for processing. The clinical data may include at least one of a baseline CNV type, a baseline visual acuity measurement, or a baseline age. In these embodiments, once the clinical data is received at step 602, step 608 may include processing both the segmented image data and the clinical data using a deep learning model to generate a predictive output.
[0093] FIG. 7 is a flow chart of a process 700 for predicting the onset of fibrosis according to one or more embodiments. In one or more embodiments, the process 700 may be implemented using the prediction system 100 described in FIG. 1 and / or the fibrosis predictor 110 described in FIGS. 1 and 3. The process 700 includes various steps and may be described with continued reference to FIGS. 1 and 3. One or more steps not explicitly shown in FIG. 7 may be included before, after, between, or as part of the steps of the process 700. In some embodiments, the process 700 may begin with step 702. The process 700 of FIG. 7 may be a more detailed version of the process 400 of FIG. 4 specific to the generation of a final output using a feature-based model.
[0094] Step 702 may optionally include receiving an optical coherence tomography (OCT) image of the retina of a subject having neovascular age-related macular degeneration (nAMD). The OCT image data may be, for example, OCT image data 116 described with reference to Figures 1 and 3. The OCT image data may be raw OCT image data generated by an OCT imager, or may be a pre-processed form of the raw OCT image data (e.g., pre-processed by any number of standardization or normalization procedures).
[0095] Step 704 may optionally include segmenting the OCT image data using a segmentation model to generate segmented image data (e.g., segmented image data 118 of FIGS. 1 and 3). The segmentation model may be, for example, segmentation model 206 of FIG. 3. In one or more embodiments, the segmentation model includes a U-Net-based architecture that is pre-trained by training OCT image data including OCT images annotated via a human rater (e.g., a certified rater). The segmentation model may be trained to automatically segment one or more retinal pathology elements (e.g., SHRM, SRF, PED, IRF, etc.), one or more retinal layer elements (e.g., ILM, OPL-HFL, RPE, BM, etc.), or both.
[0096] Step 706 optionally includes extracting retinal feature data from the segmented image data via a feature extraction model. The feature extraction model may be, for example, feature extraction model 306 of FIG. 3. The feature extraction model may receive the segmented image data from the segmentation model and may use the segmented image data to extract retinal feature data (e.g., retinal feature data 120 of FIGS. 1 and 3) from the segmented image data. The retinal feature data may include at least one first feature value associated with at least one retinal layer element or a second feature value associated with at least one retinal pathology element. The retinal feature data may include, for example, 1 to 200 retinal features (or retinal feature values).
[0097] Step 708 may optionally include identifying a choroidal neovascularization (CNV) type using a CNV classifier (e.g., CNV classifier 308). The CNV classifier may be implemented using, for example, but not limited to, a deep learning model that uses OCT image data to detect and identify the CNV type. This CNV type may be a model-generated CNV type and may differ from the baseline CNV type included in the clinical data (e.g., the CNV type is determined by a human assessor based on the FA image data).
[0098] Step 710 includes receiving at least one of retinal feature data, clinical data, or CNV type for processing. The CNV type of step 710 may be the model-generated CNV type identified in step 708. The retinal feature data may be the retinal feature data generated in step 706. The clinical data may be, for example, clinical data 122 of Figures 1 and 3. The clinical data may include at least one of a baseline CNV type, a baseline visual acuity measurement, or a baseline age.
[0099] Step 712 includes processing at least one of the clinical data, the retinal feature data, or the CNV type using a feature-based model to generate a prediction output. The feature-based model may be, for example, the feature-based model 300 of FIG. 3. The feature-based model may include, for example, a regression model. The CNV type of step 708 may be the model-generated CNV type identified in step 708.
[0100] The predictive output may be the likelihood that the retina of a subject diagnosed with nAMD will develop fibrosis. In one or more embodiments, the predictive output is a binary classification indicating whether or not fibrosis is predicted to develop. For example, the binary classification may indicate a low or high risk for fibrosis development, a positive or negative prediction for fibrosis development, or other types of binary classification. The predictive output may take the form of a score (e.g., a probability distribution value or a likelihood value) indicating whether or not fibrosis is predicted to develop.
[0101] Step 714 includes generating a final output indicative of a risk of developing fibrosis in the retina based on the prediction output. The final output may be, for example, the final output 114 described with respect to Figures 1 and 3. The final output may be similar to the final output described with respect to step 406 of Figure 4.
[0102] IV. Exemplary Images 8 is an OCT image according to one or more embodiments. OCT image 800 is an example of an implementation of an OCT image that may be included in OCT image data 116 described above in Sections II.A. and II.B. OCT image 800 may be a single OCT B-scan. In one or more embodiments, OCT image 800 may be processed as part of the model input 202 of deep learning model 200 of FIG. 2. In one or more embodiments,
[0103] 9 is a segmented image according to one or more embodiments. Segmented image 900 is an example of an implementation of a segmented image that may be included in segmented image data 118 described above in Sections II.A. and II.B. Segmented image 900 may be a representation of an OCT image (e.g., OCT image 800 of FIG. 8) with multiple masks 902 superimposed on the representation. In another example, segmented image 900 may be an OCT image (e.g., OCT image 800 of FIG. 8) with multiple masks 902 superimposed on the representation.
[0104] Here, the multiple masks 902 represent various retinal elements, which may include, but are not limited to, intraretinal exudate (IRF), subretinal exudate (SRF), subretinal hyperluminal material (SHRM), pigment epithelial detachment (PED), the interface between (may include) the inner limiting membrane (ILM) layer and the outer limiting membrane (ELM) layer, the interface between (may include) the ILM layer and the retinal pigment epithelium (RPE) layer, and the interface between (may include) the RPE layer and the Bruch's membrane (BM) layer.
[0105] V. Exemplary Training and Validation of Machine Learning Models VA Example Data Various machine learning models were trained and their performance was evaluated. Training included using training data obtained from clinical trials and / or generated based on data obtained from clinical trials. In particular, 935 eyes were selected from 1097 treatment-naive eyes of nAMD subjects who participated in phase 3 of the randomized multicenter HARBOR study. These nAMD subjects were treated with 0.5 mg or 2.0 mg ranibizumab monthly or as needed for 12 months. The HARBOR study graded CNV type as occult CNV (e.g., with occult CNV lesions), predominantly classic CNV, or minimally classic CNV based on FA images. The HARBOR study assessed the presence of fibrosis at day 0, 3 months, 6 months, 12 months, and 24 months.
[0106] The 935 eyes selected included those for which clear fibrosis documentation was available at month 12 and for which baseline OCT imaging data was available, including a baseline OCT volume scan for each eye.
[0107] For training the deep learning model, five equally spaced B-scans were selected from each of the 935 OCT volume scans, covering 1.44 mm of the central macula. Specifically, scans 49, 56, 63, 70, and 77 of the 128 B-scans were selected. The first deep learning model was trained using the raw OCT B-scans. The second deep learning model was trained using segmented images generated based on the raw OCT B-scans. The data were augmented using random horizontal and vertical flips, scaling, rotation, and shear, resulting in a total of 30,000 samples.
[0108] The OCT volume scans were segmented using a pre-trained segmentation model (e.g., an example of an implementation of segmentation model 206 in Figures 2-3). The segmentation model was pre-trained based on annotations made by certified raters. The segmentation model was trained to automatically segment four retinal pathology elements (SHRM, SRF, PED, and IRF) along with five retinal layer elements (ILM, OPL-HLF interface, inner and outer boundaries of the RPE, and BM). The elements were segmented into three topographic locations (e.g., circles of diameters 1 mm, 3 mm, and 6 mm) for each OCT volume scan.
[0109] Based on the segmented image data, retinal feature data was extracted using a feature extraction model (e.g., an example of an implementation of feature extraction model 306 in FIG. 3). The feature extraction model automatically extracted 105 quantitative retinal features. Specifically, these retinal features included 36 volumetric pathology-related features (e.g., 4 retinal pathology elements for each of the three readout variants for each of the three topographical locations), 15 layer-related volumetric features (e.g., 5 pairs of layers for each of the three topographical locations), and 54 layer-related thickness features (e.g., 6 pairs of layers for each of the three readout variants for each of the three topographical locations). All features were derived for each individual B-scan of the OCT volume scan and then combined to form a measurement of the entire volume.
[0110] Training a VB machine learning model The result of training and validating the model was the presence of fibrosis at 12 months. To ensure that the outcome variables were stratified across folds, the folds for the 5-fold cross-validation for the level of the number of subjects were predefined. This was repeated 10 times, 10 times with 5 folds, resulting in a total of 50 training / test folds. The model was always trained on the training set and then used to predict the test set. Validation was performed on all 50 folds for feature-based models (e.g., the example of the implementation of feature-based model 300 in FIG. 3), but validation was only performed on the first iteration of 5 folds for deep learning models (e.g., the example of the implementation of deep learning model 200 in FIG. 2) to limit computational effort.
[0111] For feature-based models, Lasso regularization was used to fit feature-based models (e.g., logistic regression models) to various feature configurations. A combination of selected OCT-derived quantitative retinal features and three baseline clinical variables (CNV type, BCVA, and age) was used. When OCT-derived quantitative retinal features were used, the degree of regularization was set to a constant high value.
[0112] For deep learning models (e.g., convolutional neural networks), we used a ResNet-50 architecture pre-trained on ImageNet. The architecture was either tweaked by replacing the top layer with a custom dense part, allowing for concatenating vectors of clinical variables to the OCT image data, or used as is when no clinical data was used. We applied transfer learning for 20 epochs, keeping the base ResNet-50 layers frozen, followed by 40 or 120 epochs to fine-tune the full network on segmented or raw OCT image data, both with and without clinical data.
[0113] Performance of the VC model For baseline comparisons, feature-based models were constructed for clinical data only (e.g., those with baseline CNV type only; those with baseline visual acuity (BVA) and age only; those with baseline CNV, BVA and age). Performance was evaluated using the area under the receiver operating characteristic (AUC) curve by plotting observed event rates against predicted event rates. Additionally, Youden's index was applied to the AUC curve to select cutoff points, which were reported as the positive and negative predictive values of the model's predictions. Specificity and sensitivity were also evaluated.
[0114] 10 is a table 1000 comparing statistical results of feature-based models using clinical data according to one or more embodiments. As shown in table 1000, based on the average AUC, the feature-based model using only baseline CNV type and the feature-based model using baseline CNV type with BVA and age performed best. However, the feature-based model using only baseline CNV had lower specificity than the feature-based model using baseline CNV type with BVA and age.
[0115] FIG. 11 is a table 1100 comparing statistical results of feature-based models using retinal features derived from OCT image data, according to one or more embodiments. As shown in table 1100, a first feature-based model using OCT-derived retinal features to predict the onset of fibrosis and a second feature-based model using OCT-derived retinal features along with BVA and age to predict the onset of fibrosis were similarly compared to a feature-based model using baseline CNV type (as shown in FIG. 10) based on the average AUC. Although not shown in table 1100, adding baseline CNV type to the first or second feature-based models increased the average AUC to 0.809 and 0.821, respectively. These results indicate that feature-based models using OCT-derived retinal features may be used to accurately and reliably predict the onset of fibrosis.
[0116] 12 is a table 1200 comparing statistical results of deep learning models using OCT image data and segmented image data, according to one or more embodiments. As shown in table 1200, the average AUC of the deep learning model using segmented images to predict the onset of fibrosis was slightly higher than the average AUC of the deep learning model using OCT image data to predict the onset of fibrosis. Furthermore, the average AUC of these two deep learning models performed similarly compared to the feature-based model using the baseline CNV type (as shown in FIG. 10).
[0117] 13 is a table 1300 comparing statistical results of deep learning models using OCT image data and segmented image data in combination with clinical data, according to one or more embodiments. As shown in table 1300, adding clinical data (e.g., BVA, age, and baseline CNV type) to the deep learning model increased the average AUC of the deep learning model using segmented image data over the average AUC of the deep learning model using OCT image data.
[0118] VI. Computer-implemented systems FIG. 14 is a block diagram illustrating a computer system according to various embodiments. The computer system 1400 may be an example of an implementation for the computing platform 102 of FIG. 1. In various embodiments of the present teachings, the computer system 1400 may include a bus 1402 or other communication mechanism for communicating information and a processor 1404 coupled to the bus 1402 for processing information. In various embodiments, the computer system 1400 may also include a memory, which may be a random access memory (RAM) 1406 or other dynamic storage device, coupled to the bus 1402 for determining instructions to be executed by the processor 1404. The memory may also be used to store temporary variables or other intermediate information during execution of instructions to be executed by the processor 1404. In various embodiments, the computer system 1400 may further include a read-only memory (ROM) 1408 or other static storage device coupled to the bus 1402 for storing static information and instructions for the processor 1404. A storage device 1410, such as a magnetic disk or optical disk, may be provided and coupled to the bus 1402 for storing information and instructions.
[0119] In various embodiments, the computer system 1400 may be coupled via the bus 1402 to a display 1412, such as a cathode ray tube (CRT) or liquid crystal display (LCD), for displaying information to a computer user. An input device 1414, including alphanumeric and other keys, may be coupled to the bus 1402 for communicating information and command selections to the processor 1404. Another type of user input device is a cursor control device 1416, such as a mouse, trackball, or cursor direction keys, for communicating directional information and command selections to the processor 1404 and for controlling cursor movement on the display 1412. This input device 1414 typically has two degrees of freedom in two axes, a first axis (i.e., x) and a second axis (i.e., y), that allow the device to specify a position in a plane. However, it should be understood that input devices 1414 that allow cursor movement in three dimensions (x, y, and z) are also contemplated herein.
[0120] Consistent with certain implementations of the present teachings, results may be provided by computer system 1400 in response to processor 1404 executing one or more sequences of one or more instructions contained in RAM 1406. Such instructions may be read into RAM 1406 from another computer-readable medium or computer-readable storage medium, such as storage device 1410. Execution of the sequences of instructions contained in RAM 1406 may cause processor 1404 to perform the processes described herein. Alternatively, hardwired circuitry may be used in place of or in combination with software instructions to implement the present teachings. Thus, implementations of the present teachings are not limited to any specific combination of hardware circuitry and software.
[0121] The terms "computer-readable medium" (e.g., data store, data storage, etc.) or "computer-readable storage medium" as used herein refer to any medium that participates in providing instructions to the processor 1404 for execution. Such media may take many forms, including but not limited to, non-volatile media, volatile media, and transmission media. Examples of non-volatile media may include, but are not limited to, optical, solid-state, and magnetic disks, such as the storage device 1410. Examples of volatile media may include, but are not limited to, dynamic memory, such as the RAM 1406. Examples of transmission media may include, but are not limited to, coaxial cables, copper wire, and fiber optics, including the wires that comprise the bus 1402.
[0122] Common forms of computer readable media include, for example, a floppy disk, a flexible disk, a hard disk, a magnetic tape or any other magnetic medium, a CD-ROM, any other optical medium, punch cards, paper tape, any other physical medium with a pattern of holes, RAM, PROM, and EPROM, flash EPROM, any other memory chip or cartridge, or any other tangible medium from which a computer can read.
[0123] In addition to computer-readable media, instructions or data may be provided as signals on a transmission medium included in a communication device or system to provide a sequence of one or more instructions to the processor 1404 of the computer system 1400 for execution. For example, a communication device may include a transceiver having signals indicative of instructions and data. The instructions and data are configured to cause one or more processors to implement the functions outlined in the disclosure herein. Representative examples of data communication transmission connections may include, but are not limited to, a telephone modem connection, a wide area network (WAN), a local area network (LAN), an infrared data connection, an NFC connection, and the like.
[0124] It will be appreciated that the methodologies described herein, e.g., the flowcharts, diagrams, and accompanying disclosure, may be implemented using computer system 1400 as a stand-alone device or in a distributed network of shared computer processing resources, such as a cloud computing network.
[0125] The methodologies described herein may be implemented by various means depending on the application. For example, these methodologies may be implemented in hardware, firmware, software, or any combination thereof. In the case of a hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processors (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described herein, or combinations thereof.
[0126] In various embodiments, the methods of the present teachings may be implemented as firmware and / or software programs and applications written in conventional programming languages such as C, C++, Python, etc. When implemented as firmware and / or software, the embodiments described herein may be implemented in a non-transitory computer-readable medium having stored thereon a program for causing a computer to execute the above-mentioned methods. It should be understood that the various engines described herein may be provided in a computer system, such as computer system 1400, whereby processor 1404 performs the analysis and decision provided by these engines according to instructions provided by any one or combination of memory components RAM 1406, ROM 1408, or storage device 1410, and user input provided via input device 1414.
[0127] VII. Illustrative Definitions and Context The present disclosure is not limited to the exemplary embodiments and applications described herein or to the manner in which the exemplary embodiments and applications operate or are described herein. Further, the figures may show simplified or partial views, and dimensions of elements in the figures may be exaggerated or out of proportion.
[0128] Unless otherwise defined, scientific and technical terms used in connection with the present teachings described herein shall have the meanings commonly understood by those skilled in the art.Furthermore, unless otherwise required by context, singular terms shall include plurals and plural terms shall include singulars.In general, the nomenclature utilized in connection with and techniques of chemistry, biochemistry, molecular biology, pharmacology and toxicology described herein are those well known and commonly used in the art.
[0129] Additionally, when the terms "on," "attached to," "connected to," "coupled to," or similar terms are used herein, an element (e.g., a component, material, layer, substrate, etc.) may be "on," "attached to," "connected to," or "coupled to" another element, regardless of whether the element is directly on, directly attached to, connected to, or coupled to another element, or whether there are one or more intervening elements between the one element and the other element. Additionally, when a reference is made to a list of elements (e.g., elements a, b, c), such reference is intended to include any one of the listed elements by itself, any combination of less than all of the listed elements, and / or all combinations of the listed elements. The division of sections herein is merely for ease of review and is not intended to limit any combination of the elements described.
[0130] The term "subject" may refer to a subject of a clinical trial, a person undergoing treatment, a person undergoing anti-cancer drug therapy, a person being monitored for remission or recovery, a person undergoing preventive health analysis (e.g., by medical history), or a patient of any other person or subject. In various instances, "subject" and "patient" may be used interchangeably herein.
[0131] As used herein, "substantially" means sufficient to function for the intended purpose. Thus, the term "substantially" allows for slight, insignificant variations from an absolute or perfect state, dimension, measurement, result, etc., as would be expected by one of ordinary skill in the art, but does not noticeably affect overall performance. For parameters or characteristics that are numerical values or can be expressed as numerical values, "substantially" means within 10%.
[0132] As used herein, the term "about" when used with respect to a numerical value or a parameter or characteristic that can be expressed as a numerical value means within 10% of the numerical value. For example, "about 50" means a value in the range of 45 to 55.
[0133] The term "plural ones" means two or more.
[0134] As used herein, the term "plurality" can be 2, 3, 4, 5, 6, 7, 8, 9, 10 or more.
[0135] As used herein, the term "set" means one or more. For example, a set of items includes one or more items.
[0136] As used herein, the phrase "at least one of," when used with a list of items, means that one or more different combinations of the listed items may be used, and only one of the items in the list may be used. The items may be specific objects, things, steps, operations, processes, or categories. In other words, "at least one of" means that any combination or number of items from the list may be used, but not all of the items in the list may be used. For example, but not limited to, "at least one of item A, item B, or item C" means item A; item A and item B; item B; item A, item B, item C; item B, item C; or items A and C. In some cases, "at least one of item A, item B, or item C" means, but is not limited to, two of item A, one of item B, and ten of item C; four of item B and seven of item C; or some other suitable combination.
[0137] As used herein, a "model" may include one or more algorithms, one or more mathematical techniques, one or more machine learning algorithms, or a combination thereof.
[0138] As used herein, "machine learning" may include the practice of using algorithms to analyze data, learn from the data, and then make decisions or predictions about something in the world. Machine learning may use algorithms that can learn from data without relying on rule-based programming. Deep learning may be a form of machine learning.
[0139] As used herein, "artificial neural network" or "neural network" (NN) may refer to a mathematical algorithm or computational model that mimics an interconnected group of artificial neurons that process information based on a connectionistic approach to computation. A neural network, also called a neural net, may use one or more layers of nonlinear units to predict an output for a received input. Some neural networks may include one or more hidden layers in addition to an output layer. The output of each hidden layer may be used as an input to the next layer in the network, i.e., the next hidden layer or the output layer. Each layer of the network generates an output from the received input according to the current value of each parameter set. In various embodiments, a reference to a "neural network" may be a reference to one or more neural networks.
[0140] Neural networks may process information in two ways: when the neural network is being trained, it is in training mode, and when the neural network puts into practice what it has learned, it is in inference (or prediction) mode. Neural networks may learn through a feedback process (e.g., backpropagation) that allows the network to adjust the weight coefficients of individual nodes in the intermediate hidden layers (modify their behavior) so that the output matches that of the training data. In other words, a neural network may learn by being fed training data (training examples), and eventually learn how to arrive at the correct output, even when presented with a new range or set of inputs. The neural network may include, for example, but not limited to, at least one of a feedforward neural network (FNN), a recurrent neural network (RNN), a modular neural network (MNN), a convolutional neural network (CNN), a residual neural network (ResNet), an ordinary differential equation neural network (neural ODE), a U-Net, a fully convolutional network (FCN), a stacked FCN, a stacked FCN with multi-channel learning, a squeeze-and-excite embedded neural network, a MobileNet, or another type of neural network.
[0141] As used herein, "deep learning" may refer to the use of multi-layer artificial neural networks to automatically learn representations from input data, such as images, videos, text, etc., without human-provided knowledge, to provide highly accurate predictions in tasks such as object detection / identification, speech recognition, language translation, etc.
[0142] VIII. Enumeration of Exemplary Embodiments Embodiment 1: A method comprising receiving optical coherence tomography (OCT) image data of a retina of a subject having neovascular age-related macular degeneration (nAMD), processing the OCT image data using a model system including a machine learning model to generate a predictive output, and generating a final output indicative of a risk of developing retinal fibrosis based on the predictive output.
[0143] Embodiment 2: The method of embodiment 1, wherein the machine learning model includes a deep learning model, and the processing includes segmenting the OCT image data via a segmentation model including at least one neural network to form segmented image data, and processing the segmented image data using the deep learning model of the model system to generate a predicted output.
[0144] Embodiment 3: The method of embodiment 2, wherein the machine learning model includes a regression model, and the processing further includes extracting retinal feature data from the segmented image data via a feature extraction model, the retinal feature data including at least one of a first feature value associated with at least one retinal layer element or a second feature value associated with at least one retinal pathology element, and processing the OCT image data using the regression model to generate a predicted output.
[0145] Embodiment 4: The method of any one of embodiments 1 to 3, wherein the machine learning model comprises at least one convolutional neural network.
[0146] Embodiment 5: The method of any one of embodiments 1 to 4, wherein the machine learning model includes a deep learning model, and the processing includes using the deep learning model to process the OCT image data and the clinical data to generate a predictive output, wherein the clinical data includes at least one of a baseline choroidal neovascularization (CNV) type, a baseline visual acuity measurement, or a baseline age.
[0147] Embodiment 6: The method of embodiment 5, wherein the deep learning model includes a convolutional neural network (CNN) system, a first portion of the CNN system includes a convolutional neural network, and a second portion of the CNN system includes a custom dense layer portion, and the processing of the OCT image data and the clinical data includes processing the OCT image data using the first portion of the CNN system to generate a first intermediate output, concatenating a set of vectors for the clinical data to the first intermediate output to form a second intermediate output, and processing the second intermediate output using the custom dense layer portion to generate a predicted output.
[0148] Embodiment 7: The method of any one of embodiments 1 to 6, wherein the final output includes at least one of a binary classification indicating whether or not the development of fibrosis is predicted, a clinical trial recommendation to include or exclude the subject in a clinical trial based on either the prediction output or the binary classification, or a treatment recommendation to at least one of change the type of treatment for the subject or adjust the subject's treatment regimen based on either the prediction output or the binary classification.
[0149] Embodiment 8: A method comprising receiving optical coherence tomography (OCT) image data of a retina of a subject having neovascular age-related macular degeneration (nAMD), segmenting the OCT image data using a segmentation model to generate segmented image data, processing the segmented image data using a deep learning model to generate a predictive output, and generating a final output indicative of a risk of developing retinal fibrosis based on the predictive output.
[0150] Embodiment 9: The method of embodiment 8, wherein at least one of the segmentation model or the deep learning model includes at least one convolutional neural network.
[0151] Embodiment 10: The method of embodiment 8 or embodiment 9, wherein the processing includes processing the segmented image data and the clinical data using a deep learning model to generate a predictive output, the clinical data including at least one of a baseline choroidal neovascularization (CNV) type, a baseline visual acuity measurement, or a baseline age.
[0152] Embodiment 11: The method of embodiment 10, wherein the deep learning model includes a convolutional neural network (CNN) system, a first portion of the CNN system includes a convolutional neural network, and a second portion of the CNN system includes a custom dense layer portion, and the processing of the segmented image data and the clinical data includes processing the segmented image data using the first portion of the CNN system to generate a first intermediate output, concatenating a set of vectors for the clinical data to the first intermediate output to form a second intermediate output, and processing the second intermediate output using the custom dense layer portion to generate a predicted output.
[0153] Embodiment 12: The method of any one of embodiments 8 to 11, wherein the final output includes at least one of a binary classification indicating whether or not the development of fibrosis is predicted, a clinical trial recommendation to include or exclude the subject in a clinical trial based on either the prediction output or the binary classification, or a treatment recommendation to at least one of change the type of treatment for the subject or adjust the treatment regimen for the subject based on either the prediction output or the binary classification.
[0154] Embodiment 13: A method comprising receiving at least one of clinical data or retinal characteristic data for a retina of a subject having neovascular age-related macular degeneration (nAMD), processing the at least one of the clinical data or retinal characteristic data using a regression model to generate a predictive output, and generating a final output indicative of a risk of developing retinal fibrosis based on the predictive output.
[0155] Embodiment 14: The method of embodiment 13, further comprising extracting retinal feature data from the segmented image data via a feature extraction model.
[0156] Embodiment 15: The method of embodiment 14, further comprising segmenting the OCT image data via a segmentation model including at least one neural network to form segmented image data.
[0157] Embodiment 16: The method of any one of embodiments 13 to 15, wherein the clinical data includes at least one of a baseline choroidal neovascularization (CNV) type, a baseline visual acuity measurement, or a baseline age, and the retinal characteristic data includes at least one of a first characteristic value associated with at least one retinal layer element or a second characteristic value associated with at least one retinal pathology element.
[0158] Embodiment 17: The method of any one of embodiments 13 to 16, wherein the regression model is trained using at least one of Ridge regularization, Lasso regularization, or Elastic Net regularization.
[0159] Embodiment 18: The method of any one of embodiments 13 to 17, wherein the predictive output comprises a score indicating the likelihood that fibrosis will develop.
[0160] Embodiment 19: The method of any one of embodiments 13 to 18, wherein the final output includes at least one of a binary classification indicating whether or not the development of fibrosis is predicted, a clinical trial recommendation to include or exclude the subject in a clinical trial based on either the prediction output or the binary classification, or a treatment recommendation to at least one of change the type of treatment for the subject or adjust the treatment regimen for the subject based on either the prediction output or the binary classification.
[0161] Embodiment 20: The method of any one of embodiment 3 or embodiments 13 to 19, wherein the retinal characteristic data includes at least one of a grade for subretinal hyperluminal material (SRHM), a grade for pigment epithelial detachment (PED), a maximum height of subretinal exudate (SRF), a maximum thickness between the interface of the outer plexiform layer (OPL) and Henle's fiber layer (HFL) and the retinal pigment epithelium (RPE) layer, or a thickness between the inner limiting membrane (ILM) layer and the RPE layer.
[0162] IX. Further Considerations Headings and subheadings between sections and subsections of this document are included merely to improve readability and do not imply that features cannot be combined across sections and subsections. Thus, the sections and subsections do not describe separate embodiments. Any one or more of the embodiments described herein with respect to any section or any figure may be combined or otherwise integrated with any one or more of the other embodiments described herein.
[0163] Some embodiments of the present disclosure include a system including one or more data processors. In some embodiments, the system includes a non-transitory computer-readable storage medium including instructions that, when executed on the one or more data processors, cause the one or more data processors to perform some or all of one or more methods and / or some or all of one or more processes disclosed herein. Some embodiments of the present disclosure include a computer program product tangibly embodied in a non-transitory machine-readable storage medium, the computer program product including instructions configured to cause the one or more data processors to perform some or all of one or more methods and / or some or all of one or more processes disclosed herein.
[0164] The terms and expressions employed are used as terms of description and not of limitation, and the use of such terms and expressions is not intended to exclude equivalents of the features shown and described or portions thereof, but it is recognized that various modifications are possible within the scope of the claimed invention. Thus, although the claimed invention has been specifically disclosed by embodiments and optional features, it will be understood that modifications and variations of the concepts disclosed herein may be adopted by those skilled in the art, and that such modifications and variations are deemed to be within the scope of the invention as defined by the appended claims.
[0165] The following description provides only preferred exemplary embodiments and is not intended to limit the scope, applicability or configuration of the present disclosure. Rather, the following description of preferred exemplary embodiments provides those skilled in the art with an enabling description for implementing various embodiments. It will be understood that various changes may be made in the function and arrangement of elements (e.g., block or schematic elements, flow diagram elements, etc.) without departing from the spirit and scope of the appended claims.
[0166] Specific details are given in the following description to provide a thorough understanding of the embodiments. However, it will be understood that the embodiments may be practiced without these specific details. For example, circuits, systems, networks, processes, and other components may be shown as components in block diagram form in order to avoid obscuring the embodiments in unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring the embodiments.
Claims
1. 1. A method comprising: Receiving optical coherence tomography (OCT) image data of a retina of a subject having neovascular age-related macular degeneration (nAMD); processing the OCT image data using a model system including a machine learning model to generate a predicted output; generating a final output indicative of a risk of developing fibrosis of the retina based on the predictive output; A method comprising:
2. The machine learning model includes a deep learning model, and the process further comprises: Segmenting the OCT image data via a segmentation model including at least one neural network to form segmented image data; processing the segmented image data using the deep learning model of the model system to generate the predicted output; The method of claim 1 , comprising:
3. The machine learning model includes a regression model, and the process includes: extracting retinal feature data from the segmented image data via a feature extraction model, the retinal feature data including at least one of a first feature value associated with at least one retinal layer element or a second feature value associated with at least one retinal pathology element; processing the OCT image data using the regression model to generate the predicted output; The method of claim 2 , further comprising:
4. The method of claim 1 , wherein the machine learning model comprises at least one convolutional neural network.
5. The machine learning model includes a deep learning model, and the process further comprises: and processing the OCT image data and clinical data using the deep learning model to generate the predicted output, the clinical data including at least one of a baseline choroidal neovascularization (CNV) type, a baseline visual acuity measurement, or a baseline age. The method of any one of claims 1 to 4, comprising:
6. The deep learning model includes a convolutional neural network (CNN) system, a first portion of the CNN system includes a convolutional neural network and a second portion of the CNN system includes a custom dense layer portion, and the processing of the OCT image data and the clinical data includes: processing the OCT image data using the first portion of the CNN system to generate a first intermediate output; concatenating a set of vectors for the clinical data with the first intermediate output to form a second intermediate output; processing the second intermediate output using the custom dense layer portion to generate the predicted output; The method of claim 5 , comprising:
7. The final output is a binary classification indicating whether or not the development of fibrosis is predicted; a clinical trial recommendation for including or excluding the subject in a clinical trial based on either the predictive output or the binary classification; or A treatment recommendation to at least one of change a type of treatment for the subject or adjust a treatment regimen for the subject based on either the predictive output or the binary classification. The method according to claim 1 , comprising at least one of the following:
8. 1. A method comprising: Receiving optical coherence tomography (OCT) image data of a retina of a subject having neovascular age-related macular degeneration (nAMD); Segmenting the OCT image data using a segmentation model to generate segmented image data; processing the segmented image data using a deep learning model to generate a predicted output; generating a final output indicative of a risk of developing fibrosis of the retina based on the predictive output; A method comprising:
9. The method of claim 8 , wherein at least one of the segmentation model or the deep learning model comprises at least one convolutional neural network.
10. The process, processing the segmented image data and clinical data using the deep learning model to generate the predicted output, the clinical data including at least one of a baseline choroidal neovascularization (CNV) type, a baseline visual acuity measurement, or a baseline age; The method of claim 8 or 9, comprising:
11. The deep learning model includes a convolutional neural network (CNN) system, a first portion of the CNN system includes a convolutional neural network and a second portion of the CNN system includes a custom dense layer portion, and the processing of the segmented image data and the clinical data includes: processing the segmented image data using the first portion of the CNN system to generate a first intermediate output; concatenating a set of vectors for the clinical data with the first intermediate output to form a second intermediate output; processing the second intermediate output using the custom dense layer portion to generate the predicted output; The method of claim 10, comprising:
12. The final output is a binary classification indicating whether or not the development of fibrosis is predicted; a clinical trial recommendation for including or excluding the subject in a clinical trial based on either the predictive output or the binary classification; or A treatment recommendation to at least one of change a type of treatment for the subject or adjust a treatment regimen for the subject based on either the predictive output or the binary classification. The method according to any one of claims 8 to 11, comprising at least one of the following:
13. 1. A method comprising: Receiving at least one of clinical data or retinal characteristic data for a retina of a subject having neovascular age-related macular degeneration (nAMD); processing the at least one of the clinical data or the retinal feature data using a regression model to generate a predicted output; generating a final output indicative of a risk of developing fibrosis of the retina based on the predictive output; A method comprising:
14. The method of claim 13 , further comprising extracting the retinal feature data from the segmented image data via a feature extraction model.
15. 15. The method of claim 14, further comprising segmenting the OCT image data via a segmentation model comprising at least one neural network to form the segmented image data.
16. 16. The method of claim 13, wherein the clinical data comprises at least one of a baseline choroidal neovascularization (CNV) type, a baseline visual acuity measurement, or a baseline age, and the retinal characteristic data comprises at least one of a first characteristic value associated with at least one retinal layer element or a second characteristic value associated with at least one retinal pathology element.
17. 17. The method of claim 13, wherein the regression model is trained using at least one of Ridge regularization, Lasso regularization, or Elastic Net regularization.
18. 18. The method of any one of claims 13 to 17, wherein the predictive output comprises a score indicative of the likelihood that fibrosis will develop.
19. The final output is a binary classification indicating whether or not the development of fibrosis is predicted; a clinical trial recommendation for including or excluding the subject in a clinical trial based on either the predictive output or the binary classification; or A treatment recommendation to at least one of change a type of treatment for the subject or adjust a treatment regimen for the subject based on either the predictive output or the binary classification.
19. The method of any one of claims 13 to 18, comprising at least one of the following:
20. 20. The method of claim 3 or any one of claims 13 to 19, wherein the retinal characteristic data comprises at least one of a grade for subretinal high brightness material (SRHM), a grade for pigment epithelial detachment (PED), a maximum height of subretinal exudate (SRF), a maximum thickness between the interface of the outer plexiform layer (OPL) and Henle's fiber layer (HFL) and the retinal pigment epithelium (RPE) layer, or a thickness between an inner limiting membrane (ILM) layer and the RPE layer.
Citation Information
Patent Citations
Detection, prediction, and classification for ocular disease
WO2020210891A1