Detection of diffuse retinal thickening (DRT) using optical coherence tomography (OCT) images.

Abstract

A method and system for detecting the presence of diffuse retinal thickening (DRT) in optical coherence tomography (OCT) images. Detecting the presence of DRT in OCT images includes: receiving OCT imaging data of a subject's retina, and using the OCT imaging data to form an image input for a machine learning model (e.g., a deep learning model). The machine learning model generates a diffuse retinal thickening (DRT) detection output based on the image input. The DRT detection output indicates whether the presence of DRT is detected in the retina of the subject.

Description

[0001] Inventors: D. Damopoulos, T.F. Abrecht, D.F. Cavalcanti, Lu Huanxiang, M.H. List Cross-references to related applications

[0002] This application relates to and claims the benefit of the priority dates of the aforementioned applications, and each of the aforementioned applications is incorporated herein by reference in its entirety. Technical Field

[0003] This application relates to the detection of diffuse retinal thickening (DRT), and more specifically, to the automatic classification of optical coherence tomography (OCT) imaging data into those with or without evidence of DRT. Background Technology

[0004] Retinal diseases, such as diabetic macular edema (DME) and age-related macular degeneration (AMD), are leading causes of vision loss in subjects aged 50 and older. Some subjects with DME or AMD may develop diffuse retinal thickening (DRT), in which the retina swells and develops areas of low reflectivity due to fluid buildup. Upon entering the retina, the fluid may immediately distort the subject's vision. Over time, the fluid may damage the retina itself, for example, by causing the loss of photoreceptors in the retina. Summary of the Invention

[0005] In one or more embodiments, a method is provided for detecting the presence of diffuse retinal thickening (DRT) in optical coherence tomography (OCT) imaging data. OCT imaging data of a subject's retina can be received. The OCT imaging data can be used to form a first image input for a machine learning model (e.g., a deep learning model). The machine learning model can be used to generate a diffuse retinal thickening (DRT) detection output based on the first image input. The DRT detection output indicates whether the presence of DRT is detected in the subject's retina.

[0006] In one or more embodiments, a method is provided for approximating the area of ​​diffuse retinal thickening (DRT) present in OCT imaging data. OCT imaging data of a subject's retina can be received. This OCT imaging data can be used to form an image input for a machine learning model (e.g., a deep learning model). The machine learning model can be used to generate a DRT detection output based on the first image input. The area of ​​the DRT present in the image input can be approximated.

[0007] In one or more embodiments, a method is provided for approximating the volume of diffuse retinal thickening (DRT) present in OCT imaging data. OCT imaging data of a subject's retina can be received. This OCT imaging data can be used to form an image input for a machine learning model (e.g., a deep learning model). The machine learning model can be used to generate a DRT detection output based on the first image input. The volume of DRT present in the image input can be approximated.

[0008] In one or more embodiments, a system includes: at least one data processor; and at least one memory storing instructions that, when executed by the at least one data processor, cause operation of any or more of the methods described herein or a portion thereof.

[0009] In one or more embodiments, a non-transitory computer-readable medium is provided that stores instructions that, when executed by at least one data processor, cause operation of any or more of the methods described herein or a portion thereof. Attached Figure Description

[0010] To more fully understand the principles and advantages disclosed herein, please refer to the following description in conjunction with the accompanying drawings: Figure 1 is a block diagram of a diffuse retinal thickening (DRT) detection system according to various embodiments.

[0011] Figure 2 is a block diagram of an approximate DRT model for approximating the DRT area according to various embodiments.

[0012] Figure 3 is a block diagram of an approximate DRT model for approximating the DRT volume according to various embodiments.

[0013] Figure 4 is a flowchart of various embodiments for detecting the presence of DRT.

[0014] Figure 5 is a flowchart for approximate estimation of the DRT area according to various embodiments.

[0015] Figure 6 shows example images for approximate estimation of the DRT area according to various embodiments.

[0016] Figure 7 is a flowchart for approximate estimation of DRT volume according to various embodiments.

[0017] Figures 8A and 8B show example images for approximate estimation of DRT volume according to various embodiments.

[0018] Figure 9 is a block diagram of a computer system according to various embodiments.

[0019] It should be understood that the accompanying drawings are not necessarily drawn to scale, and the objects in the drawings are not necessarily drawn to scale relative to each other. The drawings are depictions intended to clearly and understand various embodiments of the apparatuses, systems, and methods disclosed herein. Where possible, the same reference numerals will be used throughout the drawings to refer to the same or similar parts. Furthermore, it should be understood that the drawings are not intended to limit the scope of this teaching in any way. Detailed Implementation

[0020] I. Overview The embodiments described herein recognize that detecting the presence of diffuse retinal thickening (DRT) can be significant for managing retinal diseases such as diabetic macular edema (DME) and age-related macular degeneration (AMD). For example, the ability to accurately and reliably detect the presence of DRT may aid in the management of treatment for DME or AMD. For instance, having an automated system and method for detecting the presence of DRT may help in developing personalized treatment plans for subjects with retinal diseases, mitigating retinal damage, and understanding the pathogenesis of retinal diseases in subjects. Optical coherence tomography (OCT) imaging can be used to detect DRT in the retina affected by retinal diseases such as age-related macular degeneration (AMD) and diabetic macular edema (DME). OCT is an imaging technique in which light is shone onto a biological sample (e.g., biological tissue) and the light reflected from the characteristics of that biological sample is collected to capture a two-dimensional or three-dimensional, high-resolution cross-sectional image of the biological sample.

[0021] DRT is a type of edema that, unlike common measurable retinal fluid, is diffuse in nature, making it difficult for experts (such as human graders) to identify or depict. In some cases, DRT may be indicated by diffuse retinal fluid (e.g., intraretinal fluid, subretinal fluid, subepithelial retinal pigment epithelial fluid, etc.) that causes increased retinal thickness (height > 200 micrometers, width > 200 micrometers) and presents as a low-reflectivity area relative to the rest of the retina. Although OCT images can visualize such diffuse retinal fluid, depicting the presence of DRT can be difficult for human graders because, unlike intraretinal fluid cysts, there is no clearly visible cyst wall on OCT images. Therefore, manual analysis of OCT images by human graders may lack consistency both within and between graders. Consequently, manual analysis of OCT images by human graders can be time-consuming and error-prone. Furthermore, for the same reason, segmenting DRT in OCT images is more difficult for human graders than classifying DRT.

[0022] Therefore, the embodiments described herein recognize that having systems and methods for automating the detection of DRT can be beneficial. For example, having systems and methods capable of accurately and reliably classifying OCT images as having evidence of DRT (e.g., DRT positive) or not having evidence of DRT (e.g., DRT negative) can be beneficial. Thus, the embodiments described herein provide one or more technical advantages that may include, for example, but not limited to, improved model performance (e.g., accuracy), and / or improved performance (e.g., accuracy) of computer systems specifically configured to run the model to perform automatic DRT classification (e.g., the absence or presence of DRT) on OCT images.

[0023] Recognizing and taking into account the importance and practicality of the methods and systems that provide the above improvements, this specification describes various embodiments of automated DRT detection using OCT imaging data. More specifically, this specification describes various embodiments of methods and systems for accurately and reliably classifying OCT imaging data into those with or without evidence of the presence of DRT in the retina using machine learning systems (such as deep learning systems, which may be neural network systems).

[0024] II. Example of a DRT Testing System Figure 1 is a block diagram of a DRT detection system 100 according to various embodiments. The DRT detection system 100 is used to detect the presence of DRT in the retina of a subject using image input 102, which can be received or accessed via a network 104. In some embodiments, the retina is a healthy retina. In other embodiments, the retina is a retina that has been diagnosed with or is suspected of having a retinal disease. For example, the diagnosis may be age-related macular degeneration (AMD), diabetic macular edema (DME), or one of other types of retinal diseases. In some embodiments, the DRT detection system 100 detects the presence of DRT in a patient, provides an approximate estimate of the area of ​​DRT in the patient, and / or provides an approximate estimate of the volume of DRT in the patient.

[0025] As shown in Figure 1, the DRT detection system 100 includes a computing platform 106 configured to store and execute an image processor 108, a trained DRT classification model 110, and a DRT approximation model 112. Although the image processor 108, the trained DRT classification model 110, and the DRT approximation model 112 are shown as using the same computing platform (i.e., computing platform 106) for storage and execution, in some embodiments, one or more of the image processor 108, model 110, and DRT approximation model 112 may use a different computing platform than computing platform 106 for storage and execution. Typically, the image processor 108 receives or accesses an image input 102 and generates a processed image 114. This preprocessed image is the input to the trained DRT classification model 110, which uses the processed image to generate a DRT detection output 116. As shown in the figure, the DRT approximation model 112 may include either the DRT mapping algorithm 118 or the DRT volume approximation model 120. In some examples, the DRT approximation model 112 generates a DRT approximation output 122, which can be used to generate a treatment output 124, which is sent to a remote device 126 via network 104. However, in some examples, the treatment output is based on the DRT detection output 116 without referencing the DRT approximation output 122.

[0026] The DRT detection system 100 also includes a data storage 128 and a display system 130. The data storage and the display system each communicate with the computing platform 106. In some examples, the data storage, the display system, or both may be considered part of the computing platform or otherwise integrated with it. Therefore, in some examples, the computing platform 106, the data storage 128, and the display system 130 may be separate components communicating with each other; however, in other examples, combinations of these components may be integrated together.

[0027] As shown in the figure, the image input may include OCT imaging data 132, which can be generated using an OCT imaging system 134 or an OCT scanner. The OCT imaging system 134 may be a large desktop configuration for a clinical setting, a portable or handheld dedicated system, or a "smart" OCT system incorporated into a user's personal device such as a smartphone. In some cases, the OCT imaging system 134 may include an image denoiser configured to remove noise and other artifacts from the raw OCT volume image to generate the OCT volume. In one or more embodiments, the OCT imaging data 132 includes an OCT volume 136 of the subject's retina. Each of these OCT volumes may consist of multiple OCT B scans 138 of the subject's retina. These multiple OCT B scans may include, for example, but not limited to, dozens, hundreds, thousands, tens of thousands, or some other number of OCT B scans. OCT B scans may also be referred to as OCT slice images or cross-sectional OCT images.

[0028] Although only one OCT imaging system 134 and one DRT detection system 100 are shown in the figures, in other embodiments, there may be more than one of each. Furthermore, although Figure 1 shows the OCT imaging system 134 and the DRT detection system 100 as two separate components, in some embodiments, the OCT imaging system 134 and the DRT detection system 100 may be components of the same system (e.g., maintained by the same entity, such as a healthcare provider or clinical trial administrator). In some cases, a portion of the DRT detection system 100 may be implemented as part of the OCT imaging system 134. For example, the DRT detection system 100 may be configured to operate as a module implemented using the processor, microprocessor, or some other hardware component of the OCT imaging system 134. In other embodiments, the DRT detection system 100 may be implemented within a cloud computing system that can be accessed by or otherwise communicate with the OCT imaging system 134.

[0029] In one embodiment, image processor 108 is configured or programmed to receive OCT imaging data 132 (i.e., image input 102) and perform a set of processing operations thereon to form a processed image 114. The OCT imaging data may be sent as input to image processor 108, retrieved by image processor from memory, or accessed in some other way. The set of processing operations may include, but is not limited to, at least one of the following: normalization, scaling, resizing, horizontal flipping, vertical flipping, cropping, rotation, noise filtering, or some other type of preprocessing operation. Image processor 108 may be implemented using hardware, software, firmware, or a combination thereof. In one or more embodiments, image processor 108 may be implemented within computing platform 106, but in other embodiments, at least a portion of image processor 108 (e.g., a module of image processor 108) is implemented within OCT imaging system 134.

[0030] In some embodiments, the trained DRT classification model 110 is a machine learning or deep learning model trained to classify image inputs (such as one or more of the processed images 114) based on whether the presence of DRT is detected in the subject's retina. For example, the trained DRT classification model 110 may output a DRT detection output 116, which may include classifying one or more of the processed images 114 as DRT positive (e.g., evidence of DRT presence) or DRT negative (e.g., no evidence of DRT presence). The DRT detection output 116 may be a probability value indicating the probability that DRT is present in the retina. This probability value may be quantitative (e.g., percentage) or qualitative (e.g., definitively present DRT, possibly present DRT, definitively absent DRT). In some examples, the DRT detection output 116 is a binary output indicating whether DRT is present in the retina or not. This deep learning model may be implemented using one or more neural network systems. For example, this deep learning model can be implemented using any number of neural networks or combinations of neural networks. In one or more embodiments, the deep learning model includes a convolutional neural network (CNN), which itself may include one or more neural networks.

[0031] In some embodiments, the trained DRT classification model 110 is trained using training data comprising multiple OCT B-scan images of patients with DME and AMD, which have been annotated by human graders to classify DRT in the OCT images. In one training example, the training data comprises 5,133 B-scan images from 276 patients, annotated by trained graders to classify OCT images into one of four DRT categories: clearly present, possibly present, clearly absent, and not gradeable (due to poor image quality). In this example, 90% of the images were graded by four graders, and 98% of the images were graded by two or more graders. In this training example, there are a total of 19,993 annotations, which are grouped together. For example, because “probably present” and “not gradeable” ratings are relatively rare, OCT images classified as clearly present or possibly present DRT are grouped together and classified as DRT positive. Similarly, OCT images classified as DRT-negative or unclassifiable were grouped together and classified as DRT-negative. In this training example, for the validation set, the category chosen by the majority of raters was considered the ground truth, resulting in 490 images rated as DRT-negative and 293 images rated as DRT-positive. Individual images in the dataset rated as unclassifiable by the majority of raters were excluded.

[0032] In some embodiments, the trained DRT classification model 110 is trained using a splitting strategy. In this training example, the hyperparameters of the example convolutional neural network (CNN) (e.g., InceptionV3, ImageNet initialized) used for this binary classification task are selected through cross-validation nested within the training set. In this training example, five-fold cross-validation is used. Training and inference for scalable tests are repeated ten times to estimate the variance. In the 10 training repetitions, the example CNN classifies the validation set images with a mean area under the receiver operating characteristic (AUROC) of 99.2% (standard deviation 0.4%). In the DME validation subset (424 out of 783 images), the AUROC is 98.5% (standard deviation 0.6%).

[0033] Treatment output 124 may include identifying a patient as being at high risk of developing DRT or as a patient currently experiencing DRT. In some embodiments, such identification is based on DRT detection output 116 and / or DRT approximation output 122. In some embodiments, treatment output 124 may also include administering or recommending appropriate treatment based on identifying a patient as being at high risk of developing DRT or as a patient currently experiencing DRT. In some embodiments, the appropriate treatment may include anti-VEGF therapy, such as ranibizumab, aflibercept, or bevacizumab.

[0034] Figure 2 is a block diagram of a DRT approximation model 112 and a DRT approximation output 122 according to various embodiments. The DRT approximation model 112 is used to generate a DRT approximation output 122 for the retina of a subject classified as DRT positive. In some embodiments, the DRT approximation model 112 may include a DRT mapping algorithm 118. DRT may include, but is not limited to, Gradient Weighted Class Activation Mapping (Grad-CAM), a technique that provides a “visual explanation” of the decisions made by a deep learning model when performing predictions, in the form of a heatmap. That is, Grad-CAM can be implemented for a trained deep learning model to generate an attribution map or heatmap of an OCT B scan, wherein the heatmap indicates (e.g., using color, contours, annotations, etc.) the regions or locations used by the neural network model in the OCT B scan when performing DRT classification on the retina shown in the OCT B scan. In one or more embodiments, Grad-CAM can determine the importance of each pixel in the OCT B scan to the DRT classification output generated by the trained DRT classification model 110. Further details on Grad-CAM can be found in RR Selvaraju et al., “Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization,” Arxiv: 1610.02391 (2017), the entire contents of which are incorporated herein by reference. Other non-limiting examples of attribution mapping techniques include Class Activation Mapping (CAM), SmoothGrad, Low Variance Gradient Estimator for Variational Inference (VarGrad), and / or similar techniques, or combinations thereof.

[0035] The DRT mapping algorithm 118 can generate a DRT attribution map 202 as an approximate DRT output 122. The DRT attribution map 202 indicates (e.g., via a heatmap) the importance of each pixel (or region) of the image input 102 relative to the DRT detection output 116. In other words, the DRT attribution map 202 indicates the contribution of each pixel of the image input 102 to the DRT detection output 116 generated by the trained DRT classification model 110. The DRT attribution map 202 can visually indicate (e.g., via color, highlighting, shadow, pattern, outline, text, annotation, etc.) the region in the corresponding OCT B scan of the image input 102 that has the greatest influence on the DRT detection output 116 determined by the trained DRT classification model 110.

[0036] In one or more embodiments, the DRT attribution map 202 can be used to quantify the number of highly important pixels in the image input 102 to provide an approximate area of ​​the DRT. In various embodiments, the DRT attribution map 202 can be used to locate the center of the connected components of the highly important pixels to provide an approximate DRT location in an OCT B scan of the image input 102.

[0037] In some embodiments, the treatment output 124 can be generated using the DRT attribution map 202. For example, the treatment output 124 can be generated based on the approximate area of ​​the DRT quantified by calculating the number of highly important pixels in the image input 102. In other examples, the treatment output 124 can be generated based on the change in the approximate area of ​​the DRT in the image input acquired at time points after the baseline time point relative to the baseline time point. In such examples, the treatment output 124 can also be generated based on the change in the area of ​​the DRT at the approximate DRT location acquired at time points after the baseline time point relative to the baseline time point.

[0038] Figure 3 is a block diagram of a DRT approximation model 112 and a DRT approximation output 122 according to various embodiments. The DRT approximation model 112 is used to generate a DRT approximation output 122 for the retina of a subject classified as DRT positive. In some embodiments, the DRT approximation model 112 includes an image processor 300, which is an OCT segmentation system that generates a segmented image 302 using the processed image 114 of Figure 1. However, the image processor 300 may be a component separate from and in communication with the DRT approximation model 112. The segmented image is then used to identify various retinal elements in the OCT B scan of image input 102 that are associated with the DRT detection output 116. For example, one or more of the segmented images 302 may be generated from OCT imaging data according to one or more techniques described in International Publication No. WO2023205511A1, which is incorporated herein by reference in its entirety. Furthermore, in some embodiments, the image processor 300 is or includes one or more systems for automated retinal segmentation as described in International Publication No. WO2023205511A1. Retinal elements may consist of at least one of retinal layer elements or retinal pathological elements. Detection and identification of one or more retinal layer elements may be referred to as layer element (or retinal layer element) segmentation. Detection and identification of one or more retinal pathological elements may be referred to as pathological element (or retinal pathological element) segmentation. The image processor 300 uses one or more graphic indicators to identify one or more retinal elements on the segmented image 302. For example, one or more color indicators, shape indicators, pattern indicators, shading indicators, lines, curves, signs, marks, labels, text features, other types of graphic indicators, or combinations thereof may be used to identify one or more portions of the OCT image that have been identified as retinal elements (e.g., based on pixels). In some embodiments, the volume of the segmented retinal elements can be used to approximately estimate the volume of the identified DRT. For example, the volume of a retinal pathological element (e.g., any cystic intraretinal fluid (IRF) and / or subretinal fluid (SRF)) can be subtracted from the volume between two retinal layer elements (e.g., retinal layers such as, but not limited to, the internal limiting membrane (ILM) and Bruch's membrane (BM)). The resulting volume approximation can be used as an estimate of the DRT volume as well as any volume associated with healthy tissue. In some embodiments, the DRT volume approximation model 120 calculates a DRT volume approximation output 306. The DRT volume approximation model 120 can be implemented using hardware, software, firmware, or a combination thereof. The DRT volume approximation output 306 can be the DRT approximation output 122.In some embodiments, the combined volume of DRT and healthy tissue can be evaluated at a baseline time point and one or more time points after the baseline time point to assess the change of DRT volume over time. In other examples, treatment output 124 can be generated based on the change of DRT volume approximation output 306 relative to the baseline time point in image inputs acquired at time points after the baseline time point in the patient's retina.

[0039] In some embodiments, the repeatability of quantitative measurements (e.g., volume measurements, slice thickness, etc.) derived from segmented images generated by one or more automated segmentation techniques as described in International Publication No. WO2023205511A1 can be evaluated. For example, the repeatability standard deviation (SD) and / or coefficient of variation (CV) of thickness measurements can be evaluated to confirm whether the thickness measurement has sufficient repeatability for use in the DRT volume approximation as described above. As another example, the repeatability standard deviation (SD) of liquid volume measurements can be evaluated to confirm whether it is within the detection limits of a healthcare provider.

[0040] In various embodiments, the repeatability of volume measurements derived from segmented images generated according to one or more automated segmentation techniques described in International Publication No. WO2023205511A1 can be evaluated to ensure that quantitative measurements (e.g., volume measurements) used for DRT volume approximation as described above are accurate and consistent. In some embodiments, repeated OCT B scans can be used to evaluate the repeatability of quantitative measurements.

[0041] In one example of quantitative measurement reproducibility assessment, OCT B scans from a clinical trial involving repeated OCT scans were used. In this example, two comparable OCT scans were acquired for each eye at almost every patient visit (one macular cube containing 97 OCT B scans and one containing 49 OCT B scans), resulting in 10,021 image pairs for 225 independent eyes. In this example, to approximate two scans of the same site, the macular cube containing 97 OCT B scans was downsampled to 49 OCT B scans by including only odd-numbered OCT B scans to simulate repeated scans at the same density. The automated segmentation technique described in International Publication No. WO2023205511A1 was then performed to segment retinal elements (i.e., retinal layer elements and retinal pathology elements) and extract quantitative measurements. In this example, the thickness of the central subregion layer (also known as central subregion thickness, or CST) and the volumes of intraretinal fluid (IRF) and subretinal fluid (SRF) elements were used as quantitative measurements. One image pair was randomly selected from each eye, and repeatability was estimated using independent observations. The standard deviation (SD) and coefficient of variation (CSV) of repeatability were calculated. The results of this example of repeatability assessment are shown in Table 1 below.

[0042] Table 1: Examples of repeatability assessment for CST and liquid (IRF and SRF) volume measurements As shown in Table 1, the results of the repeatability assessment example indicate a low standard deviation for repeatability of liquid volume and a coefficient of variation of N / A for repeatability of liquid volume. This is because many scans without liquid volume result in a zero denominator in the coefficient of variation formula. In this example, the standard deviation and coefficient of variation for repeatability of thickness measurement are comparable to those obtained by other devices used to measure layer thickness, and the standard deviation for repeatability of liquid volume is likely within the detection limits for healthcare providers.

[0043] In some embodiments, the DRT detection system demonstrates superior accuracy and consistency in detecting patient DRT compared to expert human evaluators. The DRT detection system 100 offers the technical benefits of improved accuracy, reduced overall computational resources, and / or reduced time required to detect DRT in subjects. Furthermore, compared to other methods and systems, the DRT detection system 100 can generate treatment outcomes for subjects more efficiently and accurately.

[0044] In some embodiments, the DRT detection system 100 provides a technological improvement to the field of DRT detection and / or the field of generating DRT treatment outputs. As described above, the DRT detection system 100 detects patient DRT with greater accuracy and consistency than expert human graders, and can reduce the overall computational resources and / or time required to detect DRT in subjects.

[0045] III. Example Flowchart for Classifying DRT Existence Situations Figure 4 is a flowchart of a process 400 for detecting diffuse retinal thickening (DRT) in OCT imaging according to various embodiments. DRT can be a biomarker associated with retinal diseases such as DME or AMD. In various embodiments, process 400 is implemented using the DRT detection system shown in Figure 1.

[0046] Process 400 may optionally include step 401 of training a model (e.g., a deep learning model). Training the model may include example training of a CNN model that results in the trained DRT classification model 110 of Figure 1. The model may include a neural network system, such as a convolutional neural network (CNN). The model may be trained to process OCT images and classify the OCT images as having evidence of DRT presence or not having evidence of DRT presence. For example, the model may classify each OCT image as DRT positive or DRT negative.

[0047] Step 402 of process 400 includes receiving optical coherence tomography (OCT) imaging data of the subject's retina. This OCT imaging data may be, for example, OCT imaging data 132 as shown in Figure 1. The retina may be a retina diagnosed with or suspected of having a retinal disease. The retinal disease may be, for example, age-related macular degeneration (AMD), diabetic macular edema (DME), or some other type of retinal disease. In other embodiments, the retina may be a healthy retina or a retina that has not yet been diagnosed.

[0048] Step 404 of process 400 includes forming an image input for the model using the OCT imaging data. This image input may be, for example, the processed image 114 shown in FIG. 1. As described above, the model may be, for example, model 110 shown in FIG. 1. Step 404 can be performed in various ways. In one or more embodiments, forming the image input simply involves sending the OCT imaging data to model 110 as is. In other embodiments, forming the image input may include performing a set of preprocessing operations on the OCT imaging data using the image processor 108 of FIG. 1. This set of preprocessing operations may include, for example, but not limited to, normalization, scaling, resizing, horizontal flipping, vertical flipping, cropping, rotation, noise filtering, or at least one of some other type of preprocessing operations.

[0049] Step 406 of process 400 includes generating a diffuse retinal thickening (DRT) detection output based on the image input via model 110. This DRT detection output may be, for example, DRT detection output 116 as shown in Figure 1. In one or more embodiments, the DRT detection output may be a probability value indicating the probability that DRT is present in the retina. This probability value may be quantitative (e.g., percentage) or qualitative (e.g., DRT is definitely present, DRT may be present, DRT is definitely not present). In some examples, the DRT detection output may be a binary output indicating whether the presence of DRT is detected or whether DRT is not present in the retina. In some embodiments, step 406 further includes identifying the patient associated with the image input as a patient at high risk of developing DRT or a patient currently experiencing DRT when the DRT detection output indicates that DRT is detected.

[0050] Step 408 of process 400 includes generating a treatment output using the detection output. This treatment output may be, for example, treatment output 124 as shown in Figure 1. In one or more embodiments, the treatment output includes administering appropriate treatment to a patient identified as being at high risk of developing DRT or currently experiencing DRT. In some embodiments, the appropriate treatment includes anti-VEGF therapy, such as ranibizumab, aflibercept, or bevacizumab.

[0051] In some embodiments, process 400 achieves superior accuracy and consistency in detecting patient DRT compared to expert human ratingrs. Process 400 offers the technical benefits of improved accuracy, reduced overall computational resources, and / or reduced time required to detect DRT in subjects. Furthermore, process 400 generates treatment outcomes for subjects more efficiently and accurately compared to other methods and systems.

[0052] In some embodiments, process 400 provides a technological improvement to the field of DRT detection and / or the technological field of generating DRT therapeutic outputs. As described above, process 400 detects patient DRT with superior accuracy and consistency compared to expert human graders, and can reduce the overall computational resources and / or time required to detect DRT in subjects. In some embodiments, process 400 includes novel combinations of steps, thereby achieving technological improvements relative to conventional DRT detection methods.

[0053] IV. Example Flowchart for Approximate Area Estimation of DRT Figure 5 is a flowchart of process 500 for approximating the DRT area according to various embodiments. The DRT area in an image input that has been classified as DRT positive can be approximated by quantifying the number of highly important pixels in the image input. In various embodiments, process 500 is implemented using the DRT detection system shown in Figure 1, and more specifically, using the mapping algorithm shown in Figure 2 as the DRT approximation model 112.

[0054] Step 502 of process 500 includes receiving optical coherence tomography (OCT) imaging data of the subject's retina. This OCT imaging data may be, for example, OCT imaging data 132 as shown in Figure 1.

[0055] The retina can be a retina diagnosed with or suspected of having a retinal disease. This retinal disease can be, for example, age-related macular degeneration (AMD), diabetic macular edema (DME), or some other type of retinal disease. In other embodiments, the retina can be a healthy retina or a retina that has not yet been diagnosed.

[0056] Step 504 of process 500 includes using the OCT imaging data to form an image input for the model. This image input can be, for example, image input 102 as shown in Figure 1. As described above, the model can be, for example... Figure 1 The trained DRT classification model 110 is used. Step 504 can be performed in various ways. In one or more embodiments, forming the image input simply involves sending the OCT imaging data to the model as is. In other embodiments, forming the image input may include performing a set of preprocessing operations on the OCT imaging data. The set of preprocessing operations may include, for example, but not limited to, normalization, scaling, resizing, horizontal flipping, vertical flipping, cropping, rotation, noise filtering, or at least one of some other type of preprocessing operations.

[0057] Step 506 of process 500 includes generating a diffuse retinal thickening (DRT) detection output based on the image input via the model. The DRT detection output may be, for example, the DRT detection output 116 in Figure 1.

[0058] The model can include a neural network system, such as a convolutional neural network (CNN). The model can be trained to process OCT images and classify them as having evidence of the presence of DRT or not.

[0059] In one or more embodiments, the DRT detection output can be a probability value indicating the likelihood that DRT is present in the retina. This probability value can be quantitative (e.g., percentage) or qualitative (e.g., DRT is definitely present, DRT may be present, DRT is definitely not present). In some examples, the DRT detection output can be a binary output indicating whether the presence of DRT is detected or whether DRT is not present in the retina. For example, the model can classify each OCT image as DRT-positive or DRT-negative.

[0060] Step 508 of process 500 includes generating a DRT attribution map for the image input that has been classified as DRT positive using a DRT mapping algorithm. The DRT mapping algorithm may be, for example, DRT mapping algorithm 118 as shown in Figures 1 and 2. The DRT attribution map may be, for example, DRT attribution map 202 as shown in Figure 2.

[0061] The DRT mapping algorithm may include, but is not limited to, Gradient-Weighted Class Activation Mapping (Grad-CAM), a technique that provides a "visual explanation" of the decisions made by a deep learning model when performing predictions, in the form of a heatmap. Specifically, Grad-CAM can be implemented on a trained deep learning model to generate an attribution map or heatmap of an OCT B scan, where the heatmap indicates (e.g., using color, contours, annotations, etc.) the regions or locations used by the neural network model in the OCT B scan when performing DRT classification on the retina shown in the OCT B scan. In one or more embodiments, Grad-CAM can determine the importance of each pixel in the OCT B scan to the DRT classification output generated by the trained DRT classification model 110. Further details about Grad-CAM can be found in RR Selvaraju et al., "Grad-CAM: Visual Exlanations from Deep Networks via Gradient-based Localization," Arxiv:1610.02391 (2017), the entire contents of which are incorporated herein by reference. Other non-limiting examples of attribution mapping techniques include class activation mapping (CAM), SmoothGrad, low variance gradient estimator for variational inference (VarGrad), and / or similar techniques, or combinations thereof.

[0062] The DRT attribution map indicates (e.g., via a heatmap) the importance of each pixel (or region) of the image input relative to the DRT detection output. In other words, the DRT attribution map indicates the contribution of each pixel of the image input to the DRT detection output generated by the trained DRT classification model. The DRT attribution map can visually indicate (e.g., via color, highlighting, shadows, patterns, outlines, text, annotations, etc.) the regions in the corresponding OCT B scan of the image input that have the greatest impact on the DRT detection output determined by the trained DRT classification model.

[0063] In one or more embodiments, the DRT attribution map can be used to quantify the number of highly important pixels in the image input to provide an approximate area of ​​the DRT. In various embodiments, the DRT attribution map can be used to locate the center of the connected components of highly important pixels to provide an approximate DRT location in an OCT B scan of the image input.

[0064] Figure 6 illustrates an example of three OCT B scans (602, 604, and 606) processed as image inputs (e.g., image input 102), and a trained DRT classification model (e.g., trained DRT classification model 110) has identified its DRT detection outputs (e.g., DRT detection output 116) as DRT positive. Figure 6 also shows examples of three corresponding DRT attribution maps (e.g., DRT attribution map 202) generated by inputting OCT B scans 602, 604, and 606 into a DRT mapping algorithm. In each DRT attribution map (612, 614, and 616), a gradient from dark to light is used to indicate importance from low to high, such that lighter pixels or regions are the pixels or regions that contribute the most (or are the most important) to being identified as DRT positive in the DRT detection outputs of all three OCT B scans (602, 604, and 606). In some examples, such as DRT attribution graph 614, dark pixels or regions surrounded by light-colored pixels or regions may also indicate their importance in determining the DRT detection output as DRT positive.

[0065] Step 510 of process 500 includes generating a treatment output using the DRT attribution graph. This treatment output may be, for example, treatment output 124 as shown in Figure 1.

[0066] When the DRT approximation output 122 is the DRT attribution map 202, the treatment output 124 can be applied based on an approximate estimate of the DRT area. For example, the DRT area in the retina may indicate the presence of DME or AMD, and appropriate treatment can be applied to treat DME or AMD. In some embodiments, changes in the DRT area in the retina from a baseline time point to one or more time points after the baseline time point may indicate the severity of the disease and / or the treatment effect in a subject receiving treatment. In some examples, appropriate treatment may be an anti-VEGF therapy, such as ranibizumab, aflibercept, or bevacizumab.

[0067] In some embodiments, the treatment output may be generated based on the approximate area of ​​the DRT, quantified by calculating the number of highly important pixels in the image input. In various embodiments, the treatment output may be generated based on the change in the approximate area of ​​the DRT in the image input acquired at time points after the baseline time point relative to the baseline time point. In such examples, the treatment output may also be generated based on the change in the area of ​​the DRT at the approximate DRT location acquired at time points after the baseline time point relative to the baseline time point.

[0068] In some embodiments, process 500 occurs after step 408 of process 400 identifies the patient associated with the image input as being at high risk of developing DRT or undergoing DRT, or occurs in response to this.

[0069] In some embodiments, process 500 approximates the patient's DRT area with superior accuracy and consistency compared to expert human evaluators. Process 500 provides the technical benefits of improved accuracy, reduced overall computational resources, and / or reduced time required to provide subjects with approximate DRT areas. Furthermore, process 500 can generate treatment outcomes for subjects more efficiently and accurately compared to other methods and systems.

[0070] In some embodiments, process 500 provides a technological improvement to the field of DRT measurement and / or the technological field of generating DRT therapeutic outputs. As described above, process 500 approximates the patient's DRT area with superior accuracy and consistency compared to expert human graders, and reduces the overall computational resources and / or time required to approximate the subject's DRT area. In some embodiments, process 500 includes novel combinations of steps, thereby achieving technological improvements relative to conventional methods for approximate DRT area estimation.

[0071] V. Example Flowchart for Approximate Estimation of DRT Volume Figure 7 is a flowchart of a process 700 for approximating DRT volume using OCT images according to various embodiments. The DRT volume in an image input that has been classified as DRT-positive can be approximated by generating a segmented OCT image and subtracting the volume of retinal pathological elements (e.g., any cystic IRF and / or SRF) from the volume between two retinal layer elements. In various embodiments, process 500 is implemented using the DRT detection system shown in Figure 1, and more specifically, using the DRT volume approximation model 120 shown in Figure 3 as the DRT approximation model 112.

[0072] Step 702 of process 700 includes receiving optical coherence tomography (OCT) imaging data of the subject's retina. This OCT imaging data may be, for example, OCT imaging data 132 as shown in Figure 1. The retina may be a retina diagnosed with or suspected of having a retinal disease. The retinal disease may be, for example, age-related macular degeneration (AMD), diabetic macular edema (DME), or some other type of retinal disease. In other embodiments, the retina may be a healthy retina or a retina that has not yet been diagnosed.

[0073] Step 704 of process 700 includes using the OCT imaging data to form an image input for an image processor. This image input may be, for example, the processed image 114 in FIG. 1. The image processor may be, for example, the image processor 300 in FIG. 3. Step 704 can be performed in various ways. In one or more embodiments, forming the image input simply involves sending the OCT imaging data to the image processor 300 as is. In other embodiments, forming the image input may include performing a set of preprocessing operations on the OCT imaging data using the image processor 108 of FIG. 1. This set of preprocessing operations may include, for example, but not limited to, normalization, scaling, resizing, horizontal flipping, vertical flipping, cropping, rotation, noise filtering, or at least one of some other type of preprocessing operations.

[0074] Step 706 of process 700 includes generating a segmented image based on the image input via the image processor. Figures 8A and 8B are labeled OCT B scans, which can be used as image input (e.g., image input 102) after removing the labels, and a trained DRT classification model (e.g., trained DRT classification model 110) has identified its DRT detection output (e.g., DRT detection output 116) as DRT positive. Figures 8A and 8B can be used as input to a DRT approximation model (e.g., DRT approximation model 112), specifically as described above. Figure 3 The input to the DRT volume approximation model 120 discussed in the paper.

[0075] Step 708 of process 700 includes generating a DRT volume approximation output using the segmented image via the DRT volume approximation model. The DRT volume approximation model may be, for example, the DRT volume approximation model 120 in FIG. 3. The DRT volume approximation output may be, for example, the DRT volume approximation output 306 in FIG. 3. After processing by image processor 300, various retinal elements in FIG. 8A and FIG. 8B are segmented. In some embodiments, the segmented retinal elements may include retinal layer elements (e.g., retinal layer elements 802 and 812 in FIG. 8A and FIG. 8B, respectively, or retinal layer elements 804 and 814 in FIG. 8A and FIG. 8B, respectively). In some embodiments, the retinal layer elements include the internal limiting membrane (ILM), Bruch's membrane (BM), retinal pigment epithelium (RPE), ellipsoidal zone (EZ), outer plexiform layer (OPL), external retinal membrane (ELM), retinal nerve fiber layer (RNFL), ganglion cell layer (GCL), inner plexiform layer (IPL), inner nuclear layer (INL), and / or outer plexiform layer (OPL). In various embodiments, the segmented retinal elements may include retinal pathological elements (such as intraretinal fluid (IRF) as labeled in Figure 8B). In some embodiments, the combined volume of the DRT (as shown in Figures 8A and 8B, 806 and 816, respectively) with healthy tissue can be approximated by generating the volume between two retinal layer elements surrounding the detected DRT (e.g., DRT volume approximation output 306). For example, in Figure 8A, the combined volume of the DRT with healthy tissue (e.g., 806) can be approximated by generating the volume between retinal layer elements 802 and 804. In some embodiments, the two retinal layer elements surrounding the detected DRT may include any combination of retinal layers (e.g., ILM, BM, RPE, EZ, OPL, ELM, RNFL, GCL, IPL, INL, OPL). In some embodiments, the two retinal layer elements include ILM and BM. In other embodiments, the two retinal layer elements include RPE and EZ. In other embodiments, the two retinal layer elements include one of BM, RPE, EZ, and ELM, and one of ILM, RNFL, GCL, IPL, INL, and OPL. In some embodiments, the DRT volume approximation model 120 is programmed to approximate the volume of the DRT using the segmented image.In other embodiments, the combined volume of DRT and healthy tissue (e.g., DRT volume approximation output 306) can be generated by subtracting the volume of the segmented retinal pathology element from the volume between two retinal layer elements surrounding the detected DRT. For example, in Figure 8B, the combined volume of DRT and healthy tissue (e.g., 816) can be approximated by subtracting the volume of the IRF from the volume between retinal layer elements 812 and 814.

[0076] Step 710 of process 700 includes generating a treatment output using the approximate estimate of the DRT volume. In one or more embodiments, the treatment output includes identifying a patient associated with the image input as either a patient at high risk of developing DRT or a patient currently undergoing DRT. In some embodiments, the treatment output may be, for example, treatment output 124 as shown in FIG1. ​​When the approximate DRT output 122 is the approximate DRT volume output 306, treatment output 124 may be applied based on the approximate estimate of the DRT volume. For example, the DRT volume in the retina may indicate the presence of DME or AMD, and appropriate treatment may be applied to treat DME or AMD. In some embodiments, changes in the DRT volume in the retina from a baseline time point to one or more time points after the baseline time point may indicate the severity of the disease and / or the effectiveness of treatment in a subject receiving treatment. In some examples, appropriate treatment may be an anti-VEGF therapy, such as ranibizumab, aflibercept, or bevacizumab.

[0077] In some embodiments, process 700 occurs after step 408 of process 400 identifies the patient associated with the image input as being at high risk of developing DRT or undergoing DRT, or occurs in response to this.

[0078] In some embodiments, process 700 approximates the patient's DRT volume with superior accuracy and consistency compared to expert human evaluators. Process 700 provides the technical benefits of improved accuracy, reduced overall computational resources, and / or reduced time required to provide subjects with approximate DRT volumes. Furthermore, process 700 can generate treatment outcomes for subjects more efficiently and accurately compared to other methods and systems.

[0079] In some embodiments, process 700 provides a technological improvement to the field of DRT measurement and / or the technological field of generating DRT therapeutic outputs. As described above, process 700 approximates the patient's DRT volume with superior accuracy and consistency compared to expert human evaluators, and reduces the overall computational resources and / or time required to approximate the subject's DRT volume. In some embodiments, process 700 includes novel combinations of steps, thereby achieving technological improvements over conventional methods for approximate DRT volume estimation.

[0080] VI. Example Implementation of a Computer System Figure 9 is a block diagram of a computer system 900 according to various embodiments. The computer system 900 may be... Figure 1 An example of an implementation of the computing platform 106 is provided. In one or more examples, the computer system 900 may include a bus 902 or other communication mechanism for transmitting information, and a processor 904 coupled to the bus 902 for processing information. In various embodiments, the computer system 900 may also include a memory (which may be random access memory (RAM) 906 or other dynamic storage device) coupled to the bus 902 for determining instructions to be executed by the processor 904. The memory may also be used to store temporary variables or other intermediate information during the execution of instructions to be executed by the processor 904. In various embodiments, the computer system 900 may further include a read-only memory (ROM) 908 or other static storage device coupled to the bus 902 for storing static information and instructions for the processor 904. A storage device 910 (such as a disk or optical disk) may be provided and coupled to the bus 902 for storing information and instructions.

[0081] In various embodiments, computer system 900 may be coupled to display 912 (such as a cathode ray tube (CRT) or liquid crystal display (LCD)) via bus 902 for displaying information to a computer user. Input device 914, including alphanumeric keys and other keys, may be coupled to bus 902 for transmitting information and command selections to processor 904. Another type of user input device is cursor control 916 (such as a mouse, joystick, trackball, gesture input device, gaze-based input device, or cursor direction keys) for transmitting directional information and command selections to processor 904 and for controlling cursor movement on display 912. This input device 914 typically has two degrees of freedom on two axes (a first axis (e.g., x) and a second axis (e.g., y)), allowing the device to specify a position in a plane. However, it should be understood that input device 914 that allows three-dimensional (e.g., x, y, and z) cursor movement is also contemplated herein.

[0082] Consistent with certain implementations of this teaching, results may be provided by computer system 900 in response to processor 904 executing one or more sequences of one or more instructions contained in RAM 906. Such instructions may be read into RAM 906 from another computer-readable medium or computer-readable storage medium, such as storage device 910. Execution of the sequence of instructions contained in RAM 906 may cause processor 904 to perform the processes described herein. Alternatively, this teaching may be implemented using hardwired circuitry instead of or in combination with software instructions. Therefore, implementations of this teaching are not limited to any particular combination of hardware circuitry and software.

[0083] In some embodiments, network 104 may be implemented using a single network or a combination of multiple networks. Network 104 may be implemented using any number of wired communication links, wireless communication links, optical communication links, or combinations thereof. For example, in various embodiments, network 104 may include the Internet or one or more intranets, wired networks, wireless networks, and / or other suitable types of networks. In another example, network 104 may include a wireless telecommunications network (e.g., a cellular telephone network) adapted to communicate with other communication networks, such as the Internet. In some cases, network 104 includes at least one of a local area network (LAN), a virtual local area network (VLAN), a wide area network (WAN), a public land mobile network (PLMN), the Internet, or other types of networks. OCT imaging system 134 and DRT detection system 100 may each include one or more electronic processors, electronic memories, and other suitable electronic components for executing instructions (such as program code) and / or data stored on one or more computer-readable media to implement the various applications, data, and steps described herein. For example, such instructions may be stored in one or more computer-readable media, such as internal and / or external memory or data storage devices (such as data storage 128) of the components of the DRT detection system 100, and / or accessible via network 104.

[0084] As used herein, the terms "computer-readable medium" (e.g., data storage, data memory, memory device, data storage apparatus, etc.) or "computer-readable storage medium" refer to any medium that participates in providing instructions to processor 904 for execution. Such media can take many forms, including but not limited to non-volatile media, volatile media, and transmission media. Examples of non-volatile media may include, but are not limited to, optical discs, solid-state drives, and magnetic disks (such as storage device 910). Examples of volatile media may include, but are not limited to, dynamic memory, such as RAM 906. Examples of transmission media may include, but are not limited to, coaxial cables, copper wires, and optical fibers, including the wires constituting bus 902.

[0085] Common forms of computer-readable media include, for example, floppy disks, collapsible disks, hard disks, magnetic tapes, or any other magnetic media; CD-ROMs, any other optical media; punched cards, paper tapes, any other physical media with a perforated pattern; RAM, PROMs and EPROMs, FLASH-EPROMs, any other memory chips or cartridges; or any other tangible media that a computer can read.

[0086] In addition to computer-readable media, instructions or data may also be provided as signals on a transmission medium included in a communication device or system to provide one or more sequences of instructions to a processor 904 of a computer system 900 for execution. For example, a communication device may include a transceiver having signals indicating instructions and data. The instructions and data are configured to cause one or more processors to perform the functions outlined in this disclosure. Representative examples of data communication transmission connections may include, but are not limited to, telephone modem connections, wide area networks (WANs), local area networks (LANs), infrared data connections, NFC connections, optical communication connections, etc.

[0087] It should be recognized that the methods, flowcharts, diagrams and accompanying disclosures described herein can be implemented using the computer system 900 as a standalone device or on a distributed network sharing computer processing resources, such as a cloud computing network.

[0088] Depending on the application, the methods described herein can be implemented in various ways. For example, these methods can be implemented in hardware, firmware, software, or any combination thereof. For hardware implementation, the processing unit can be implemented within one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described herein, or combinations thereof.

[0089] In various embodiments, the methods of this teaching can be implemented as firmware and / or software programs as well as applications written in traditional programming languages ​​such as C, C++, and Python. If implemented as firmware and / or software, the embodiments described herein can be implemented on a non-transitory computer-readable medium, wherein a stored program causes a computer to perform the methods described above. It should be understood that the various engines described herein can be provided on a computer system, such as computer system 900, wherein processor 904 will perform analysis and determination provided by these engines based on instructions provided by any memory component RAM 906, ROM 908, or storage device 910, or combinations thereof, and user input provided via input device 914.

[0090] VII. Definitions and Content Examples This disclosure is not limited to these exemplary embodiments and applications, nor to the manner in which the exemplary embodiments and applications operate or as described herein. Furthermore, the accompanying drawings may show simplified or partial views, and the dimensions of elements in the drawings may be exaggerated or disproportionate.

[0091] Furthermore, when the terms “on,” “attached to,” “connected to,” “coupled to,” or similar words are used herein, an element (e.g., a component, material, layer, substrate, etc.) may be “on another element,” “attached to another element,” “connected to another element,” or “coupled to another element,” regardless of whether an element is directly on, directly attached to, directly connected to, or directly coupled to another element, or whether one or more intermediate elements exist between the element and the other. Additionally, when referring to a list of elements (e.g., elements a, b, c), such references are intended to include any single element listed, any combination of fewer than all listed elements, and / or any combination of all listed elements. Section divisions in the specification are for ease of review only and do not limit any combination of elements discussed.

[0092] The term "subject" can refer to an individual in a clinical trial, a person receiving treatment, a person receiving anti-cancer treatment, a person undergoing remission or recovery monitoring, a person undergoing preventative health analysis (e.g., due to their medical history), or a person or patient for any other purpose. In various contexts, "subject" and "individual" may be used interchangeably in this document.

[0093] Unless otherwise defined, scientific and technical terms used in conjunction with the teachings described herein shall have the meanings commonly understood by one of ordinary skill in the art. Furthermore, unless the context requires otherwise, singular terms shall include plural forms, and plural terms shall include singular forms. Generally, this document describes nomenclature and techniques used in conjunction with chemistry, biochemistry, molecular biology, pharmacology, and toxicology that are well-known and commonly used in the art.

[0094] As used herein, “substantially” means sufficient to achieve the intended purpose. Therefore, the term “substantially” allows for minor, insignificant variations relative to the absolute or ideal state, size, measurement, result, etc., as would be expected by one of ordinary skill in the art, without significantly affecting overall performance. When used relative to numerical values ​​or parameters or characteristics that can be expressed as numerical values, “substantially” means within ten percent.

[0095] As used herein, the term “about” for numerical values ​​or parameters or characteristics that can be represented as numerical values ​​means within ten percent of the numerical value. For example, “about 50” means a value in the range of 45 to 55, including the extreme values.

[0096] The term "ones" means more than one.

[0097] As used in this article, the term "multiple" can refer to 2, 3, 4, 5, 6, 7, 8, 9, 10 or more.

[0098] As used in this article, the term "set" refers to one or more items. For example, a set of items includes one or more items.

[0099] As used herein, the phrase “at least one of…” when used with a list of items indicates that different combinations of one or more of the listed items may be used, and that only one item from the list may be required. An item can be a specific object, thing, step, action, process, or category. In other words, “at least one of…” refers to any combination or number of items that may be used in the list, but not all items in the list are required. For example, but not restrictively, “at least one of item A, item B, or item C” means item A; item A and item B; item B; item A, item B, and item C; item B and item C; or item A and item C. In some cases, “at least one of item A, item B, or item C” means, but is not limited to, two of item A, one of item B, and ten of item C; four of item B and seven of item C; or some other suitable combination.

[0100] As used in this article, a “model” may include one or more algorithms, one or more mathematical techniques, one or more machine learning algorithms, or a combination thereof.

[0101] As used in this article, "machine learning" is the practice of using algorithms to analyze data, learn from it, and then make judgments or predictions about things in the world. Machine learning uses algorithms that can learn from data without relying on rule-based programming.

[0102] As used herein, an "artificial neural network" or "neural network" (NN) refers to a mathematical algorithm or computational model that simulates a set of interconnected artificial neurons that process information based on connectionist computational methods. A neural network (also called a neural network) can use one or more layers of linear units, nonlinear units, or both to predict outputs in response to received inputs. In addition to the output layer, some neural networks also include one or more hidden layers. The output of each hidden layer serves as the input to the next layer in the network, i.e., the next hidden layer or output layer. Each layer of the network generates an output from the received input based on the current values ​​of its corresponding set of parameters. In various embodiments, the reference to "neural network" can refer to one or more neural networks.

[0103] Neural networks process information in two ways: in training mode when they are trained, and in inference (or prediction) mode when they put what they have learned into practice. Neural networks learn through a feedback process (e.g., backpropagation), which allows the network to adjust the weights of individual nodes in intermediate hidden layers (modifying their behavior) so that the output matches the output of the training data. In other words, a neural network learns by being fed training data (learning examples) and eventually learns how to obtain the correct output, even when presented with a new range or set of inputs. Neural networks can include, for example, but not limited to, at least one of feedforward neural networks (FNNs), recurrent neural networks (RNNs), modular neural networks (MNNs), convolutional neural networks (CNNs), residual neural networks (ResNet), ordinary differential equation neural networks (neural-ODEs), squeezed and excited embedded neural networks, MobileNet, or other types of neural networks.

[0104] As used in this article, "deep learning" can refer to the use of multi-layered artificial neural networks to automatically learn representations from input data (such as images, videos, text, etc.) without human knowledge, in order to provide highly accurate predictions in tasks such as object detection / identification, speech recognition, and language translation.

[0105] VIII. Description of Exemplary Embodiments Example 1: A method comprising: receiving optical coherence tomography (OCT) imaging data of a subject's retina; using the OCT imaging data to form a first image input for a machine learning model; and generating a diffuse retinal thickening (DRT) detection output based on the first image input via the machine learning model, wherein the DRT detection output indicates whether the presence of DRT is detected in the subject's retina.

[0106] Example 2: According to the method of Example 1, the DRT detection output is a positive detection when the machine learning model determines that the presence of diffuse retinal fluid indicates DRT, and the DRT detection output is a negative detection when the machine learning model determines that DRT does not exist.

[0107] Example 3: The method according to any one of Examples 1 to 2, wherein the DRT detection output includes at least one of the following: a probability value indicating the probability that DRT is present in the retina, a binary classification of the presence of DRT, or a value indicating the amount of diffuse retinal fluid in the retina.

[0108] Example 4: According to any one of Examples 1 to 3, the method of preprocessing the OCT imaging data to generate the first image input includes: performing a set of preprocessing operations on the OCT imaging data to form the first image input, the set of preprocessing operations including at least one of the following: normalization operation, scaling operation, resizing operation, horizontal flipping operation, vertical flipping operation, cropping operation, rotation operation, noise filtering operation, or some other type of preprocessing operation.

[0109] Example 5: The method according to any one of Examples 1 to 4 further includes applying treatment based on the DRT detection output.

[0110] Example 6: The method according to Example 5, wherein the treatment is anti-VEGF therapy.

[0111] Example 7: The method according to any one of Examples 1 to 6 further includes: training the machine learning model using a training dataset comprising a plurality of training OCT images, wherein the training OCT images in the plurality of training OCT images are labeled as belonging to a category selected from the group consisting of: explicitly present DRT, possibly present DRT, explicitly absent DRT, and non-scalable.

[0112] Example 8: According to the method of Example 7, wherein the plurality of training OCT images includes training OCT images labeled by a plurality of human ratingrs, and wherein the category selected by the majority of the plurality of human ratingrs for a particular training OCT image in the training OCT images is used as the ground truth for calculating the loss.

[0113] Example 9: The method according to any one of Examples 1 to 8 further includes: training the machine learning model using a training dataset comprising a plurality of training OCT images, wherein the training OCT images in the plurality of training OCT images are labeled as belonging to a category selected from the group consisting of: DRT positive, DRT negative, and ungradeable.

[0114] Example 10: The method according to any one of Examples 1 to 9 further includes: training the machine learning model using a training dataset comprising a plurality of training OCT images, wherein the training OCT images in the plurality of training OCT images are labeled as DRT positive or DRT negative.

[0115] Example 11: The method according to any one of Examples 1 to 10, wherein the machine learning model includes a deep learning model.

[0116] Example 12: The method according to any one of Examples 1 to 11, wherein the machine learning model includes a convolutional neural network (CNN).

[0117] Example 13: The method according to any one of Examples 1 to 12, wherein the first image input includes an OCT B scan; wherein the DRT detection output indicates the presence of DRT in the retina of the subject; and wherein the method further includes: using the OCT B scan to form a second image input for an image processor; using the image processor to generate a segmented OCT image; and using a DRT volume approximation model to generate a DRT volume approximation value based on the segmented image.

[0118] Example 14: According to the method of Example 13, the segmented OCT image identifies a first approximate volume of retinal pathological elements and a second approximate volume between two retinal layer elements; wherein generating the DRT volume approximation value based on the segmented image using the DRT volume approximation model includes subtracting the first approximate volume from the second approximate volume; and wherein the difference between the first approximate volume and the second approximate volume is the DRT volume approximation value.

[0119] Example 15: The method according to any one of Examples 1 to 12, wherein the first image input includes an OCT B scan; wherein the DRT detection output indicates the presence of DRT in the retina of the subject; wherein the method further includes generating a DRT attribution map using a DRT mapping algorithm and the OCT B scan; and wherein the DRT attribution map indicates the region or location used by the machine learning model in the OCT B scan when generating the DRT detection output, the DRT detection output indicating the presence of DRT in the retina of the subject.

[0120] Example 16: A method comprising: receiving optical coherence tomography (OCT) imaging data of a subject's retina; using the OCT imaging data to form an image input for a machine learning model; generating a diffuse retinal thickening (DRT) detection output based on the image input via the machine learning model, wherein the DRT detection output indicates whether the presence of DRT is detected in the subject's retina; and approximating the area of ​​the DRT present in the image input.

[0121] Example 17: A method comprising: receiving optical coherence tomography (OCT) imaging data of a subject's retina; using the OCT imaging data to form an image input for a machine learning model; generating a diffuse retinal thickening (DRT) detection output based on the image input via the machine learning model, wherein the DRT detection output indicates whether the presence of DRT is detected in the subject's retina; and approximating the volume of the DRT present in the image input.

[0122] Example 18: The method according to Example 16 or Example 17 further includes applying treatment based on the approximate estimate of the area or volume of the DRT present in the image input.

[0123] Example 19: The method according to Example 18, wherein the treatment is anti-VEGF therapy.

[0124] Example 20: A system comprising: one or more data processors; and a non-transitory computer-readable storage medium containing instructions that, when executed on the one or more data processors, cause the one or more data processors to perform part or all of the methods disclosed in Examples 1 to 19.

[0125] Example 21: A computer program product tangibly embodied in a non-transitory machine-readable storage medium, the computer program product comprising instructions configured to cause one or more data processors to perform part or all of one or more methods disclosed in Examples 1 to 19.

[0126] IX. Other Considerations Although this teaching has been described in conjunction with various embodiments, it is not intended to be limited to such embodiments. Rather, this teaching encompasses various alternatives, modifications, and equivalents that will be understood by those skilled in the art.

[0127] For example, the flowcharts and block diagrams described above illustrate the architecture, functionality, and / or operation of possible implementations of various methods and system embodiments. Each box in a flowchart or block diagram may represent a portion of a module, segment, function, operation, or step, or a combination thereof. In some alternative implementations of the embodiments, one or more functions labeled within a box may occur in a non-linear order. For example, in some cases, two boxes shown consecutively may be executed substantially simultaneously. In other cases, the boxes may proceed in reverse order. Furthermore, in some cases, one or more boxes may be added to replace or supplement one or more other boxes in the flowchart or block diagram.

[0128] Therefore, in describing various embodiments, the specification may present methods and / or processes as specific sequences of steps. However, if a method or process does not depend on the specific sequence of steps described herein, the method or process should not be limited to the specific sequence of steps listed, and those skilled in the art will readily understand that these sequences may be different and still remain within the spirit and scope of the various embodiments.

Claims

1. A method comprising: Receive optical coherence tomography (OCT) imaging data of the subject's retina; The OCT imaging data is used to form the first image input for the machine learning model; as well as The machine learning model generates a diffuse retinal thickening (DRT) detection output based on the first image input, wherein the DRT detection output indicates whether the presence of DRT is detected in the retina of the subject.

2. The method of claim 1, wherein the DRT detection output is a positive detection when the machine learning model determines that the presence of diffuse retinal fluid indicates DRT, and the DRT detection output is a negative detection when the machine learning model determines that DRT is not present.

3. The method according to any one of claims 1 to 2, wherein the DRT detection output includes at least one of the following: a probability value indicating the probability of DRT being present in the retina, a binary classification of the presence of DRT, or a value indicating the amount of diffuse retinal fluid in the retina.

4. The method according to any one of claims 1 to 3, wherein preprocessing the OCT imaging data to generate the first image input comprises: The OCT imaging data is subjected to a set of preprocessing operations to form the first image input. The set of preprocessing operations includes at least one of the following: normalization, scaling, resizing, horizontal flipping, vertical flipping, cropping, rotation, noise filtering, or some other type of preprocessing operation.

5. The method according to any one of claims 1 to 4, further comprising administering treatment based on the DRT detection output.

6. The method of claim 5, wherein the treatment is an anti-VEGF therapy.

7. The method according to any one of claims 1 to 6, further comprising: The machine learning model is trained using a training dataset comprising multiple training OCT images, wherein the training OCT images are labeled as belonging to a category selected from the group consisting of: explicitly present DRT, possibly present DRT, explicitly absent DRT, and not scalable.

8. The method of claim 7, wherein the plurality of training OCT images comprises training OCT images labeled by a plurality of human ratingrs, and wherein the category selected by a majority of the plurality of human ratingrs for a particular training OCT image in the training OCT images is used as the ground truth for calculating the loss.

9. The method according to any one of claims 1 to 8, further comprising: The machine learning model is trained using a training dataset comprising multiple training OCT images, wherein the training OCT images are labeled as belonging to a category selected from the following groups: DRT positive, DRT negative, and ungradeable.

10. The method according to any one of claims 1 to 9, further comprising: The machine learning model is trained using a training dataset comprising multiple training OCT images, wherein the training OCT images are labeled as DRT positive or DRT negative.

11. The method according to any one of claims 1 to 10, wherein the machine learning model comprises a deep learning model.

12. The method according to any one of claims 1 to 11, wherein the machine learning model comprises a convolutional neural network (CNN).

13. The method according to any one of claims 1 to 12, The first image input includes an OCT B scan; The DRT detection output indicates the presence of DRT in the subject's retina; as well as The method further includes: The OCT B scan is used to form a second image input for the image processor; The image processor is used to generate segmented OCT images; and The DRT volume approximation model is used to generate DRT volume approximations based on the segmented image.

14. The method according to claim 13, The segmented OCT image identifies a first approximate volume of retinal pathological elements and a second approximate volume between two retinal layer elements; Generating the DRT volume approximation value based on the segmented image using the DRT volume approximation model includes subtracting the first approximation volume from the second approximation volume; and The difference between the first approximate volume and the second approximate volume is the approximate value of the DRT volume.

15. The method according to any one of claims 1 to 12, The first image input includes an OCT B scan; The DRT detection output indicates the presence of DRT in the subject's retina; The method further includes generating a DRT attribution map using a DRT mapping algorithm and the OCT B scan; and The DRT attribution map indicates the region or location used by the machine learning model in the OCT B scan when generating the DRT detection output indicating the presence of DRT in the subject's retina.

16. A method comprising: Receive optical coherence tomography (OCT) imaging data of the subject's retina; The OCT imaging data is used to form image inputs for machine learning models; The machine learning model generates a diffuse retinal thickening (DRT) detection output based on the image input, wherein the DRT detection output indicates whether the presence of DRT is detected in the retina of the subject. as well as An approximate estimate is made of the area of ​​the DRT present in the image input.

17. A method comprising: Receive optical coherence tomography (OCT) imaging data of the subject's retina; The OCT imaging data is used to form image inputs for machine learning models; The machine learning model generates a diffuse retinal thickening (DRT) detection output based on the image input, wherein the DRT detection output indicates whether the presence of DRT is detected in the retina of the subject. as well as The volume of the DRT present in the image input is approximated.

18. The method of claim 16 or claim 17, further comprising applying treatment based on the approximate estimate of the area or volume of the DRT present in the image input.

19. The method of claim 18, wherein the treatment is an anti-VEGF therapy.

20. A system comprising: One or more data processors; And a non-transitory computer-readable storage medium containing instructions that, when executed on the one or more data processors, cause the one or more data processors to perform part or all of the methods disclosed in claims 1 to 19.

21. A computer program product tangibly embodied in a non-transitory machine-readable storage medium, comprising instructions configured to cause one or more data processors to perform part or all of the methods disclosed in claims 1 to 19.

Images