Method and system for estimating three-dimensional structures using two-dimensional images

Abstract

A method of estimating optical coherence tomography (OCT) parameters includes: obtaining a retinal fundus image; and segmenting the retina fundus image for generating a 2D image of the target region. The method further comprises the steps of inputting the 2D image of the target area into a neural network; estimated OCT parameters are generated using a neural network.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 471,706, filed June 7, 2023, entitled “Method and System for Estimating Three-Dimensional Structures Using Two-Dimensional Images,” the entire disclosure of which is incorporated herein by reference for all purposes. Technical Field

[0003] This disclosure relates to image processing, and more particularly to a method and system for estimating optical coherence tomography parameters. Background Technology

[0004] Optical coherence tomography (OCT) is a non-contact, non-invasive, and objective structural imaging device used for cross-sectional and three-dimensional (3D) observation of the macula and optic nerve head. For several years, OCT has been commercially available and accepted as the clinical standard of ophthalmology for diagnosing retinal diseases. In addition, OCT can be used for 3D imaging in neuro-ophthalmology and neurodegenerative diseases. Therefore, OCT has become an important tool for diagnosing, monitoring, and predicting the prognosis of various diseases.

[0005] However, OCT instruments are bulky, making operation time-consuming and laborious for some operators, and the procedure may be unbearable for some patients. Therefore, the application of OCT is limited and it is not commonly used in general medical clinics or community screening settings. Thus, there is a need in the art to improve methods and systems related to structural imaging. Summary of the Invention

[0006] Embodiments of this disclosure relate to image processing. More specifically, embodiments of this disclosure provide methods and systems for estimating 3D OCT parameters using 2D fundus camera images. In specific embodiments, a machine learning system is utilized to estimate one or more OCT parameters using 2D images as input. This disclosure is applicable to 3D parameter estimation applications beyond optometry and ophthalmology, which include other image processing applications.

[0007] As mentioned above, OCT devices provide 3D images of the eye. Fundus cameras can be used to acquire low-cost retinal images (e.g., non-mydriatic retinal images) because they are portable, quick and easy to operate, and the data interpretation is readily available. Patient acceptance and tolerance of fundus camera imaging procedures are also high, and operators can easily use them. Furthermore, studies have confirmed that the use of fundus camera imaging in community settings is practical and effective.

[0008] Embodiments of this disclosure employ machine learning methods and systems, applying artificial intelligence (AI) methods that, for example, estimate integrated OCT parameters using non-mydriatic retinal images based on hidden features present in fundus images. These OCT parameters include: retinal nerve fiber layer (RNFL) thickness; optic head (ONH) parameters: marginal area, optic disc area, mean cup-to-disc ratio (C / D), vertical C / D, and cup volume; and mean and minimum ganglion cell-inner plexiform layer (GCIPL) thickness.

[0009] As described herein, methods and systems are provided for estimating comprehensive OCT parameters based on color fundus retinal images. Results indicate that embodiments of this disclosure can serve as convenient, economical, and accurate tools for screening, diagnosing, and monitoring glaucoma and other diseases that can be assessed via OCT, particularly in telemedicine systems and community settings where OCT is unavailable.

[0010] Compared to conventional techniques, the technology disclosed herein offers numerous advantages. For example, embodiments of this disclosure provide a convenient, economical, and accurate tool for screening, assisting in diagnosis, indicating disease severity, and monitoring disease progression, particularly in telemedicine systems and community settings where OCT may be unavailable. Using embodiments of this disclosure, the emergence of artificial intelligence technology can be combined with telemedicine to create more community healthcare services. For instance, in developing world environments where rural residents have limited access to appropriate healthcare, the economical and practical screening tool described herein is crucial in facilitating timely referrals to clinicians and significantly aids subsequent diagnosis, treatment, and follow-up, thereby alleviating the healthcare burden. These and other embodiments of this disclosure, along with their many advantages and features, will be described in more detail below with reference to the corresponding figures.

[0011] Patients diagnosed with one or more neuroophthalmic disorders or at increased risk of developing one or more neuroophthalmic disorders using any of the methods described herein may require additional testing using standard diagnostic and monitoring methods, such as intraocular pressure measurement (to measure intraocular pressure), mydriasis and imaging tests (to detect optic nerve damage), visual field testing (to detect areas of decreased vision), corneal pachymetry (to measure corneal thickness), and spherical examination (to check drainage angle), where one or more of the aforementioned neuroophthalmic disorders include glaucoma, non-arterial ischemic optic neuropathy (NAION), Leber hereditary optic neuropathy (LHON), compressive optic neuropathy, optic disc edema, idiopathic intracranial hypertension, or neuroretinitis (NMO) spectrum disorders. Where deemed appropriate by the attending physician, patients may receive further early treatment to prevent or delay the onset of symptoms, reduce the severity of symptoms, and / or prevent the development of more severe damaging symptoms. For example, patients may be prescribed and given eye drops containing the following medications: prostaglandins (to lower intraocular pressure), beta-blockers (to reduce intraocular fluid secretion / accumulation, thereby lowering intraocular pressure), alpha-adrenergic agonists (to reduce intraocular fluid secretion and increase intraocular fluid outflow), carbonic anhydrase inhibitors (to reduce intraocular fluid secretion), rho kinase inhibitors (to lower intraocular pressure), miotic or cholinergic agents (to increase intraocular fluid outflow).

[0012] Similarly, patients believed to have one or more neurodegenerative diseases or at increased risk of developing such diseases may require additional diagnostic testing to provide further diagnostic information (e.g., brain imaging via CT scans or other imaging techniques to show excessive brain volume loss, or cognitive tests to show accelerated decline). Physicians or other healthcare professionals can then prescribe appropriate treatment or preventative measures to manage / alleviate existing symptoms or delay future disease onset. These neurodegenerative diseases include Alzheimer's disease (AD), mild cognitive impairment (MCI), Parkinson's disease, dementia, or Huntington's disease. The U.S. Food and Drug Administration (FDA) has approved several cholinesterase inhibitors, including donepezil (Aricept™, a racemic cholinesterase inhibitor approved for the treatment of all stages of AD, including moderate to severe AD), livastimin (Exelon™, approved for the treatment of mild to moderate AD), galantamine (Razadyne™, for mild to moderate patients), and namenda™. Based on the methods described herein, any one or more of these medications may be prescribed for the treatment of patients diagnosed with MCI or AD. Another possible treatment option is the use of trazodone, which is currently approved as an antidepressant and has been reported as an effective agent for improving symptoms of MCI or AD.

[0013] For patients considered to have conditions such as hypertensive retinopathy, hypertension, intracranial tumors affecting the visual pathway, sphenoid and parasphenoid sinus lesions, axonal degeneration of the eye in patients with multiple sclerosis (MS), diabetic peripheral neuropathy (DPN), type 1 diabetes, or brain atrophy or spinal cord lesions detected by MRI, or at high risk of having these conditions, additional routine testing may be performed for regular monitoring and / or diagnosis of related diseases. Where deemed appropriate, this may be used as a follow-up tool to remind patients to use antihypertensive medications (such as amlodipine, carvedilol, furosemide, etc.), cholesterol-lowering medications, and insulin. Attached Figure Description

[0014] Figure 1 This is a simplified schematic diagram illustrating a system for estimating 3D features from a 2D image according to one embodiment of the present disclosure.

[0015] Figure 2 This is a simplified flowchart illustrating a method for estimating OCT parameters using a color image according to one embodiment of the present disclosure.

[0016] Figure 3 This is a simplified flowchart illustrating a method for training a neural network according to one embodiment of the present disclosure.

[0017] Figure 4 This is a simplified flowchart illustrating a method for estimating 3D features using a trained neural network according to one embodiment of the present disclosure.

[0018] Figure 5A This is a segmentation map of the target region of the optic disc of a normal eye according to one embodiment of the present disclosure.

[0019] Figure 5B This is a segmentation map of the target region of the optic disc of an eye suffering from glaucomatous optic neuropathy (GON) according to one embodiment of the present disclosure.

[0020] Figure 5C This is a segmentation map of the target region of the macula of a normal eye according to one embodiment of the present disclosure.

[0021] Figure 5D This is a segmentation map of the macula of an eye with GON according to one embodiment of the present disclosure.

[0022] Figures 6A-6H This is a box plot of OCT parameters according to one embodiment of the present disclosure.

[0023] Figures 7A-7H This is a violin plot illustrating the actual OCT parameters and the estimated OCT parameters according to one embodiment of the present disclosure.

[0024] Figure 8 This is a simplified schematic diagram illustrating an OCT parameter estimation system according to one embodiment of the present disclosure. Detailed Implementation

[0025] Embodiments of this disclosure relate to image processing. More specifically, embodiments of this disclosure provide methods and systems for estimating 3D OCT parameters using 2D fundus camera images. In one specific embodiment, a machine learning system is used to estimate one or more OCT parameters using a 2D image as input. This disclosure is applicable to 3D parameter estimation applications beyond optometry and ophthalmology, which include other image processing applications.

[0026] As described herein, embodiments of this disclosure provide methods and systems applicable to a variety of diseases related to OCT parameters. Embodiments of this disclosure provide a convenient, economical, and accurate tool for screening, assisting clinicians in diagnosis, indicating disease severity, and monitoring disease progression, particularly in telemedicine systems and community settings where OCT may not be available.

[0027] OCT is an ocular imaging technique that provides high-resolution cross-sectional (i.e., 3D) images. Currently, as a clinical tool, OCT is particularly suitable for: structural measurements of pericapillary RNFL thickness; ONH volume analysis, including optic disc area, marginal area, mean cup-to-disc ratio, vertical cup-to-disc ratio, and cup volume; and macular anatomy, including GCIPL thickness.

[0028] OCT measurements help identify the integrity of the visual pathway in intracranial lesions, neuro-ophthalmological processes, and neurodegenerative diseases of the visual pathway. For several years, OCT has been commercially available and accepted as a clinical standard for diagnosing retinal diseases in ophthalmology. Additionally, RNFL thickness, ONH parameters, and GCIPL thickness are particularly useful in neuro-ophthalmology. OCT has become an important tool for diagnosis, disease monitoring, and prognosis prediction in neuro-ophthalmology, including diseases such as non-ischemic optic neuropathy (NAION), Leber hereditary optic neuropathy (LHON), compressive optic neuropathy, optic disc edema, idiopathic intracranial hypertension, and neuroretinitis (NMO) spectrum disorders.

[0029] Recently, OCT has been found to be very useful in detecting neurodegenerative diseases such as Alzheimer's disease (AD), mild cognitive impairment (MCI), and Parkinson's disease. RNFL and GCIPL thickness have been reported to be associated with hypertensive retinopathy and blood pressure, stroke risk (e.g., ischemic or hemorrhagic stroke), intracranial tumors affecting the visual pathway, sphenoid and parasphenoid lesions, axonal degeneration of the eye in patients with multiple sclerosis (MS), diabetic peripheral neuropathy (DPN), and other surrogate markers for monitoring disease activity, such as brain atrophy and spinal cord lesions measured by MRI. RNFL thinning is also associated with the severity of dementia, Huntington's disease, papilledema, autism, attention deficit / hyperactivity disorder (ADHD), autism spectrum disorder (ASD), and mental health diagnoses including depression. Furthermore, one study showed that RNFL and GCIPL thickness were decreased in type 1 diabetic patients without retinopathy compared to healthy controls. Therefore, embodiments of this disclosure provide deep insights into the retinal nerve fiber layer of the eye, making certain embodiments applicable not only to the detection and treatment of glaucoma, but also to a variety of systemic diseases, including the aforementioned diseases.

[0030] Automated classification using AI methods has been used to detect various retinal lesions and other diseases, achieving good performance on fundus images. However, simple classification does not provide sufficient information for disease screening, diagnosis, and follow-up. Furthermore, AI methods such as deep learning (DL) often lack interpretability in disease detection. Embodiments of this disclosure utilize trained DL models to learn fundamental features from data from multi-layer networks, thereby providing quantitative assessments of OCT parameters used by clinicians for diagnosis, indicating severity, and monitoring the progression of various diseases associated with OCT parameters. Additionally, embodiments of this disclosure increase the interpretability of automated retinal image analysis methods and improve their classification performance.

[0031] As described herein, embodiments of this disclosure utilize color fundus retinal images (e.g., obtained using automated retinal image analysis (ARIA) methods) to objectively estimate integrated OCT parameters measured by OCT disc and macular scans, including ONH parameters (disc area, marginal area, mean C / D ratio, vertical C / D ratio, and cup volume), mean RNFL thickness, and mean and minimum GCIPL thickness. To better estimate OCT parameters at different levels, patients with / without glaucomatous optic neuropathy (GON), a typical neuropathy characterized by variations in RNFL thickness, ONH parameters, and GCIPL thickness, were selected.

[0032] From a medical perspective, embodiments of this disclosure predict continuous parameters of OCT scans, including 3D parameters, based on 2D non-mydriatic color fundus retinal images. Some embodiments do not focus on a single OCT parameter, but rather on eight parameters related to the optic disc and macular region, which can provide comprehensive retinal information for better disease estimation. Third, images captured using non-mydriatic photography techniques can be utilized compared to non-mydriatic images, which is more convenient, feasible, and tolerable. Although non-mydriatic retinal images may pose challenges for training due to their lower resolution and relative blurriness, embodiments of this disclosure still achieve excellent performance. While non-mydriatic retinal images can be used, embodiments of this disclosure are not limited to their use. The reason for using non-mydriatic images is that telemedicine is often performed without the use of vasodilators, using non-mydriatic images, as a practical, convenient, and beneficial solution for patients and healthcare systems.

[0033] From a methodological perspective, the implementation methods disclosed herein are interpretable, avoiding the "black box" problem associated with certain deep learning networks. Furthermore, the methods and systems described herein are more accurate than traditional machine learning methods and can be periodically improved by retraining the deep learning network.

[0034] Figure 1 This is a simplified schematic diagram illustrating a system for estimating 3D features from a 2D image according to one embodiment of the present disclosure. System 100 includes a fundus camera 110 operable to acquire 2D images 112, such as a non-mydriatic RGB image of a patient's retina. The 2D image 112 is typically segmented according to a target region, which will be discussed more fully below in conjunction with the ARIA method. System 100 also includes an OCT camera 120 operable to acquire 3D images 122 and generate 3D (OCT) features 124 of the retina. These 3D (OCT) features 124 may include features corresponding to the optic disc region and the macula region.

[0035] System 100 also includes a neural network 130, which is trained using a 2D image 112 (e.g., an automatically segmented 2D image divided into one or more target regions) and 3D (OCT) features 124. During operation, as per... Figure 4 More fully, the neural network 130 can receive a 2D image as input and output estimated OCT parameters 140, i.e., based on the 3D features of the 2D image.

[0036] Figure 2 This is a simplified flowchart illustrating a method for estimating OCT parameters using a color image according to one embodiment of the present disclosure. Figure 2 The process shown can be called the model generation process, and... Figure 1 The system 100 shown has common elements. Therefore, regarding Figure 1 The explanation may be applied as appropriate. Figure 2 .like Figure 2 As shown in method 200, a labeled color (e.g., RGB) image (e.g., a non-mydriatic color image obtained using a fundus camera) is received as input (210), and features are generated by applying a ResNet-50 deep network, which takes a retinal image as input and features generated at the “fc1000_softmax” layer as output, along with an ARIA-based automatic feature generation method based on pixels associated with a disease such as GON (212). As described herein, training of the DL can utilize OCT parameters corresponding to the color image (i.e., the color image can be labeled using OCT parameters, and the labeled image can be used as input to the training process).

[0037] Despite Figure 2The ResNet-50 DL transmission network is illustrated, but embodiments of this disclosure may utilize other DL networks or machine learning models. Examples of machine learning models include: random forest models, including deep random forests; neural networks, including recurrent neural networks and convolutional neural networks; graph-based convolutional neural networks; quadruple neural networks; restricted Boltzmann machines; recurrent tensor networks; and gradient boosting trees. Therefore, various models, including deep learning models (e.g., neural networks with many layers), random forest models, decision trees, support vector machines (SVMs), neural networks, and K-nearest neighbors (KNNs), including those using boosting (i.e., AdaBoost), are included within the scope of this disclosure.

[0038] Then, the Glmnet method is applied to select a subset of features, such as the most important subset of features highly correlated with a certain disease, e.g., GON (214). This process can be viewed as a statistical process, reducing noise and computational complexity from less important features. Although some implementations are discussed in the context of glaucoma, as discussed more fully herein, implementations of this disclosure are applicable to a wide variety of other eye conditions, eye diseases, health conditions, and / or illnesses related to OCT parameters. To avoid overfitting, a validation method is applied, e.g., 10-fold cross-validation using a random forest (RF) bagtree model to generate more robust results (216). Although in Figure 2 The example illustrates a 10-fold cross-validation using a random forest (RF) bagtree model, but this particular validation process is not required, and other validation processes are included within the scope of this disclosure. Additionally, in some embodiments, validation is optional. Finally, method 200 includes verifying the predictive performance of the random forest model on a validation dataset (218). In some embodiments, this verification process is also optional.

[0039] It should be understood that, Figure 2 The specific steps shown provide a particular method for estimating OCT parameters using a color image according to one embodiment of this disclosure. Other sequences of steps may also be performed according to other embodiments. For example, other embodiments of this disclosure may perform the above steps in a different order. Additionally, Figure 2 The steps shown may include multiple sub-steps, which may be adapted to be performed in different orders. Additionally, extra steps may be added or removed depending on the specific application. Many variations, modifications, and alternatives will be apparent to those skilled in the art.

[0040] Figure 3This is a simplified flowchart illustrating a method for training a neural network according to one embodiment of this disclosure. Method 300 includes capturing multiple 2D images (310). The 2D images may be color (e.g., RGB) retinal images captured using a color fundus camera. Method 300 also includes segmenting a first 2D image based on a target region (ROI) (312). As follows regarding... Figures 5A-5D The ROI can be the optic disc region, the macula of the retina, or other ROIs. Additionally, in some embodiments, the ROI includes the entire retinal image or substantially the entire retinal image. In these embodiments, the segmentation process shown in step 312 is largely bypassed to provide a raw retinal image for subsequent processing.

[0041] Method 300 also includes determining whether there are additional ROIs (314). For example, if the first segmentation process generates a optic disc image as the first ROI, the additional ROI could be the macula, and the original 2D image could be segmented in process 312 to generate a macula image. Once the desired ROI has been utilized in the segmentation process, a set of segmented ROI images is generated (316). For the remaining available 2D images, processes 312 and 314 are repeated using the remaining 2D images until multiple sets of segmented ROI images are generated (320). If no additional 2D images remain (318), the multiple sets of segmented ROI images are used as input to the neural network (340).

[0042] To train the neural network, in addition to multiple sets of segmented ROI images (320), OCT images corresponding to the 2D images are captured (330), and OCT features are determined based on the OCT images (332). In some embodiments, OCT features are used to label the 2D images included in the multiple sets of segmented ROI images. Therefore, embodiments pair 2D images with corresponding OCT images to provide training data for the neural network. Thus, the neural network is trained (340) using multiple sets of segmented ROI images and OCT features as input.

[0043] It should be understood that, Figure 3 The specific steps shown provide a particular method for training a neural network according to one embodiment of this disclosure. Other sequences of steps may also be performed according to other embodiments. For example, other embodiments of this disclosure may perform the above steps in a different order. Additionally, Figure 3 The steps shown may include multiple sub-steps, which may be adapted to be performed in different orders. Additionally, extra steps may be added or removed depending on the specific application. Many variations, modifications, and alternatives will be apparent to those skilled in the art.

[0044] Figure 4 This is a simplified flowchart illustrating a method for estimating 3D features using a trained neural network according to one embodiment of the present disclosure. Method 400 includes obtaining a 2D image (e.g., a retinal fundus image) (410), and segmenting the 2D image to generate a 2D image of a target region (412). The segmentation of the 2D image can be performed using the ARIA method discussed above.

[0045] The method also includes inputting a 2D image of the target region into a neural network (414) and using the neural network to estimate OCT parameters (416). If additional 2D images of the target region are available (420), processes 412, 414, and 416 are repeated to estimate additional OCT parameters. Once all 2D images of the target region have been processed, the method terminates (422).

[0046] OCT parameters may include ONH parameters, such as edge area, optic disc area, average C / D ratio, vertical C / D ratio, or cup volume. For these ONH parameters, the 2D image of the target region is an optic disc image, and the ONH parameters are derived from an OCT optic disc scan. In other embodiments, OCT parameters include retinal nerve fiber layer (RNFL) thickness, such as average RNFL thickness. For this OCT parameter, the 2D image of the target region includes an optic disc image, and the RNFL thickness is derived from an OCT optic disc scan.

[0047] In some implementations, the OCT parameters include GCIPL thickness. GCIPL thickness can be either average GCIPL thickness or minimum GCIPL thickness. In this case, the 2D image of the target region includes a macular image, and the GCIPL thickness is derived from an OCT macular scan.

[0048] The method may further include recommending medical procedures based on estimated OCT parameters, or obtaining one or more additional 2D images, including color retinal fundus images, based on estimated OCT parameters.

[0049] Although examples of using retinal fundus images as 2D images and OCT parameters as 3D features have been discussed... Figure 4These 2D images and 3D features are determined based on 2D images of the target region formed by segmenting retinal fundus images. However, it is understood that the embodiments described herein are applicable to a variety of 2D images and a variety of 3D features. Therefore, although method 400 is discussed in the context of a specific 2D image (e.g., retinal fundus image) and a specific set of 3D features (e.g., OCT parameters), it is understood that the method is applicable to other 2D images and other 3D features. Therefore, the applicability of this method is not limited to ophthalmic or medical applications, but also includes other applications where 3D features can be estimated from 2D images. Many variations, modifications, and alternatives will be recognized by those skilled in the art.

[0050] It should be understood that, Figure 4 The specific steps shown provide a particular method for estimating OCT parameters using a trained neural network according to one embodiment of this disclosure. Other sequences of steps may also be performed according to other embodiments. For example, other embodiments of this disclosure may perform the above steps in a different order. Additionally, Figure 4 The steps shown may include multiple sub-steps, which may be adapted to be performed in different orders. Additionally, extra steps may be added or removed depending on the specific application. Many variations, modifications, and alternatives will be apparent to those skilled in the art.

[0051] To generate accurate optic disc and macular features, embodiments of this disclosure locate and segment the optic disc and macula by setting the ROI as rectangular regions around the optic disc or macula, respectively. To compute the retinal data discussed herein, the ARIA method, developed for acquiring and analyzing retinal images, is used. The ARIA method is described in more detail in U.S. Patent No. 8,787,638, the entire disclosure of which is incorporated herein by reference. Methods including Harik texture feature analysis, fractal analysis, and a set of modified pre-trained deep networks (i.e., a modified transport network resnet50) are used to create highly correlated feature pixels.

[0052] Figure 5A This is a segmentation map of the target region of the optic disc of a normal eye according to one embodiment of the present disclosure. Figure 5B This is a segmentation map of a target region of the optic disc in an eye suffering from glaucomatous optic neuropathy (GON) according to one embodiment of this disclosure. For example, after taking an image of the retina using a color fundus camera, the image is segmented into one or more target regions. Figure 5A In the process, the ROI of the visual disc region was determined, and the image was segmented, for example, by automatically generating images using the ARIA method. Figure 5A The image shown. The coordinates of the ROI in the original image can also be determined.

[0053] Figure 5C This is a segmentation map of the target region of the macula of a normal eye according to one embodiment of the present disclosure. Figure 5D This is a segmentation map of the macula of an eye suffering from GON, according to one embodiment of this disclosure. Figure 5A and Figure 5B The segmentation process shown is similar, segmenting the ROI of the retinal image to provide... Figure 5C and Figure 5D The image shows a segmented macular image.

[0054] In order to generate Figures 5A-5D The ROIs shown are located using an intensity-based method. To ensure consistency in the detection of the optic disc and macula, the intensities of various images were pre-normalized to correct for background brightness. In retinal mode, Figure 5A and Figure 5B The optic disc shown in the image exhibits the highest intensity, while the orbital cavity (i.e., Figure 5C and Figure 5D The center of the macula (as shown in the image) appears to have lower intensity in the fundus image. The peak of the probability density function (PDF) is defined as the highest intensity value in the histogram of the optic disc. Pixels are then assigned to optic disc regions. Therefore, we can locate the macular region with the lowest intensity by calculating the intensity value within the 99% confidence interval (CI) of the PDF corresponding to the center of the quality. Figure 5C and Figure 5D In the image, the center of the macula is represented by a square near the center as shown in the segmented image. After segmentation, retinal analysis data are calculated, for example using the ARIA method, and are shown in the table.

[0055] Figures 5A-5D Two ROIs are shown. Additionally, in some embodiments, the methods and systems described herein utilize raw retinal images. Many variations, modifications, and alternatives will be apparent to those skilled in the art.

[0056] All images used for training and testing were from the Department of Ophthalmology, Zhongshan Hospital, Fudan University. (For more information...) Figure 2 As described above, we applied the ResNet-50 deep network and the ARIA automatic feature generation method to generate features, and used the Glmnet method to select relevant features. Then, we applied 10-fold cross-validation using a random forest bagtree model on the main dataset, and finally confirmed the estimation performance of our random forest model on the validation dataset. The estimation performance of the quantified glaucoma parameters was evaluated by calculating the root mean square error (RMSE), mean absolute error (MAE), and Pearson correlation coefficient.

[0057] In our ensemble random forest bagtree model, we obtained a Pearson correlation coefficient of 0.640, an RMSE of 11.998, and a MAE of 9.096 mm in 10-fold cross-validation, and a Pearson correlation coefficient of 0.466, an RMSE of 13.834, and a MAE of 10.783 mm in the validation dataset.

[0058] The primary dataset consists of a total of 1,131 images paired with OCT optic disc scans and 1,021 images paired with OCT macular scans from 544 patients. The validation dataset comprises 269 images from 130 patients. In both the main and validation datasets, there were significant correlations (p < 0.001) between the estimated and true values ​​of all OCT parameters, including RNFL thickness (correlation coefficients r = 0.640 and r = 0.466), edge area (r = 0.624 and r = 0.385), optic disc area (r = 0.480 and r = 0.330), mean C / D ratio (r = 0.648 and r = 0.511), vertical C / D ratio (r = 0.678 and r = 0.529), cup volume (r = 0.578 and r = 0.443), mean GCIPL thickness (r = 0.583 and r = 0.368), and minimum GCIPL thickness (r = 0.557 and r = 0.344).

[0059] The database includes medical history and comprehensive ophthalmological examination information, including non-mydriatic color fundus retinal images (TOPCONTRC-NW100 non-mydriatic retinal camera, Tokyo, Japan), Cirrus HD-OCT (Carl Zeiss Meditec 5000, Dublin, California, USA), and Humphrey Field Analyzer (HFA, Carl Zeiss Meditec 750i, Dublin, California, USA). The Cirrus HD-OCT optic disc scan automatically positions a 3.46 mm diameter circle uniformly around the center of the optic disc and generates 200 × 200 optic disc cube data using a 6 mm square grid. The Cirrus HD-OCT macular scan includes ganglion cell OU analysis, measuring the total thickness of the ganglion cell layer and inner plexiform layer (GCL+IPL) using 512 × 128 macular cube data centered on the orbit, and generating cube data using a 6 mm square grid.

[0060] In addition to retinal images, we also collected patient medical history and OCT reports. Each image was paired with a corresponding OCT image whose date was closest to the image date. Inclusion criteria for subjects were: 1) age equal to or greater than 18 years; 2) gradeable color fundus retinal images; 3) reliable OCT scans; and 4) an interval of less than one month between the image date and the corresponding OCT date. Exclusion criteria were: 1) other ocular or systemic diseases that may affect the optic nerve; or 2) missing OCT scan data. Poor image quality was defined as the absence of visible blood vessels on the optic disc or macula. Following the manufacturer's recommendations, unreliable OCT reports were defined as inaccurate data due to obvious eye movements, involuntary blinking, or eye rotations, or reports with a signal intensity index less than 6 were excluded.

[0061] Dataset A, consisting of all images with paired OCT optic disc scans, was used to estimate RNFL thickness and ONH parameters, including marginal area, optic disc area, mean C / D ratio, vertical C / D ratio, and cup volume on color fundus retinal images of the OCT optic disc scans. Dataset B, consisting of all images with paired OCT macular scans, was used to estimate the mean GCIPL thickness and minimum GCIPL thickness of the OCT macular scans.

[0062] We labeled all images in the main datasets A and B: 1. Images of individuals without GON (Gonadotropin-Aldrich syndrome); 2. Images that may indicate GON; and 3. Images confirming GON.

[0063] Two ophthalmologists with three years of clinical experience in glaucoma assess GON based on paired, gradeable images and reliable OCT reports, according to the grading definitions shown in Table 1. If the results of the ophthalmologists are inconsistent, the final decision is made by another expert with more than ten years of experience in glaucoma.

[0064]

[0065] Table 1. Definitions of GON Classification

[0066] The Department of Ophthalmology at Zhongshan Hospital affiliated with Fudan University also collected a validation dataset. In addition, the database includes medical history, OCT (Cirrus, Carl Zeiss 5000), and visual field testing (Humphrey automated visual acuity test). Besides the inclusion and exclusion of the main dataset, subjects were excluded if their fixed visual field test loss exceeded 33% or their false positive error exceeded 15%.

[0067] These images were matched with OCT optic disc scans, OCT macular scans, and reliable VF tests. The performance of ARIA was further validated using a validation dataset with OCT optic disc and macular scans to assess RNFL thickness, ONH parameters, and GCIPL thickness on unseen images. Patients in the validation dataset did not overlap with those in the main dataset to test the generality of the methods described in this paper. The validation dataset contained three groups: 1. Control group (healthy); 2. Premyopic glaucoma group (PPG); and 3. Glaucoma group based on color fundus retinal images, OCT ONH scans, and VF tests with categorical details. Adding VF testing provides more functional glaucoma defects than structural defects, allowing for further stratified analysis by group. Premyopic glaucoma (PPG) is defined as the presence of characteristic glaucoma lesions but without VF defects. Glaucoma is defined as the presence of characteristic glaucoma lesions with corresponding VF defects. The definition of VF defect in glaucoma is: (1) at least one half-field of view has a probability of less than 5% of three point clusters on the pattern deviation map, of which at least one point has a probability of less than 1%, or two point clusters have a probability of less than 1%; (2) the glaucoma half-field of view test is outside the normal range; or (3) the pattern standard deviation is less than 5%.

[0068] The performance of the random forest bagtree model on glaucoma damage in quantified images was evaluated by assessing the accuracy and variance of predictions through the calculation of R-squared, root mean square error (RMSE), and mean absolute error (MAE). We also calculated the Pearson correlation coefficient to estimate the agreement between observed OCT parameters and the algorithm's predictions. For patient demographic data, one-way ANOVA was used to compare continuous data, and the chi-square test was used for categorical data. A p-value less than 0.05 was considered statistically significant.

[0069] As mentioned above, main dataset A consists of 1131 images paired with OCT optic disc scans, and main dataset B consists of 1021 images paired with OCT macular scans. Further details about these datasets are provided in Tables 2 and 3. In total, a total of 544 patients (190 without GON, 174 possibly with GON, and 180 diagnosed with GON) were included in the main datasets. There were significant differences in age among the three groups of patients (P < 0.001), while there were no significant differences in sex (P = 0.051).

[0070]

[0071] Table 2. Demographic data of patients and images in the main dataset

[0072] Table 3 shows the predictive performance of the random forest bagtree model on datasets A and B with 10-fold cross-validation. The correlation coefficients are interpreted as 0.00–0.10, 0.11–0.39, 0.40–0.69, 0.70–0.89, and 0.90–1.00, representing negligible, weak, moderate, strong, and very strong correlations, respectively. Our algorithm shows a significant (p<0.001) moderate correlation between the algorithm's predictions and the true values ​​of the OCT scans across all OCT parameters. Furthermore, our algorithm's estimate of the vertical C / D ratio shows the highest correlation coefficient at 0.678, with R0. 2 The correlation coefficient was 0.46, RMSE was 0.127, and MAE was 0.092; followed by the vertical C / D with a correlation coefficient of 0.648. 2 The coefficient of performance (R²) was 0.42, the RMSE was 0.128, and the MAE was 0.093. This model showed the weakest correlation in predicting visual disc area, with a coefficient of 0.480 and an R² of 0.42. 2 The value was 0.23, RMSE was 0.411, and MAE was 0.311.

[0073]

[0074] Table 3. Predictive performance of the ensemble random forest bagtree model with 10x cross-validation on the main datasets.

[0075] As described above, the validation dataset consisted of 269 images paired with OCT optic disc scans, OCT macular scans, and VF tests, including 136 glaucoma-free images, 52 PPG images, and 81 glaucoma images from 130 patients. This validation dataset is shown in Table 4. There were also significant differences in age among the three groups of patients (P<0.001), while there were no significant differences in sex (P=0.436).

[0076] Table 5 shows the predictive performance of the ensemble random forest bagtree model on the validation dataset. The column labeled "True" corresponds to the mean OCT measurements of various OCT parameters for all patients in the validation dataset (i.e., n=269 patients). The column labeled "Estimated" corresponds to the estimated values ​​of various OCT parameters for all patients in the validation dataset. Table 5 shows the estimation performance of the RF model on the validation dataset, with all OCT parameters showing a significant correlation with the predicted values ​​(p<0.001). Similarly, our algorithm also performed best, showing moderate correlation in vertical C / D and average C / D, with correlation coefficients of 0.529, RMSE of 0.130, and MAE of 0.098, and correlation coefficients of 0.511, RMSE of 0.130, and MAE of 0.100, respectively. Likewise, the model showed the weakest correlation in optic disc area prediction, with a correlation coefficient of 0.330 and R0.001. 2 The value was 0.23, RMSE was 0.411, and MAE was 0.311.

[0077]

[0078] Table 4. Demographic data in the validation dataset (n=269)

[0079] Table 5. Validation of the predictive performance of the random forest bagtree model ensembled on the dataset (n=269)

[0080] In addition, we performed auxiliary analyses on the control, PPG, and glaucoma groups, as shown in Table 6. For clarity, Table 6 is presented in three columns: Table 6A, Table 6B, and Table 6C, which together form Table 6 and provide the RNFL thickness (μm) and edge area (mm²) for the control, PPG, and glaucoma groups, respectively. 2 ), Display area (mm) 2 Average C / D, Vertical C / D, Cup Volume (mm) 3 The values ​​of average GCIPL thickness (μm) and minimum GCIPL thickness (μm) represent the predictive performance of the bagtree model, which is a centralized ensemble of random forests on a validation dataset stratified by glaucoma group.

[0081] <0.05; <0.01 Table 6A. Predictive performance of the ensemble random forest bagtree model on the validation dataset of the control group. <0.05; <0.01 Table 6B. Predictive performance of the ensemble random forest bagtree model on the validation dataset of the PPG group. <0.05; <0.01 Table 6C. Predictive performance of the ensemble random forest bagtree model on the validation dataset of the glaucoma group. Figures 6A-6H This is a box plot of OCT parameters according to one embodiment of the present disclosure. Figures 6A-6F The data presented is estimated using segmented visual disc images. Figures 6G-6H The data presented is estimated using segmented macular images.

[0082] Figure 6A Box plots showing the mean RNFL thickness (in micrometers) of patients with no GON, possible GON, and confirmed GON. Figure 6B Box plots showing the marginal area (in square millimeters) of patients who did not have GON, were likely to have GON, and were diagnosed with GON. Figure 6C Box plots showing optic disc area (in square millimeters) for patients who did not have GON, were likely to have GON, and were diagnosed with GON. Figure 6D The average C / D box plots are shown for patients who do not have GON, may have GON, and are diagnosed with GON. Figure 6E Vertical C / D box plots are shown for patients who do not have GON, may have GON, and have been diagnosed with GON. Figure 6F Box plots showing cup volumes (in square millimeters) of patients who did not have GON, were likely to have GON, and were diagnosed with GON.

[0083] Figure 6G Box plots showing the mean GCIPL thickness (in micrometers) in patients with no GON, possible GON, and confirmed GON. Figure 6H Box plots showing the minimum GCIPL thickness (in micrometers) for patients without GON, those possibly with GON, and those diagnosed with GON.

[0084] The estimated OCT parameters in the glaucoma group were superior to those in the control and PPG groups. In the glaucoma group, most predicted values, except for the optic disc area parameter, were significantly correlated with the true OCT values. In the control, PPG, and glaucoma groups, three OCT parameters showed significant correlations across all groups: mean C / D (p<0.001, 0.019, <0.001), vertical C / D (p=0.001, 0.017, <0.001), and cup volume (p<0.001, 0.016, 0.045).

[0085] Figures 7A-7G This is a violin plot illustrating the true OCT parameters and the estimated OCT parameters according to one embodiment of the present disclosure. Figures 7A-7G The violin plots for each OCT parameter shown also illustrate the relationship between the true OCT values ​​and the estimated values ​​of the OCT parameters estimated using the methods and systems described in this paper, stratified by group in the validation dataset.

[0086] Figure 7A A violin plot showing the mean RNFL thickness (in micrometers) of patients in the control group, PPG group, and glaucoma group is presented. Figure 7B Violin plots showing the marginal area (in square millimeters) of patients in the control group, PPG group, and glaucoma group are presented. Figure 7C Violin plots showing the optic disc area (in square millimeters) of patients in the control group, PPG group, and glaucoma group are presented. Figure 7D A violin plot showing the mean C / D ratio for patients in the control group, PPG group, and glaucoma group is presented. Figure 7E The vertical C / D ratio of patients in the control group, PPG group, and glaucoma group is shown in violin diagrams. Figure 7F The violin diagram shows the cup volume (in square millimeters) of patients in the control group, PPG group, and glaucoma group.

[0087] Figure 7G A violin plot showing the mean GCIPL thickness (in micrometers) of patients in the control group, PPG group, and glaucoma group is presented. Figure 7H A violin plot showing the minimum GCIPL thickness (in micrometers) in patients in the control group, PPG group, and glaucoma group is presented.

[0088] In our study, the ARIA method demonstrated good predictive performance and consistency in the quantitative assessment of eight consecutive OCT parameters. In the 10-fold cross-validation and validation datasets, the OCT values ​​estimated by ARIA showed significant correlations with the true OCT values ​​for all parameters. Therefore, embodiments of this disclosure provide methods and systems for developing and validating automated analysis methods that estimate comprehensive OCT parameters by evaluating non-mydriatic color fundus retinal images. Thus, embodiments of this disclosure, including those incorporating the ARIA method, provide a convenient, cost-effective, and accurate tool for screening, diagnosing, and monitoring glaucoma and various other diseases assessable by OCT scanning.

[0089] Furthermore, by comprehensively evaluating the combined RNFL thickness, ONH parameters, and GCIPL thickness according to the methods described in this paper, it may be possible to accurately detect the progressive changes of the disease over time.

[0090] The method and system described in this paper perform well in predicting mean C / D and vertical C / D, with correlation coefficients of 0.648 and 0.678, respectively, in 10-fold cross-validation, and 0.511 and 0.529, respectively, in the validation dataset. This can be explained by the two-dimensional nature of these parameters, which should be more direct and easier to estimate based on 2D fundus images.

[0091] In stratified validation, ARIA outperformed the control and PPG groups in the glaucoma group. Therefore, our algorithm may be more beneficial for monitoring glaucoma progression in patients. On the other hand, glaucoma severity significantly impacts the diagnostic performance of CirrusHD-OCT. One study found that for the OCT parameter of mean RNFL thickness, the AUCs were 0.962, 0.932, 0.886, and 0.822 when the standard automated perimeter visual field index (VFI) was 70%, 80%, 90%, and 100%, respectively. Therefore, accurate estimation of OCT parameters in the glaucoma group also contributes to accurate glaucoma detection, providing information for ophthalmologists and automated methods.

[0092] Figure 8 This is a simplified schematic diagram illustrating an OCT parameter estimation system according to one embodiment of the present disclosure. The OCT parameter estimation system 800 includes a fundus camera 810 capable of capturing color (e.g., RGB) images of the retina. The OCT parameter estimation system 800 also includes an OCT camera 812 operable for capturing and characterizing 3D parameters corresponding to the retina.

[0093] The OCT parameter estimation system 800 also includes a controller 820, a processor 822, an input / output system 824, and a memory 826. The controller 820 may be a computer controller for operating various system components, such as acquiring and processing images captured using a fundus camera 810 and OCT parameters obtained from an OCT camera 812. The processor 822 may execute the ARIA method described herein. Additionally, the processor 822 may implement the neural networks and other processing described herein. Thus, captured images are provided to the processor 822, which may be a computer processor coupled to the input / output system 824. The various components of the OCT parameter estimation system 800 are connected via an interface bus 830, which provides control and data signals for sending and receiving control and data signals from one or more of the various components.

[0094] Various examples of this disclosure are provided below. As used below, any reference to a series of examples should be understood as a reference to each individual example in the examples (e.g., "Examples 1-4" should be understood as "Examples 1, 2, 3 or 4").

[0095] Example 1 is a method for estimating optical coherence tomography (OCT) parameters, the method comprising: obtaining a retinal fundus image; segmenting the retinal fundus image to generate a 2D image of a target region; inputting the 2D image of the target region into a neural network; and using the neural network to generate estimated OCT parameters.

[0096] Example 2 is the method of Example 1, where the OCT parameters include the thickness of the retinal nerve fiber layer (RNFL).

[0097] Example 3 is the method of Example 1-2, where the RNFL thickness includes the average RNFL thickness.

[0098] Example 4 is the method of Examples 1-3, where the 2D image of the target region includes the optic disc image, and the RNFL thickness is derived from the OCT optic disc scan.

[0099] Example 5 is the method of Examples 1-4, where the OCT parameters include the optic nerve head (ONH) parameters.

[0100] Example 6 is the method of Examples 1-5, where the ONH parameters include edge area, viewing area, average cup-to-dish ratio (C / D), vertical C / D ratio, or cup volume.

[0101] Example 7 is the method of Examples 1-6, where the 2D image of the target region includes a visual disc image, and the ONH parameter is derived from an OCT visual disc scan.

[0102] Example 8 is the method of Examples 1-7, where the OCT parameters include the thickness of the ganglion cell-internal plexiform layer (GCIPL).

[0103] Example 9 is the method of Examples 1-8, where the GCIPL thickness includes the average GCIPL thickness.

[0104] Example 10 is the method of Examples 1-9, where the GCIPL thickness includes the minimum GCIPL thickness.

[0105] Example 11 is a method of Examples 1-10, wherein the 2D image of the target region includes a macular image, and the GCIPL thickness is derived from an OCT macular scan.

[0106] Example 12 is the method of Examples 1-11, further including recommending medical procedures based on estimated OCT parameters.

[0107] Example 13 is a method of Examples 1-12, further comprising obtaining one or more additional retinal fundus images based on estimated OCT parameters.

[0108] Example 14 is a method for diagnosing, indicating severity, or predicting and monitoring the progression of one or more diseases associated with OCT parameters, the method comprising: providing multiple color images; receiving multiple OCT datasets, each OCT dataset corresponding to one of the multiple color images; training a deep learning model using the multiple color images and the multiple OCT datasets; providing a patient color image; providing the patient color image as input to the deep learning model; and estimating the OCT parameters using the deep learning model.

[0109] Example 15 is the method of Example 14, wherein one or more diseases include neuro-ophthalmic diseases, including glaucoma, non-arterial ischemic optic neuropathy (NAION), Leber hereditary optic neuropathy (LHON), compressive optic neuropathy, optic disc edema, idiopathic intracranial hypertension, or neuroretinitis (NMO) spectrum disorders.

[0110] Example 16 is the approach of Examples 14-15, wherein one or more diseases include neurodegenerative diseases, including Alzheimer's disease (AD), mild cognitive impairment (MCI), Parkinson's disease, dementia, Huntington's disease, autism spectrum disorder (ASD), attention deficit hyperactivity disorder (ADHD), or mental health diagnoses including depression.

[0111] Example 17 is a method of Examples 14-16, wherein one or more diseases include diseases related to one or more OCT parameters, including hypertensive retinopathy, hypertension, intracranial tumors affecting the visual pathway, sphenoid sinus and parasphenoid sinus lesions, axonal degeneration of the eye in patients with multiple sclerosis (MS), diabetic peripheral neuropathy (DPN), or type 1 diabetes.

[0112] Example 18 is a method of Examples 14-17, in which one or more diseases include brain atrophy or spinal cord lesions as detected by magnetic resonance imaging.

[0113] Example 19 is a method of Examples 14-18, in which multiple color images include 2D fundus camera images.

[0114] Example 20 is a method of Examples 14-19, where multiple color images include segmented target region images.

[0115] Example 21 is a method of Examples 14-20, which further includes: applying statistical analysis to the features corresponding to the deep learning model; and extracting a subset of the features to provide a set of salient features.

[0116] Example 22 is a system including: a fundus camera; a memory; and a processor coupled to the memory, wherein the processor is configured to: acquire a retinal fundus image; segment the retinal fundus image to generate a 2D image of a target region; and use the processor to generate estimated OCT parameters.

[0117] Example 23 is a system similar to Example 22, where the processor implements a neural network.

[0118] Example 24 is the system of Examples 22-23, where the OCT parameters include the thickness of the retinal nerve fiber layer (RNFL).

[0119] Example 25 is the system of Examples 22-24, where the RNFL thickness includes the average RNFL thickness.

[0120] Example 26 is a system similar to Examples 22-25, where the 2D image of the target region includes a visual disc image, and the RNFL thickness is derived from an OCT visual disc scan.

[0121] Example 27 is the system of Examples 22-26, where the OCT parameters include the optic nerve head (ONH) parameters.

[0122] Example 28 is the system of Examples 22-27, where the ONH parameters include edge area, viewing area, average cup-to-disc ratio (C / D), vertical C / D ratio, or cup volume.

[0123] Example 29 is a system similar to Examples 22-28, where the 2D image of the target region includes a visual disc image, and the ONH parameters are derived from an OCT visual disc scan.

[0124] Example 30 is the system of Examples 22-29, where the OCT parameters include the thickness of the ganglion cell-internal plexiform layer (GCIPL).

[0125] Example 31 is the system of Examples 22-30, where the GCIPL thickness includes the average GCIPL thickness.

[0126] Example 32 is a system of Examples 22-31, where the GCIPL thickness includes the minimum GCIPL thickness.

[0127] Example 33 is a system similar to Examples 22-32, where the 2D image of the target region includes a macular image, and the GCIPL thickness is derived from an OCT macular scan.

[0128] Example 34 is a system of Examples 22-33, wherein the processor is further configured to obtain one or more additional retinal fundus images based on the estimated OCT parameters.

[0129] In the foregoing specification, this disclosure has been described with reference to specific embodiments. However, it will be apparent that various modifications and alterations can be made thereto without departing from the broader spirit and scope of this disclosure. Therefore, this specification and accompanying drawings should be considered illustrative rather than restrictive.

[0130] In fact, it is understood that the systems and methods of this disclosure each have multiple innovative aspects, none of which is the sole responsibility or necessary condition for the ideal properties disclosed herein. The various features and processes described above can be used independently or in combination in various ways. All possible combinations and sub-combinations fall within the scope of this disclosure.

[0131] Some features described in this specification as individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. Furthermore, although the features described above may be described as operating in certain combinations, or even initially claimed to be so, in some cases one or more features in the claimed combination may be removed from the combination, and the claimed combination may be for a sub-combination or a variation thereof. No single feature or set of features is necessary or indispensable to every embodiment.

[0132] It is understood that the conditional language used herein, such as “may,” “can,” “possibly,” “maybe,” “for example,” etc., unless specifically stated otherwise or understood in the context, is generally intended to express that certain embodiments include certain features, elements, and / or steps, while other embodiments do not include these features, elements, and / or steps. Therefore, such conditional language generally does not imply that features, elements, and / or steps are necessary for one or more embodiments, nor does it imply that one or more embodiments must include logic for determining, with or without author input or prompting, whether such features, elements, and / or steps are included in any particular embodiment, or whether they are to be performed in any particular embodiment. The terms “including,” “comprising,” “having,” etc., are synonyms and are used in an open-ended manner, not excluding additional elements, features, behaviors, operations, etc. Additionally, the term “or” is used in an inclusive sense (not in an exclusive sense), so, for example, when used to connect a series of elements, the term “or” refers to one, some, or all of the elements in that series. Furthermore, unless otherwise stated, the articles “a,” “an,” and “the” used in this application and the appended claims should be understood as “one or more” or “at least one.” Similarly, while the operations in the accompanying drawings may be depicted in a specific order, it should be understood that these operations do not necessarily have to be performed in the specific order or sequence shown, nor is it necessary to perform all the operations shown to achieve the desired effect. Furthermore, the drawings may schematically depict one or more example processes in the form of flowcharts. However, other operations not depicted may be added to the illustrative example methods and processes. For example, one or more additional operations may be performed before, after, simultaneously with, or between any of the operations shown. Additionally, these operations may be rearranged or reordered in other embodiments. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system components in the above embodiments should not be construed as requiring such separation in all embodiments; it should be understood that the described program components and systems can generally be integrated into a single software product or packaged into multiple software products. Additionally, other embodiments also fall within the scope of the following claims. In some cases, the actions described in the claims may be performed in a different order, but the desired effect may still be achieved.

[0133] Therefore, the claims are not intended to be limited to the embodiments shown herein, but are to be given the broadest scope consistent with this disclosure, its principles, and its novel features. It should also be understood that the examples and embodiments described herein are for illustrative purposes only, and various modifications or alterations can be made to these examples and embodiments by those skilled in the art, and such modifications or alterations should be included within the spirit and scope of this disclosure and the appended claims.

Claims

1. A method for estimating optical coherence tomography (OCT) parameters, the method comprising: Obtain images of the retina and fundus; The retinal fundus image is segmented to generate a 2D image of the target region; The 2D image of the target region is input into the neural network; as well as The estimated OCT parameters are generated using the neural network.

2. The method according to claim 1, wherein, The OCT parameters include the thickness of the retinal nerve fiber layer (RNFL).

3. The method according to claim 2, wherein, The RNFL thickness includes the average RNFL thickness.

4. The method according to claim 2, wherein, The target area 2D image includes a visual disc image, and the RNFL thickness is derived from an OCT visual disc scan.

5. The method according to claim 1, wherein, The OCT parameters include optic nerve head (ONH) parameters.

6. The method according to claim 5, wherein, The ONH parameters include edge area, viewing area, average cup-to-dish ratio (C / D), vertical C / D ratio, or cup volume.

7. The method according to claim 6, wherein, The target area 2D image includes a visual disc image, and the ONH parameters are derived from an OCT visual disc scan.

8. The method according to claim 1, wherein, The OCT parameters include the thickness of the ganglion cell-internal plexiform layer (GCIPL).

9. The method according to claim 8, wherein, The GCIPL thickness includes the average GCIPL thickness.

10. The method according to claim 8, wherein, The GCIPL thickness includes the minimum GCIPL thickness.

11. The method according to claim 8, wherein, The target region 2D image includes a macular image, and the GCIPL thickness is derived from an OCT macular scan.

12. The method of claim 1, further comprising recommending a medical procedure based on the estimated OCT parameters.

13. The method of claim 1, further comprising obtaining one or more additional retinal fundus images based on the estimated OCT parameters.

14. A method for diagnosing, indicating severity, or predicting and monitoring the progression of one or more diseases associated with OCT parameters, the method comprising: Provides multiple color images; Receive multiple OCT datasets, each of which corresponds to one of the multiple color images; A deep learning model was trained using the multiple color images and the multiple OCT datasets; Provide color images of the patient; The patient's color image is provided as input to the deep learning model; as well as The deep learning model is used to estimate the OCT parameters.

15. The method according to claim 14, wherein, One or more of the diseases include neuroophthalmic diseases, including glaucoma, non-arterial ischemic optic neuropathy (NAION), Leber hereditary optic neuropathy (LHON), compressive optic neuropathy, optic disc edema, idiopathic intracranial hypertension, or neuroretinitis (NMO) spectrum disorders.

16. The method of claim 14, wherein, The one or more diseases mentioned include neurodegenerative diseases, including Alzheimer's disease (AD), mild cognitive impairment (MCI), Parkinson's disease, dementia, Huntington's disease, autism spectrum disorder (ASD), attention deficit hyperactivity disorder (ADHD), or mental health diagnoses including depression.

17. The method of claim 14, wherein, The one or more diseases mentioned include diseases associated with one or more OCT parameters, including hypertensive retinopathy, hypertension, intracranial tumors affecting the visual pathway, sphenoid sinus and parasphenoid sinus lesions, axonal degeneration of the eye in patients with multiple sclerosis (MS), diabetic peripheral neuropathy (DPN), or type 1 diabetes.

18. The method according to claim 14, wherein, The one or more diseases mentioned include brain atrophy or spinal cord lesions as detected by magnetic resonance imaging.

19. The method of claim 14, wherein, The multiple color images include 2D fundus camera images.

20. The method of claim 14, wherein, The plurality of color images include segmented target region images.

21. The method of claim 14, further comprising: Statistical analysis is applied to the features corresponding to the deep learning model; as well as Extract a subset of the features to provide a set of salient features.

22. A system comprising: Fundus camera; Memory; as well as A processor is connected to the memory, wherein the processor is configured to: Obtain images of the retina and fundus; Segmenting the retinal fundus image to generate a 2D image of the target region; and The processor is used to generate estimated OCT parameters.

23. The system according to claim 22, wherein, The processor implements a neural network.

24. The system according to claim 22, wherein, The OCT parameters include the thickness of the retinal nerve fiber layer (RNFL).

25. The system according to claim 24, wherein, The RNFL thickness includes the average RNFL thickness.

26. The system according to claim 24, wherein, The target area 2D image includes a visual disc image, and the RNFL thickness is derived from an OCT visual disc scan.

27. The system according to claim 22, wherein, The OCT parameters include optic nerve head (ONH) parameters.

28. The system according to claim 27, wherein, The ONH parameters include edge area, viewing area, average cup-to-dish ratio (C / D), vertical C / D ratio, or cup volume.

29. The system according to claim 27, wherein, The target area 2D image includes a visual disc image, and the ONH parameters are derived from an OCT visual disc scan.

30. The system according to claim 22, wherein, The OCT parameters include the thickness of the ganglion cell-internal plexiform layer (GCIPL).

31. The system according to claim 30, wherein, The GCIPL thickness includes the average GCIPL thickness.

32. The system according to claim 30, wherein, The GCIPL thickness includes the minimum GCIPL thickness.

33. The system according to claim 30, wherein, The target region 2D image includes a macular image, and the GCIPL thickness is derived from an OCT macular scan.

34. The system according to claim 22, wherein, The processor is further configured to obtain one or more additional retinal fundus images based on the estimated OCT parameters.

Images