Multi-modal ophthalmic imaging system and method for imaging eye of patient

Abstract

The invention relates to a multi-modal ophthalmic imaging system and method for imaging an eye of a patient. A multi-modality ophthalmic imaging system for acquiring an ophthalmic image of a patient's eye includes a first imaging module operable in a first imaging modality for acquiring a first ophthalmic image of a first portion of the patient's eye and one or more further imaging modules operable in a second imaging modality for acquiring a second ophthalmic image of a second portion of the patient's eye. The one or more further imaging modules are each operable in a different imaging modality. The imaging system further comprises a control module arranged to detect an anomaly in the first ophthalmic image indicative of an ocular disease, determine a region of interest in the eye of the patient based on the detected anomaly, where the region of interest is a region within the eye of the patient predicted to contain a second anomaly, and determine a region of interest in the eye of the patient based on the detected anomaly. And determining a second imaging modality for imaging the determined region of interest based on the second anomaly, and controlling the selected second imaging module to acquire a second ophthalmic image of the determined region of interest.

Description

[0001] field

[0002] Example aspects herein generally relate to multi-modality ophthalmic imaging systems and methods of imaging a patient's eye using multi-modality ophthalmic imaging systems. background

[0003] Multimodal ophthalmic imaging devices allow clinicians to acquire ophthalmic images of a patient's eye using a variety of different imaging modalities. The imaging modality with which the ophthalmic imaging system operates is typically selected by the clinician based on the type of pathology the clinician wants to image within the patient's eye. In some cases, the clinician may acquire multiple ophthalmic images of a patient's eye using one or more imaging modalities of the ophthalmic imaging system.

[0004] When a clinician images a patient's eye, the clinician may first acquire an en-face scanning laser ophthalmoscopy (SLO) image of the patient's retina. The clinician may then review the en-face SLO image of the patient's retina to identify one or more areas that contain some indication of pathology that is not present in comparable images of a healthy eye. The presence of such areas may indicate a disease or other unhealthy state of the subject. Once the clinician has identified one or more areas that contain indications of pathology, the clinician may capture additional ophthalmic images using one or more different imaging modalities, such as an ophthalmic coherence tomography (OCT) image of the area of ​​the eye.

[0005] The process of capturing an initial ophthalmic image, identifying areas indicative of disease, and then capturing additional ophthalmic images of the patient's eye using additional imaging modalities is very time consuming for both the clinician and the patient. Furthermore, if the ophthalmic imaging system is operated by an imaging technician who is not skilled in diagnosing ophthalmic disease, the initial ophthalmic image may have to be sent to be reviewed by an ophthalmologist for analysis to determine whether additional ophthalmic images are needed. This may result in the patient having to be recalled to the clinic multiple times for multiple imaging by an imaging technician based on feedback from the ophthalmologist.

[0006] The process of reviewing initial ophthalmic images to identify pathology is a time-consuming process that requires a highly skilled ophthalmologist to review the images. This can significantly reduce the clinic's patient throughput and often requires the patient to be recalled to the clinic to capture additional ophthalmic images in order to diagnose the eye disease.

[0007] Overview

[0008] According to a first example aspect, a multimodal ophthalmic imaging system for acquiring an ophthalmic image of a patient's eye is provided, the multimodal ophthalmic imaging system comprising: a first imaging module, which can be operated in a first imaging modality for acquiring a first ophthalmic image of the patient's eye; one or more additional imaging modules, each of which can be operated in a corresponding imaging modality, wherein the respective imaging modalities of the one or more additional imaging modules are different from the first imaging modality; and a control module, which is arranged to: detect an abnormality in the first ophthalmic image acquired by the first imaging module; determine a region of interest in the patient's eye based on the abnormality detected in the first ophthalmic image, wherein the determined region of interest is a region within the patient's eye that is predicted to contain a second abnormality; determine a second imaging modality for imaging the determined region of interest based on the second abnormality, the second imaging modality being different from the first imaging modality; select a second imaging module that can be operated in the determined second imaging modality from one or more additional imaging modules; and control the selected second imaging module to acquire a second ophthalmic image of the determined region of interest.

[0009] In an example embodiment, the abnormality detected in the first ophthalmic image may indicate a first stage of a multi-stage or progressive eye disease, and the second abnormality may indicate a second stage of a multi-stage or progressive eye disease.

[0010] In another example embodiment, the first imaging module may include a scanning laser ophthalmoscope (SLO), and the first ophthalmic image may be a fundus image of the patient's eye.

[0011] In another example embodiment, the control module may be further arranged to determine one or more image acquisition parameters of the second imaging module based on the second abnormality for controlling the second imaging module to acquire a second ophthalmic image of the determined region of interest.

[0012] In an example embodiment, the control module may be communicatively coupled to a database comprising medical records indicating a medical history of the patient, and wherein the control module is arranged to retrieve the medical records of the patient from the database and determine the region of interest based on both the detected abnormality and the medical history of the patient.

[0013] In an example embodiment, the control module may be configured to detect a discontinuity at a boundary of the second ophthalmic image, wherein the discontinuity at the boundary indicates a second abnormal cropping located within the second ophthalmic image. In response to detecting the discontinuity at the boundary of the second ophthalmic image, the control module may be arranged to: increase the size of the determined region of interest to generate an enlarged region of interest; and control the second imaging module to acquire an ophthalmic image of the enlarged region of interest.

[0014] In another example embodiment, the control module may be arranged to: determine a second region of interest based on the detected abnormality, wherein the second region of interest is a region containing the detected abnormality; determine a third imaging modality for imaging the determined second region of interest based on the detected abnormality, the third imaging modality being different from the first imaging modality; select a third imaging module operable under the determined third imaging modality from one or more additional imaging modules; and control the selected third imaging module to acquire a third ophthalmic image of the determined second region of interest.

[0015] According to a second example aspect, a method for imaging a patient's eye is provided, the method comprising: acquiring a first ophthalmic image of the patient's eye using a first imaging module operating under a first imaging modality; detecting an abnormality in the first ophthalmic image acquired by the first imaging module; determining a region of interest in the patient's eye based on the abnormality detected in the first ophthalmic image, wherein the determined region of interest is a region within the patient's eye predicted to contain a second abnormality; determining a second imaging modality for imaging the determined region of interest based on the second abnormality, the second imaging modality being different from the first imaging modality; and acquiring a second ophthalmic image of the determined region of interest using a second imaging module operating under the determined second imaging modality.

[0016] In an example embodiment, the abnormality detected in the first ophthalmic image may indicate a first stage of a multi-stage or progressive eye disease, and the second abnormality may indicate a second stage of a multi-stage or progressive eye disease.

[0017] In an example embodiment, the method may include determining whether the second anomaly is captured within the second ophthalmic image by detecting a discontinuity at a boundary of the second ophthalmic image, wherein detecting the presence of a discontinuity at the boundary may indicate cropping of the second anomaly within the second ophthalmic image.

[0018] In an example embodiment, the method may include, in response to detecting the presence of a discontinuity at a boundary of the second ophthalmic image, increasing the size of the determined region of interest to generate an enlarged region of interest, and acquiring a further ophthalmic image of the enlarged region of interest.

[0019] In another example embodiment, the method may include receiving a medical record of the patient including a medical history of the patient, and wherein the region of interest may be determined based on a combination of the abnormality detected in the first ophthalmic image and the received medical history of the patient.

[0020] In another example embodiment, the method may further include determining one or more image acquisition parameters of the second imaging module based on the second abnormality, and operating the second imaging module with the determined image acquisition parameters to acquire a second ophthalmic image.

[0021] In an example embodiment, the method may include determining a second region of interest based on the detected abnormality, wherein the second region of interest is a region containing the detected abnormality; determining a third imaging modality for imaging the determined second region of interest based on the detected abnormality, the third imaging modality being different from the first imaging modality; selecting a third imaging module operable under the determined third imaging modality from one or more additional imaging modules; and controlling the selected third imaging module to acquire a third ophthalmic image of the determined second region of interest.

[0022] In a third example aspect, a multimodal ophthalmic imaging system for acquiring an ophthalmic image of an eye of a patient is provided, the multimodal ophthalmic imaging system comprising: a first imaging module, the first imaging module being operable in a first imaging modality for acquiring a first ophthalmic image of the eye of the patient; one or more additional imaging modules, each of the one or more additional imaging modules being operable in a corresponding imaging modality, wherein the respective imaging modalities of the one or more additional imaging modules are different from the first imaging modality; and a control module, the control module being arranged to: detect an abnormality in the first ophthalmic image acquired by the first ophthalmic imaging module, the abnormality being indicative of a first stage of a multi-stage eye disease segment; predicting a second stage of a multi-stage eye disease based on the detected abnormality; determining a region of interest in the patient's eye based on the predicted second stage of the multi-stage eye disease, wherein the determined region of interest is a region within the patient's eye predicted to contain a second abnormality indicative of the second stage of the multi-stage eye disease; determining a second imaging modality for imaging the determined region of interest based on the second abnormality, the second imaging modality being different from the first imaging modality; selecting a second imaging module operable under the determined second imaging modality from one or more additional imaging modules; and controlling the selected second imaging module to acquire a second ophthalmic image of the determined region of interest.

[0023] In a fourth example aspect, a method for imaging an eye of a patient is provided, the method comprising: acquiring a first ophthalmic image of the patient's eye using a first imaging module operating under a first imaging modality; detecting an abnormality in the first ophthalmic image acquired by the first ophthalmic imaging module; predicting a second stage of a multi-stage ocular disease based on the detected abnormality; determining a region of interest in the patient's eye based on the predicted second stage of the multi-stage ocular disease, wherein the determined region of interest is a region within the patient's eye predicted to contain a second abnormality indicating the second stage of the multi-stage ocular disease; determining a second imaging modality for imaging the determined region of interest based on the second abnormality, the second imaging modality being different from the first imaging modality; and acquiring a second ophthalmic image of the determined region of interest using a second imaging module operating under the determined second imaging modality. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Example embodiments will now be explained in detail, by way of non-limiting example only, with reference to the accompanying drawings described below.Unless otherwise indicated, like reference numerals appearing in different figures of the drawings may represent the same element or a functionally similar element.

[0025] Figure 1 is a schematic diagram of a multi-modality ophthalmic imaging system according to a first example embodiment of the present invention.

[0026] Figure 2 Is Figure 1 Schematic diagram of a fundus image captured by a multimodal ophthalmic imaging system.

[0027] Figure 3 is a flow chart illustrating a method of imaging a patient's eye according to example embodiments herein.

[0028] Figure 4 is a schematic diagram of a multi-modality ophthalmic imaging system according to a second example embodiment of the present invention.

[0029] Figure 5 Is Figure 1 or Figure 4 Schematic diagram of a first ophthalmic image captured by a first imaging module within a multi-modal ophthalmic imaging system.

[0030] Figure 6 is an example hardware implementation of an apparatus operable as a control module of a multi-modality ophthalmic imaging system according to example embodiments herein. Detailed Description

[0031] In view of the background information provided above, the inventors have designed a multimodal ophthalmic imaging system having a control module or processor arranged to detect an abnormality in a first ophthalmic image indicating a first stage of a progressive or multi-stage ophthalmic disease, the first ophthalmic image being acquired by a first ophthalmic imaging module operating in a first imaging modality. Upon detecting the abnormality in the first ophthalmic image, the control module is arranged to determine one or more regions of interest within the patient's eye based on the detected abnormality. The one or more determined regions of interest may be regions of the patient's eye that are susceptible to the progression or further stages of the multi-stage ophthalmic disease indicated by the detected abnormality. The control module may determine a second imaging modality for acquiring a second ophthalmic image of the determined region of interest based on the detected abnormality. The control module is configured to select a second imaging module operable under the determined second imaging modality, use the second imaging module to acquire a second ophthalmic image of one or more regions of interest, and to acquire an additional ophthalmic image of a region within the patient's eye that is susceptible to progression of the multi-stage ocular disease and is predicted or expected to contain a second abnormality indicative of progression of the multi-stage ocular disease.

[0032] For example, the multimodal ophthalmic imaging system may be an ophthalmic imaging system having a scanning laser ophthalmoscope (SLO) imaging module and an optical coherence tomography (OCT) imaging module. The SLO imaging module may acquire a fundus image of the patient's eye. The control module may analyze or search the captured fundus image to check for the presence of an abnormality indicating a multi-stage or progressive ocular disease. Upon detecting an abnormality in the fundus image, the control module may determine a region of interest in the fundus image that is predicted to contain a second abnormality associated with the progression of the multi-stage ocular disease indicated by the detected abnormality. The determined region of interest may be a region for further imaging by a second imaging modality. For example, the region of interest may be a region in the periphery of the patient's retina that is predicted to contain a second abnormality associated with the progression of the multi-stage ocular disease based on the detection of the abnormality in the fundus image.

[0033] The control module is configured to select a second imaging modality for acquiring an ophthalmic image of the determined region of interest based on the detected abnormality and / or based on a pathology type that is expected to manifest as a second abnormality in the determined region of interest based on an expected stage of progression of the multi-stage ocular disease indicated by the detected abnormality. For example, if the control module predicts that the second abnormality will be a retinal hemorrhage, such that the retinal hemorrhage may be present at the determined region of interest, the control module may determine that the OCT angiography (OCTA) imaging module should be operated in an OCT imaging modality to acquire a second ophthalmic image of the determined region of interest. The OCT imaging module is then operated in an OCTA imaging modality to acquire a second ophthalmic image of the determined region of interest.

[0034] Advantageously, the multimodal ophthalmic imaging system allows for the acquisition of multiple images of a patient's eye in a single imaging session. This can be helpful in capturing images necessary to accurately diagnose the extent of progression of a multi-stage ocular disease in a patient's eye during a single visit to a clinician. In addition, the control module can select a second imaging modality for imaging a predicted second abnormality contained in a determined region of interest based on the detected abnormality and / or based on a second abnormality expected to be contained in the determined region of interest. This advantageously allows the multimodal ophthalmic imaging system to be operated to capture high-quality ocular images of abnormalities in the patient's eye, which helps the ophthalmologist diagnose the progression state of the patient's ocular disease. In addition, the control module can change the acquisition parameters of the second imaging module based on the detected abnormality and / or based on a predicted second abnormality associated with the progression of the multi-stage ocular disease to optimize the quality of the second ophthalmic image, which is expected to be present in the determined region of interest. Optimizing the quality of the second ophthalmic image allows the ophthalmic imaging system to acquire high-quality images of the patient's eye, so that the ophthalmologist can accurately diagnose and assess the severity of the ocular disease present in the patient's eye without requiring the patient to return to the clinic for further imaging.

[0035] In order to place the example embodiments into proper context, reference will now be made to Figure 1 , Figure 1 A multimodal ophthalmic imaging system 10 for imaging a patient's eye 18 is schematically shown. The multimodal ophthalmic imaging system 10 includes a scanning laser ophthalmoscope (SLO) imaging module 14 and an optical coherence tomography (OCT) imaging module 16 for acquiring SLO images and OCT images of the patient's eye 18, respectively. The SLO imaging module 14 is an example of a first imaging module or system, and the OCT imaging module 16 is an example of a second imaging module or system. The SLO imaging module 14 and the OCT imaging module 16 can each operate in a plurality of different imaging modalities.

[0036] As in the present example embodiment, the ophthalmic imaging system 10 may further include an optical system 15 optically coupled to the SLO imaging module 14 and the OCT imaging module 16 and arranged to transmit light from the SLO imaging module 14 and the OCT imaging module 16 to and from the patient's eye 18. The optical system 15 may include focusing optics and a scanning system (not shown) for scanning a light beam emitted by the SLO imaging module 14 and / or the OCT imaging module 16 across a target imaging region of the patient's eye 18. For example, the scanning system may include one or more scanning mirrors or galvanometers for scanning a light beam emitted by either of the imaging modules 14, 16 across a target imaging region of the eye 18, such as across a region of interest in the patient's retina. The SLO imaging module 14 and the OCT imaging module 16 may include one or more photodetectors (not shown) for detecting light reflected by the patient's eye 18, and a signal processing device (not shown) arranged to generate an ophthalmic image based on the output of the one or more photodetectors. The optical system 15 is arranged to transmit the light reflected by the patient's eye 18 to the photodetectors located within the SLO imaging module 14 and the OCT imaging module 16, so that the signal processing device can generate an ophthalmic image of the patient's eye 18 based on the light reflected or scattered by the eye 18.

[0037] The ophthalmic imaging system 10 also includes a processor or control module 12 coupled to the SLO imaging module 14, the OCT imaging module 16, and the optical system 15. The control module 12 is configured to control the SLO imaging module 14, the OCT imaging module 16, and the optical system 15. The processor or control module 12 is a component of the multimodal ophthalmic imaging system 10 and is coupled to the SLO imaging module 14, the OCT imaging module 16, and the optical system 15 for controlling these components and for exchanging data and information therebetween.

[0038] The control module 12 is arranged to select an operating mode of the multimodal ophthalmic imaging system 10, and to operate one or more of the SLO imaging module 14 and the OCT imaging module 16 according to the selected operating mode to acquire an ophthalmic image of the patient's eye 18. In addition, the control module 12 can control the optical system 15 to select a target imaging area of ​​the patient's eye 18 to be imaged, so that a scanning system within the optical system 15 can scan a light beam emitted by any one of the SLO imaging module 14 and the OCT imaging module 16 on the target imaging area within the patient's eye 18 to acquire one or more ophthalmic images of the selected imaging area.

[0039] The multimodal ophthalmic imaging system 10 may also be communicatively coupled to a database 11. The database 11 may include medical records of patients imaged by the multimodal ophthalmic imaging system 10. The patient's medical records include the patient's medical history. The control module 12 may access the patient's medical records and retrieve the patient's medical history. Advantageously, the control module 12 may review the retrieved medical history for conditions that may manifest as ocular diseases, as described below. Figure 3 Further detailed discussion.

[0040] The multi-modality ophthalmic imaging system 10 is arranged to acquire a first ophthalmic image of a patient's eye 18 using a first imaging module operable in a first imaging modality. Figure 1 , the first ophthalmic imaging module may be an SLO imaging module 14 operating in a fundus imaging modality. The SLO imaging module 14 may acquire a fundus image of the patient's eye 18. The fundus image acquired by the SLO imaging module 14 is an example of a first ophthalmic image acquired by the first ophthalmic imaging system.

[0041] Now go to Figure 2 , a schematic fundus image 20 of a patient's eye 18 is shown. The fundus image 20 may be acquired by the SLO imaging module 14 of the multimodal ophthalmic imaging system 10. The fundus image 20 includes a retinal vasculature 22 that extends from an optic disc 24 through a patient's retina 21 toward the retinal periphery. The fundus image 20 includes an abnormality 26 located on the patient's retina 21, the abnormality 26 indicating a first stage of a multi-stage or progressive ocular disease. The abnormality 26 is a feature in the fundus image 20 that indicates a multi-stage or progressive ocular disease within the eye 18, such as diabetic retinopathy or a tumor. The abnormality 26 may be one or more of a lesion, a retinal break, a retinal detachment, a fluid accumulation, a tumor, a discoloration of the retina 21, a shadow on the retina 21, a hemorrhage in the retinal vasculature 22, a microaneurysm in the vasculature 22, or any other feature indicating a multi-stage ocular disease that manifests as an abnormality 26 on the surface or subsurface of the patient's retina 21.

[0042] The fundus image 20 is an example of a first ophthalmic image captured by a first imaging module, which in this example embodiment may be an SLO imaging module 14. Upon acquiring the fundus image 20, the SLO imaging module 14 may transmit the fundus image 20 to the control module 12. The control module 12 is configured to analyze the received fundus image 20 to detect the presence of one or more anomalies 26 indicative of a first stage of a multi-stage ocular disease within the fundus image 20. The control module 12 may detect anomalies 26 indicative of a multi-stage ocular disease within the ophthalmic image using image processing techniques or a machine learning algorithm that has been trained on ocular image data including images of healthy and diseased eyes to detect the presence of anomalies within the fundus image 20. Such techniques for searching for anomalies in ophthalmic images are further described in detail in WO 2020 / 127233A1, the contents of which are incorporated herein by reference in their entirety.

[0043] If the control module 12 detects the presence of one or more anomalies 26 indicative of a stage of a multi-stage or progressive ocular disease within the fundus image 20, the control module 12 is configured to determine a region of interest 28 based on the detection of the anomaly 26 in the fundus image 20, wherein the determined region of interest is predicted or expected to contain a second anomaly 27 indicative of a further stage or progression of the multi-stage ocular disease. The second anomaly 27 may be an anomaly that is not visible in the first ophthalmic image. For example, the second anomaly 27 may be a sub-surface feature of the first ophthalmic image 20 or a feature outside the field of view.

[0044] If the abnormality 26 indicates a retinal hemorrhage, then in the case of diabetic retinopathy, the abnormality 26 may indicate preproliferative retinopathy. In this case, the next stage of diabetic retinopathy is proliferative retinopathy, which may lead to retinal detachment. Thus, the region of interest 28 may be an area within the eye 18 susceptible to retinal detachment, such as a peripheral area of ​​the retina, and the second abnormality 27 may be predicted to be a retinal detachment.

[0045] The determined region of interest 28 is a region of the patient's eye 18 that is predicted to contain the second abnormality 27 and to be imaged by an imaging module operating in an imaging modality different from the imaging modality used to acquire the first ophthalmic image (in this example, the fundus image 20). The determined region of interest 28 may be a region surrounding the detected abnormality 26, and the second abnormality 27 may be predicted to be an enlargement of the detected abnormality 26, or, as Figure 2As shown, the region of interest 28 may be a region away from the detected abnormality 26, which is determined to be a region of interest in the patient's eye that is susceptible to the disease associated with the detected abnormality 26 and is predicted to contain the second abnormality 27. For example, if the abnormality 26 indicates a pathology that begins in the central region of the retina 21 and progresses to the periphery of the retina 21, the determined region of interest 28 may be located at the periphery of the retina 21. The size, location, and shape of the region of interest 28 may be determined by the control module 12 based on the characteristics of the detected abnormality 26.

[0046] The detected abnormality 26 may be an area of ​​swelling in a blood vessel on the patient's retina 21 that indicates an early stage of diabetic retinopathy. For example, the swelling may be a microaneurysm that indicates mild non-proliferative diabetic retinopathy. In the example where the abnormality 26 indicates an early stage of diabetic retinopathy, a region of interest 28 may be determined as a region surrounding the detected microaneurysm, and an additional ophthalmic image of the determined region of interest 28 may be acquired using an OCTA (OCT angiography) imaging modality to determine if a second abnormality 27 is associated with a retinal hemorrhage that indicates pre-proliferative retinopathy.

[0047] The control module 12 is configured to determine a second imaging modality for acquiring a second ophthalmic image of the determined region of interest 28 based on the detected abnormality 26 in the first ophthalmic image and / or based on the predicted second abnormality 27. The second imaging modality for imaging the determined region of interest 28 may be determined based on a type of pathology that is expected to be present in the determined region of interest 28 as the predicted second abnormality 27. The type of pathology that the second abnormality 27 is expected to be present in the determined region of interest 28 is a pathology associated with the progression of the multi-stage ophthalmic disease indicated by the detected abnormality 26.

[0048] The control module 12 may determine a multi-stage eye disease indicated by the abnormality 26 based on one or more characteristics of the abnormality 26 detected in the first ophthalmic image 20. The characteristics of the abnormality 26 may be any one or more of the following: size, shape, location, color, number of occurrences of the abnormality, or any other feature indicative of a multi-stage eye disease. Based on the characterization of the abnormality 26, the control module 12 is arranged to predict a location within the eye and also predict a type of pathology that may be present as a second abnormality 27 at the predicted location, and then determine a second imaging modality for acquiring a second ophthalmic image of the determined region of interest 28 based on the predicted second abnormality 27.

[0049] The control module 12 may use a machine learning algorithm or image processing technique to determine characteristics of the detected abnormality 26 to predict the type of multi-stage eye disease indicated by the detected abnormality 26. The size, shape, and location of the region of interest 28 are determined based on the type of multi-stage eye disease indicated by the detected abnormality 26 and / or based on the second abnormality 27 predicted to be present in the region of interest 28.

[0050] Based on the predicted second abnormality 27, the control module 12 determines a second imaging modality to be used to image the determined region of interest 28, and then selects a second imaging module that can be operated in the second imaging modality to acquire a second ophthalmic image. For example, the control module 12 can determine that an OCT image of the region of interest 28 should be acquired to obtain cross-sectional image data of the predicted second abnormality 27. In this case, the control module 12 can select the OCT imaging module 16 operating in the OCT imaging mode to acquire the second ophthalmic image. Alternatively, the control module 12 can determine that an autofluorescence modality is required to obtain a second ophthalmic image of the predicted second abnormality 27. In this case, the control module 12 can select the SLO imaging module 14 operating in the autofluorescence imaging modality to acquire the second ophthalmic image of the region of interest 28.

[0051] The control module 12 is configured to operate the selected second imaging module to acquire a second ophthalmic image of the determined region of interest 28. Imaging the determined region of interest 28 predicted to contain the second abnormality 27 associated with the progression of the multi-stage ophthalmic disease (synonymous with the detected abnormality 26) using the imaging modality selected based on the characteristics of the detected abnormality 26 and / or based on the predicted second abnormality 27 associated with the progression of the multi-stage ophthalmic disease in the first ophthalmic image allows high-quality images of the abnormality 26 and / or high-quality images of the second abnormality 27 associated with the progression of the multi-stage ophthalmic disease to be acquired with minimal input from an operator of the ophthalmic imaging system 10. This advantageously allows a relatively unskilled operator to use the multi-modality ophthalmic imaging system 10 to image a patient while still capturing high-quality ophthalmic images of the patient's eye 18 that can be transmitted to an ophthalmologist for diagnosis and evaluation of the multi-stage ophthalmic disease.

[0052] The control module 12 is also configured to determine acquisition characteristics or parameters of the second imaging module based on the characteristics of the detected abnormality 26 and / or based on the type of pathology predicted to be present in the determined region of interest 28 in the form of the second abnormality 27. For example, the control module 12 may determine acquisition parameters of the second imaging module, which may include determining one or more of the following: a scan field, an imaging depth, an imaging wavelength, a scan pattern, a scan speed, and a scan density for imaging the region of interest 28. The acquisition characteristics of the second imaging module may be selected based on the type of pathology associated with the predicted progression stage of the multi-stage ocular disease. For example, if the control module 12 predicts that the second abnormality 27 associated with the predicted progression stage of the multi-stage ocular disease is sub-retinal fluid, the acquisition characteristics of the OCT imaging module 16 (or the second imaging module) may be set to acquire a large field of view, large depth OCT image of the region of interest 28 so as to capture the entire area predicted to contain sub-retinal fluid. In addition, the position and shape of the determined region of interest 28 may affect the acquisition characteristics of the second imaging module. For example, the scanning pattern, scanning field, imaging depth, and imaging wavelength may be determined based on the determined position and shape of the region of interest 28 .

[0053] The ophthalmic imaging system 10 may include a display (not shown) arranged to display images acquired by the ophthalmic imaging system 10 to a user of the ophthalmic imaging system 10. The control module 12 may be coupled to the display such that the display may show the first ophthalmic image (fundus image 20) to the user of the ophthalmic imaging system 10. In addition, the display may overlay the determined region of interest 28 on the fundus image 20 such that the user of the ophthalmic imaging system 10 may view the position of the determined region of interest 28 relative to the patient's retina 21 on the first ophthalmic image. In an example embodiment, the clinician may reposition the determined region of interest 28 and / or adjust the size of the determined region of interest 28.

[0054] Figure 1 The ophthalmic imaging system 10 is communicatively coupled to a database 11. Figure 1 As shown, the control module 12 may be coupled to a database 11. The database 11 may form part of the ophthalmic imaging system 10, or the database may be a remote database 11. For example, the database 11 may be stored on the cloud and accessed by the control module 12 wirelessly.

[0055] The database 11 is a database containing medical information related to the patient imaged by the ophthalmic imaging system 10. For example, the database 11 may include digital medical records related to the patient's medical history. The control module 12 is configured to access the database 11 to retrieve the medical records of the patient imaged by the ophthalmic imaging system 10. For example, the control module 12 can access the database 11 to retrieve the patient's medical records to view records of medical conditions that may manifest an eye disease, particularly but not limited to a multi-stage eye disease. For example, the control module 12 can review the patient's medical history to check for previous instances of eye disease or conditions such as diabetes or the patient's age that may put the patient at risk for a multi-stage eye disease such as diabetic retinopathy or age-related macular degeneration.

[0056] Upon receiving a first ophthalmic image (e.g., fundus image 20), the control module 12 is arranged to analyze the first ophthalmic image for abnormalities 26 within the image. The control module 12 may modify the analysis of the first ophthalmic image based on the patient's medical records. For example, if the patient is diabetic, the control module 12 may search the first ophthalmic image for abnormalities 26 indicative of diabetic retinopathy, such as microaneurysms, retinal hemorrhages, retinal detachments, and blocked or leaking blood vessels in the macula.

[0057] Furthermore, if the control module 12 detects the presence of an abnormality 26 indicative of an eye disease, the control module 12 may determine a region of interest 28 predicted to contain a second abnormality 27 and / or select a second imaging modality for imaging the region of interest 28 based on a combination of the patient's medical history and the characterization of the abnormality 26. For example, if the patient's medical history shows that the patient suffers from diabetes and an abnormality 26 indicative of neovascularisation is detected, the control module 12 may determine the region of interest 28 and select a second imaging modality to investigate the progression of diabetic retinopathy. In this example, the control module 12 may determine that an OCT angiography (OCTA) imaging modality (a second imaging modality) should be used to assess the severity of neovascularisation within the retina, and the control module 12 may then select an OCT imaging module 16 operating in an OCTA modality as the second imaging module to acquire a second ophthalmic image. If the patient's medical records are available, the control module 12 may determine the second imaging modality based on a combination of one or more of the following: the characteristics of the detected abnormality 26, the type of pathology predicted to be present in the determined region of interest, and the patient's medical history.

[0058] Figure 3 is a flow chart showing a method of imaging a patient's eye 18. Figure 1 The method is performed using a multi-modality ophthalmic imaging system 10.

[0059] In step 301, the multimodal ophthalmic imaging system 10 is operated in a first imaging modality to acquire a first ophthalmic image of a patient's eye 18. Acquiring the first ophthalmic image of the patient's eye 18 may include operating a first imaging module operable in the first imaging modality to acquire the first image of the patient's eye 18. For example, the SLO imaging module 14 of the ophthalmic imaging system 10 may be operated in a fundus imaging modality to acquire a fundus image 20 of a patient's retina 21.

[0060] Next, in step 302, the first ophthalmic image of the patient's eye 18 acquired in step 301 is analyzed or searched for the presence of an abnormality indicative of a multi-stage or progressive ocular disease. Step 302 includes detecting an abnormality 26 in the first ophthalmic image captured by the first ophthalmic imaging module, wherein the detected abnormality 26 is an abnormality indicative of a multi-stage ocular disease. Detecting the presence of an abnormality 26 indicative of an ocular disease in the first ophthalmic image may include using image processing techniques or machine learning algorithms to detect the presence of the abnormality 26 in the first ophthalmic image. For example, an artificial intelligence ophthalmic image analysis model may be used to detect the presence of the abnormality 26 in the first ophthalmic image. Detecting the abnormality 26 may include detecting one or more of: discoloration on the retina, lesions, high or low rates of the vasculature, a hole in the retina, or any other abnormality indicative of an ocular disease.

[0061] Detecting the abnormality 26 in the first ophthalmic image may also include determining the stage of a multi-stage eye disease indicated by the abnormality 26. For example, if the abnormality 26 is a microaneurysm on the vasculature of the patient's retina, the method may further include determining that the abnormality 26 indicates background retinopathy, which is a first stage of diabetic retinopathy.

[0062] Upon detecting the abnormality 26 in the first ophthalmic image in step 302, the method includes determining a region of interest 28 within the patient's eye 18 that is predicted to contain a second abnormality 27 in step 303. Determining the region of interest 28 may include determining a region of interest 28 within the patient's eye 18 to be imaged using a second imaging modality based on the detection of the abnormality 26 in the first ophthalmic image. Determining the region of interest 28 may include predicting a location or region within the patient's eye 18 that is susceptible to a stage of progression of a multi-stage ocular disease, and predicting that the second abnormality 27 will manifest within the location or region. The determined region of interest 28 may be a region within the patient's eye 18 that is susceptible to a second stage or stage of progression of a multi-stage ocular disease indicated by the detected abnormality 26, and that is predicted or expected to contain the second abnormality 27.

[0063] In the above example where the abnormality 26 indicates background retinopathy in a patient with diabetic retinopathy (stage 1 in diabetic retinopathy), the region of interest 28 may be the area surrounding the detected abnormality 26. The region of interest 28 in this example may be an area of ​​the retina 21 susceptible to hemorrhage caused by proliferative pre-retinopathy (stage 2 of diabetic retinopathy), and the predicted second abnormality 27 may be a hemorrhage associated with proliferative pre-retinopathy.

[0064] However, in another example embodiment, the region of interest 28 may be a location away from the abnormality 26 detected in the first ophthalmic image. For example, if the abnormality 26 detected in the first ophthalmic image indicates extensive retinal hemorrhage, which would indicate preproliferative retinopathy (stage 2 of diabetic retinopathy), the region of interest 28 may be an area of ​​the periphery of the patient's retina 21, and the predicted second abnormality 27 may be a retinal detachment. This would allow images of the peripheral area of ​​the patient's retina 21 to be captured, allowing the clinician to obtain images of areas of the patient's retina that are at risk for retinal detachment. Retinal detachment indicates proliferative retinopathy (stage 3 of diabetic retinopathy).

[0065] The size and location of the region of interest 28 may be determined based on the characteristics of the detected abnormality and / or based on the predicted second abnormality 27. For example, if the abnormality 26 indicates a microaneurysm, the region of interest 28 may be determined as the area surrounding the abnormality 26, and the predicted second abnormality 27 may be an enlarged hemorrhage area. The determined region of interest 28 may be positioned so that an OCT angiography image with an enlarged area around the microaneurysm may be acquired to assess the severity of the retinal hemorrhage. However, if the abnormality 26 indicates extensive retinal hemorrhage, the region of interest 28 may be a peripheral area of ​​the retina 21, and the predicted second abnormality 27 may be a retinal detachment.

[0066] In another example, if the abnormality 26 indicates subretinal fluid, the region of interest 28 may be a larger area including, for example, an area between two and ten times the area of ​​the detected abnormality 26. Determining the region of interest 28 may include determining the size and shape of the region of interest 28 based on characteristics of the detected abnormality 26 and / or based on a predicted second abnormality associated with a stage of progression of a multi-stage ocular disease associated with the detected pathology.

[0067] In step 304, a second imaging modality for imaging the determined region of interest 28 is determined based on the type of pathology or second abnormality 27 predicted to be present in the predicted progression stage or second stage of the multi-stage ocular disease indicated by the detected abnormality 26. Determining the second imaging modality may include determining which of the plurality of imaging modalities will produce the highest quality ophthalmic image of the predicted second abnormality 27, such that the second ophthalmic image may be used to diagnose the pathology predicted to be present in the determined region of interest 28. This may be performed using, for example, a machine learning algorithm trained to select an imaging modality based on characteristics of the abnormality detected in the first ophthalmic image or according to a lookup table that allows selection of the second imaging modality based on the predicted progression stage of the multi-stage ocular disease.

[0068] Returning to the above example, if the detected abnormality 26 indicates a microaneurysm that is synonymous with background retinopathy, the predicted progression stage of the multi-stage ocular disease is preproliferative retinopathy, and the predicted second abnormality 27 is retinal hemorrhage. Therefore, the second imaging modality can be selected based on the imaging modality most suitable for evaluating the severity of preproliferative retinopathy. For example, in this case, the OCT angiography imaging modality can be selected to visualize the severity of the hemorrhage in the retina 21. However, if the abnormality indicates subretinal fluid, the second imaging modality can be selected based on the imaging modality most suitable for evaluating the severity and size of the subretinal fluid to determine the degree of progression of the spread of the subretinal fluid. For example, the OCT imaging modality can be selected to obtain a cross-sectional image of the patient's retina so that the progression or spread of the subretinal fluid can be observed in the OCT image.

[0069] Step 304 may also include selecting image acquisition characteristics or control parameters of the second imaging modality based on the characteristics of the detected abnormality and / or based on a predicted pathology type associated with the progression of the abnormality detected in the first ophthalmic image. For example, the scanning field, imaging depth, imaging wavelength, scanning pattern, and scanning speed may be changed based on the characteristics of the detected abnormality and / or based on a predicted pathology type associated with the progression of the abnormality detected in the first ophthalmic image to maximize the quality of the ophthalmic image captured by the second imaging modality.

[0070] Steps 303 and 304 may also include receiving a medical record indicating a medical history of the patient. Determining the region of interest in step 303 may be based on a combination of the detected abnormality and the patient's medical history to predict a second abnormality 27 associated with a progression or second stage of the multi-stage eye disease. Additionally, selecting a second imaging modality for imaging the determined region of interest 28 in step 304 may be performed based on a combination of the detected abnormality 26 and the patient's medical history.

[0071] In step 305, a second ophthalmic image of the determined region of interest 28 is acquired using a second imaging module operating under the selected second imaging modality (selected in step 304). The first imaging module and the second imaging module may be the same imaging module operating under different imaging modalities. The method may include displaying, storing and / or transmitting the first ophthalmic image and / or the second ophthalmic image.

[0072] The method may also include step 306 of determining whether a predicted second abnormality 27 located in the determined region of interest 28 is captured within the second ophthalmic image. This is particularly beneficial when imaging abnormalities indicative of subretinal disease. For example, if the abnormality indicates subretinal fluid, it may not be possible to assess the extent to which the subretinal fluid has spread before capturing and evaluating the second ophthalmic image.

[0073] Determining whether the second anomaly 27 has been captured within the region of interest 28 in step 306 may include determining whether there is a discontinuity at a border of the second ophthalmic image. If a discontinuity is detected at a border or edge of the second ophthalmic image frame, it may be inferred that the region of interest 28 determined in step 303 is not large enough to surround the second anomaly 27. For example, if the detected anomaly indicates subretinal fluid, the second imaging modality selected for imaging the region of interest 28 may be an OCT imaging modality. When acquiring the second ophthalmic image (in this example, the OCT image), the method may include determining whether there is a structural discontinuity at a frame border of the second ophthalmic image, indicating that the region of interest 28, and therefore the OCT image field, should be larger in a given dimension to fully capture the second anomaly 27. In this case, step 303 may be repeated to resize the region of interest 28 based on the detection of a discontinuity at a border of the second ophthalmic image, and the second ophthalmic image may be recaptured. This process may be repeated until no discontinuity is detected at a border of the second ophthalmic image, indicating that the second anomaly 28 has been fully captured within the second ophthalmic image.

[0074] Figure 4 is a schematic diagram of a multimodal ophthalmic imaging system 40 according to a second exemplary embodiment of the present invention. The multimodal ophthalmic imaging system 40 includes a first imaging module 42, a second imaging module 44, a third imaging module 46, an optical system 15, and a control module 12. Figure 1 The same reference numerals are used to mark the parts and elements shown in the first exemplary embodiment as those in the Figure 1 are the same and will not be described again.

[0075] The multimodal ophthalmic imaging system 40 includes three imaging modules 42, 44, 46, each of which can operate in one or more different imaging modalities. Each imaging module 42, 44, 46 is optically coupled to the optical system 15 and includes a light source for emitting a light beam to the optical system for scanning across a target imaging area on the patient's eye 18. In addition, each imaging module may include one or more photodetectors for detecting light reflected back from the patient's eye 18 via the optical system 15. Each imaging module may also include a signal processing device that is arranged to generate an ophthalmic image based on the output of the photodetector. Although the multimodal ophthalmic imaging system 40 includes three imaging modules 42, 44, 46, those skilled in the art will understand that this is merely exemplary and that any number of imaging modules 42, 44, 46 may be implemented in the multimodal ophthalmic imaging system 40.

[0076] As mentioned above Figures 1 to 3 As described above, the first imaging module 42 is operable to acquire a first ophthalmic image. The first imaging module 42 may be a SLO imaging module (e.g. Figure 1 ), and can be operated in a fundus imaging modality to obtain fundus images of a patient's eye.

[0077] Figure 5 An example first ophthalmic image 50 captured by the first imaging module 42 is shown. Figure 5 As shown, the first ophthalmic image 50 includes an abnormality 52 located on a branch of the retinal vasculature 22 . Figure 5 The abnormality 52 shown in FIG. 5 may be a microaneurysm or a bulge leaking a small amount of blood. Figure 5 The abnormality 52 shown in the first ophthalmic image 50 may be an example of an abnormality 52 indicative of a multi-stage eye disease such as diabetic retinopathy. As described above, the control module 12 is arranged to detect the presence of the abnormality 52 in the first ophthalmic image 50 and determine the stage of the multi-stage eye disease indicated by the abnormality 52. ​​In the example where the abnormality 52 is a microaneurysm, the control module 12 may determine that the abnormality 52 indicates background retinopathy associated with a first stage of diabetic retinopathy.

[0078] Upon detecting the abnormality 52 in the first ophthalmic image 50, the control module 12 is arranged to determine a region of interest 28 predicted to contain a second abnormality 57 based on the detection of the abnormality 52 in the first ophthalmic image 50. In this example of an abnormality 52 indicating a first stage of diabetic retinopathy, the control module 12 may determine that a region of interest associated with pathologies indicative of preproliferative retinopathy and proliferative retinopathy should be imaged to determine the extent of progression of the diabetic retinopathy in the patient. Preproliferative retinopathy and proliferative retinopathy are typically the second and third stages of progression, respectively, for patients with diabetic retinopathy. In addition, the control module 12 may receive a medical record of the patient from the database 11 indicating that the patient suffers from diabetes, thereby confirming that the abnormality 52 detected in the first ophthalmic image is likely to indicate diabetic retinopathy.

[0079] like Figure 5 As shown, the control module 12 determines a first region of interest 53 surrounding anomaly 52 that should be imaged using the second imaging modality and a second region of interest 55 that is distal to the anomaly 52 and is predicted to contain a second anomaly 57 that should be imaged by the third imaging modality.

[0080] The first region of interest 53 is an area surrounding the abnormality 52. ​​The first region of interest 53 is an area within the patient's eye 18 that is susceptible to the progression of a multi-stage eye disease, in this example, diabetic retinopathy. The abnormality 52 in this example is a bulge or microaneurysm that the control module 12 has determined to indicate background retinopathy. In order to assess whether the diabetic retinopathy has progressed to the second stage of diabetic retinopathy, i.e., preproliferative retinopathy, the control module 12 has determined an area around the abnormality 52 to assess the extent of retinal hemorrhage present, as this is typically the second stage in patients with diabetic retinopathy. The control module 12 is arranged to determine a second imaging modality that should be selected to image the region of interest 53. In this example, the control module 12 may select an OCT angiography imaging modality to image the region of interest 53, thereby determining the extent of retinal hemorrhage, which indicates the second stage of diabetic retinopathy, i.e., preproliferative retinopathy.

[0081] The control module 12 may also identify a second region of interest 55 that is associated with yet another progressive stage of the multi-stage ocular disease and is predicted to contain a second abnormality 57. The second region of interest 55 may be an area within the patient's eye 18 that is distant from the abnormality 52 detected by the control module 12 in the first ophthalmic image. For example, where the control module 12 determines that the detected abnormality 52 indicates a first stage of diabetic retinopathy, the second region of interest 55 may be an area susceptible to retinal detachment, and the predicted second abnormality 57 may be a retinal detachment. Retinal detachment may be present in patients with proliferative retinopathy, which is a third stage of diabetic retinopathy. The control module 12 may determine that a wide field of view or ultra-wide field of view infrared imaging modality should be selected to acquire a reflected image of the second region of interest 55 located in the periphery of the patient's eye 18 to help the clinician accurately diagnose and assess the progression of retinal detachment by highlighting subretinal features that may not have been visible in the first ophthalmic image.

[0082] Figure 6 is a schematic diagram of a programmable signal processing device 600 configured to control imaging modules within a multi-modality ophthalmic imaging system 10. The programmable signal processing device 600 can perform at least a portion of the functions of the control module 12, and in an example embodiment herein, at least a portion of the device 600 is included in the control module 12. The programmable signal processing device 600 includes a communication interface (I / F) 610 for receiving ophthalmic images acquired by the imaging module and / or for outputting a determined imaging modality to operate the selected imaging module under the determined imaging modality. The communication interface (I / F) 610 can also receive medical records from the database 11. In an example embodiment herein, the communication interface (I / F) 610 can input / output any information obtained as part of the method described herein.

[0083] The signal processing device 600 also includes a processor (e.g., a central processing unit CPU and / or a graphics processing unit GPU) 620, a working memory 630 (e.g., a random access memory), and an instruction storage device 640 storing a computer program 645, the computer program 645 including computer readable instructions that, when executed by the processor 620, cause the processor 620 to perform various functions, including Figure 1 and Figure 4 Those functions of the control module 12 in the apparatus 600 and / or the functions of the method described herein. In an example embodiment herein, only the processor 620 is included in the control module 12, but in other examples, one or more additional components of the apparatus 600 are also included in the control module 12.

[0084] The working memory 630 stores information used by the processor 620 during the execution of the computer program 645. The instruction storage device 640 includes, for example, a ROM (e.g., in the form of an electrically erasable programmable read-only memory (EEPROM) or flash memory) preloaded with computer-readable instructions. Alternatively, the instruction storage device 640 includes a RAM or similar type of memory, and the computer-readable instructions of the computer program 645 can be input to the instruction storage device 640 from a computer program product (e.g., a non-transitory computer-readable storage medium 650 in the form of a CD-ROM, DVDROM, etc.) or a computer-readable signal 660 carrying the computer-readable instructions. In any case, the computer program 645, when executed by the processor 620, causes the processor 620 to perform the methods described herein, including but not limited to the method of imaging a patient's eye as described above.

[0085] In an example embodiment herein, the control module 12 of the above example embodiment includes a computer processor 620 and an instruction storage device 640 storing computer-readable instructions, which, when executed by the computer processor 620, causes the computer processor 620 to perform the methods described herein, including but not limited to the method of imaging a patient's eye using the multi-modal ophthalmic imaging system 10 as described herein.

[0086] In the foregoing description, example aspects have been described with reference to several example embodiments. Therefore, the description should be regarded as illustrative rather than restrictive. Similarly, the drawings shown in the accompanying drawings highlighting the features and advantages of the example embodiments are presented for illustrative purposes only. The architecture of the example embodiments is sufficiently flexible and configurable that it can be utilized in ways other than those shown in the accompanying drawings.

[0087] In an example embodiment, the software embodiments of the examples presented herein may be provided as computer programs or software, such as one or more programs having instructions or instruction sequences included or stored in an article (e.g., a machine-accessible or machine-readable medium, an instruction storage device, or a computer-readable storage device, each of which may be non-transitory). The program or instructions on a non-transitory machine-accessible medium, a machine-readable medium, an instruction storage device, or a computer-readable storage device may be used to program a computer system or other electronic device. Machine-readable or computer-readable media, instruction storage devices, and storage devices may include, but are not limited to, floppy disks, optical disks, and magneto-optical disks or other types of media / machine-readable media / instruction storage devices / storage devices suitable for storing or transmitting electronic instructions. The techniques described herein are not limited to any particular software configuration. They may be applied in any computing or processing environment. The terms "computer-readable", "machine-accessible medium", "machine-readable medium", "instruction storage device", and "computer-readable storage device" used herein shall include any medium capable of storing, encoding, or transmitting instructions or instruction sequences for execution by a machine, a computer, or a computer processor and causing the machine / computer / computer processor to perform any of the methods described herein. Furthermore, it is common in the art to refer to software in one form or another (e.g., program, procedure, process, application, module, unit, logic, etc.) as taking an action or causing a result. Such expressions are merely a shorthand way of stating that execution of the software by a processing system causes the processor to perform an action to produce a result.

[0088] Some embodiments may also be implemented by the preparation of application specific integrated circuits, field programmable gate arrays, or by interconnecting an appropriate network of conventional component circuits.

[0089] Some embodiments include computer program products. A computer program product may be one or more storage media, instruction storage devices, or storage devices on or in which instructions that can be used to control or cause a computer or computer processor to perform any process of the exemplary embodiments described herein are stored. Storage media / instruction storage devices / storage devices may include, by way of example and without limitation, optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memories, flash memory cards, magnetic cards, optical cards, nanosystems, molecular memory integrated circuits, RAIDs, remote data storage / archiving / warehousing devices, and / or any other type of device suitable for storing instructions and / or data.

[0090] Some embodiments stored on any one of one or more computer-readable media, instruction storage devices, or storage devices include hardware for controlling the system and software for enabling the system or microprocessor to interact with a human user or other mechanism using the results of the exemplary embodiments described herein. Such software may include, without limitation, device drivers, operating systems, and user applications. Ultimately, as described above, such computer-readable media or storage devices also include software for executing exemplary aspects of the present invention.

[0091] Software modules for implementing the processes described herein are included in the programming and / or software of the system. In some example embodiments herein, the modules include software, but in other example embodiments herein, the modules include hardware or a combination of hardware and software.

[0092] Although various exemplary embodiments of the present invention have been described above, it should be understood that they are presented by way of example and not limitation. It will be apparent to those skilled in the relevant art that various changes may be made in form and detail. Therefore, the present invention should not be limited to any of the above exemplary embodiments, but should only be defined in accordance with the appended claims and their equivalents.

[0093] It should also be understood that any process recited in the claims does not need to be performed in the order presented.

[0094] Although this specification contains many specific embodiment details, these should not be understood as limitations on the scope of any invention or content that may be claimed, but should be understood as descriptions of features specific to the specific embodiments described herein. Certain features described in this specification in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, the various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable sub-combination. In addition, although features may be described above as acting in a specific combination and even initially claimed as such, one or more features from the claimed combination may be deleted from the combination in some cases, and the claimed combination may be directed to a sub-combination or a variant of the sub-combination.

[0095] In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of the various components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

Claims

1. A multimodal ophthalmic imaging system (10, 40) for acquiring an ophthalmic image of an eye (18) of a patient, the multimodal ophthalmic imaging system (10, 40) comprising: a first imaging module (14, 42) operable in a first imaging modality for acquiring a first ophthalmic image (20, 50) of the patient's eye (18); one or more additional imaging modules (16, 44, 46), each of the one or more additional imaging modules (16, 44, 46) being capable of operating in a respective imaging modality, wherein the respective imaging modality of the one or more additional imaging modules (16, 44, 46) is different from the first imaging modality; and A control module (12) arranged to: detecting an abnormality (26, 52) in the first ophthalmic image (20, 50) acquired by the first imaging module (14, 42); determining a region of interest (28, 53, 55) in the patient's eye based on the abnormality detected in the first ophthalmic image, wherein the determined region of interest (28, 53, 55) is a region within the patient's eye (18) predicted to contain a second abnormality (27, 57); determining a second imaging modality for imaging the determined region of interest (28, 53, 55) based on the second anomaly (27, 57), the second imaging modality being different from the first imaging modality; selecting a second imaging module (16, 44, 46) from the one or more additional imaging modules (16, 44, 46) that is operable in the determined second imaging modality; and The selected second imaging module (16, 44, 46) is controlled to acquire a second ophthalmic image of the determined region of interest (28, 53, 55).

2. The ophthalmic imaging system of claim 1, wherein: The abnormality (26, 52) detected in the first ophthalmic image is indicative of a first stage of a multi-stage eye disease, and wherein the second abnormality (27, 57) is indicative of a second stage of the multi-stage eye disease.

3. An ophthalmic imaging system according to claim 1 or claim 2, wherein: The first imaging module (14, 42) comprises a scanning laser ophthalmoscope (SLO), and the first ophthalmic image is a fundus image of the patient's eye (18).

4. An ophthalmic imaging system according to any one of the preceding claims, wherein: The control module (12) is further arranged to determine one or more image acquisition parameters of the second imaging module (16, 44) based on the second abnormality (27, 57) for controlling the second imaging module (16, 44) to acquire the second ophthalmic image of the determined region of interest (28, 53).

5. An ophthalmic imaging system according to any one of the preceding claims, wherein: The control module (12) is communicatively coupled to a database (11), the database (11) comprising medical records indicating a medical history of the patient, and wherein the control module (12) is arranged to retrieve the medical records of the patient from the database (11) and determine the region of interest (28, 53) based on both the detected abnormality (26) and the medical history of the patient.

6. An ophthalmic imaging system according to any one of the preceding claims, wherein: The control module (12) is configured to detect a discontinuity at a border of the second ophthalmic image, wherein the discontinuity at the border indicates a clipping of the second anomaly (27, 57) located within the second ophthalmic image.

7. The ophthalmic imaging system of claim 6, wherein: In response to detecting the discontinuity at the boundary of the second ophthalmic image, the control module (12) is arranged to: increasing the size of the determined region of interest to generate an enlarged region of interest; and The second imaging module is controlled to acquire an ophthalmic image of the enlarged region of interest.

8. An ophthalmic imaging system according to any one of the preceding claims, wherein: The control module is arranged to: determining a second region of interest based on the detected anomaly, wherein the second region of interest is a region containing the detected anomaly; determining a third imaging modality for imaging the determined second region of interest based on the detected abnormality, the third imaging modality being different from the first imaging modality; selecting a third imaging module operable in the determined third imaging modality from the one or more additional imaging modules; and The selected third imaging module is controlled to acquire a third ophthalmic image of the determined second region of interest.

9. A method of imaging an eye (18) of a patient, the method comprising: acquiring a first ophthalmic image (20, 50) of the patient's eye (18) using a first imaging module (14, 42) operating in a first imaging modality; detecting an abnormality (26, 52) in the first ophthalmic image (20, 50) acquired by the first imaging module (14, 42); determining a region of interest (28) in the patient's eye (18) based on the abnormality (26) detected in the first ophthalmic image, wherein the determined region of interest (28) is a region within the patient's eye (18) predicted to contain a second abnormality (27, 57); determining a second imaging modality for imaging the determined region of interest (28) based on the second anomaly (27, 57), the second imaging modality being different from the first imaging modality; and A second ophthalmic image of the determined region of interest (28) is acquired using a second imaging module (16, 44) operating in the determined second imaging modality.

10. The method according to claim 9, wherein: The abnormality (26) detected in the first ophthalmic image is indicative of a first stage of a multi-stage eye disease, and wherein the second abnormality (27) is indicative of a second stage of the multi-stage eye disease.

11. The method according to claim 9 or claim 10, further comprising determining whether the second anomaly is captured within the second ophthalmic image by detecting a discontinuity at a boundary of the second ophthalmic image, wherein: Detecting the presence of a discontinuity at the boundary indicates cropping of the second abnormality within the second ophthalmic image.

12. The method according to any one of claims 9 to 11, comprising: In response to detecting the presence of the discontinuity at the boundary of the second ophthalmic image, increasing the size of the determined region of interest to generate an enlarged region of interest, and acquiring a further ophthalmic image of the enlarged region of interest.

13. The method according to any one of claims 9 to 12, comprising receiving a medical record of the patient comprising a medical history of the patient, wherein: The region of interest is determined based on a combination of abnormalities detected in the first ophthalmic image and the received medical history of the patient. 14 . The method according to claim 9 , further comprising determining one or more image acquisition parameters of the second imaging module based on the second abnormality, and operating the second imaging module with the determined image acquisition parameters to acquire the second ophthalmic image.

15. The method according to any one of claims 9 to 14, further comprising: determining a second region of interest based on the detected anomaly, wherein the second region of interest is a region containing the detected anomaly; determining a third imaging modality for imaging the determined second region of interest based on the detected abnormality, the third imaging modality being different from the first imaging modality; selecting a third imaging module from the one or more additional imaging modules that is operable in the determined third imaging modality; and The selected third imaging module is controlled to acquire a third ophthalmic image of the determined second region of interest.

Images