Method and apparatus for stereoscopic color eye imaging

Abstract

An ophthalmic imaging device, comprising: illumination optics having an illumination light source that outputs illumination light; a scanner that redirects the illumination light toward a portion of an object to be imaged; and an optical image capture device including a camera that receives backscattered light scattered by the object from the illumination light and captures a first image and a second image of the backscattered light. The device further comprises: a control processor that controls the scanner and the optical image capture device such that the optical image capture device captures the first image and the second image of the object. The first image and the second image are captured by the camera at different times and are extracted from different portions of the backscattered light. The device further comprises: an image processor that generates a stereoscopic image from the first image and the second image of the object.

Description

[0001] Cross - Reference to Related Applications

[0002] This application claims priority to U.S. Provisional Application No. 62 / 898,812, filed on September 11, 2019, the entire content of which is incorporated herein by reference. Technical Field

[0003] The present disclosure generally relates to methods for ophthalmic imaging (e.g., stereoscopic color eye imaging), devices for ophthalmic imaging, and computer-readable media for ophthalmic imaging. Background Art

[0004] The "Background" description provided herein is for the purpose of generally presenting the background of the present disclosure. Aspects of the work of the currently named inventors and aspects that may not constitute prior art as described in this Background section, as of the time of filing the application, are not admitted to be prior art to the present invention, either expressly or implicitly.

[0005] Stereo disk photography has been used to record structural abnormalities and longitudinal changes in ophthalmology. For example, stereo color eye photographs have been used for optic nerve head (ONH) imaging for analysis and diagnosis. In stereo color eye photography, two images are created photographically, and when viewed, these two images may fuse in the observer's brain to give an impression of depth. That is, when the observer views the two images, the observer's left eye views the left image, the observer's right eye views the right image, and the observer's brain reconstructs the depth relationship (or parallax) that existed during photography.

[0006] Imaging techniques that provide retinal topology information include traditional stereo-based topography, confocal scanning laser ophthalmoscopy (cSLO), and optical coherence tomography. For example, stereo-based topography uses a pair of images obtained from a traditional stereo imager to computationally reconstruct the depth of the retina. According to this traditional technique, the 3D depth profile can only be reconstructed by the observer's brain. In other words, it can only be qualitatively evaluated. cSLO is an imaging technique that uses a laser to raster scan the retina instead of the bright white flashlamp used in a standard eye or fundus camera, which provides better comfort for the patient. The reflected light is then captured through a small aperture (confocal pinhole) that blocks scattered light, which provides a clear, better-quality, and high-contrast image. Successive cSLO scans captured at increasing depths can be combined to create a three-dimensional topographical image of the retina or optic disc. The image stacks are aligned to create a final composite image to provide retinal thickness measurements of the macula or optic nerve head. Unfortunately, due to the relatively long imaging duration, eye movement may degrade the quality of the cSLO image. This will result in an increase in measurement error in the 3D depth profile. Optical coherence tomography (OCT) is an imaging technique that uses low-coherence light to capture two-dimensional and three-dimensional images with micron resolution from within an optically scattering medium (e.g., biological tissue). OCT is based on low-coherence interferometry, typically using near-infrared light. From the OCT A-line profile (the backscattered light signal at each depth), depth information (or topological analysis) can be computationally extracted by segmenting the retinal layers (multiple A-lines along the scan axis) for each OCT B-frame. This method has been widely used clinically.

[0007] Digital light ophthalmoscopy (DLO) uses a confocal imaging technique such as that described in U.S. Patent No. 8,237,835 in 2012. A digital light projector (DLP) based on a digital micromirror device (DMD) is configured to rapidly project a series of adjacent illumination lines over the field of view. Only the backscattered light from the target is de-scanned (i.e., multi-scattering imaging) to improve DLO contrast, and is directly imaged onto a monochromatic complementary metal oxide semiconductor (CMOS) sensor using a rolling shutter detection method. In this configuration, the rolling shutter serves as a flexible electronic aperture that can be fine-tuned in position and width in real time by software.

[0008] For example, the DMD described in U.S. Patent No. 5,061,049 is a micro-optical-electro-mechanical system (MOEMS). The DMD chip can have hundreds of thousands of micromirrors arranged in a rectangular array on its surface, and the micromirrors correspond to the pixels in the image to be displayed (or projected). The micromirrors can be individually rotated by ±10 - 12° to reach the on or off state. In the on state, the light from the projector bulb is reflected into the lens, making the pixel appear bright on the screen. In the off state, the light is directed elsewhere (usually to a heat sink or a black body), making the pixel appear dark. The activation of each micromirror on the DMD can be controlled in real time to project an optical scanning beam onto a given sample and collect the backscattered light from the same imaging sample.

[0009] [Citation List]

[0010] [Patent Document]

[0011] PTL 1: U.S. Patent No. 5,061,049

[0012] PTL 2: U.S. Patent No. 8,237,835, 2012

[0013] PTL 3: U.S. Patent No. 6,758,564B2

[0014] [Non-Patent Document]

[0015] NPL 1: Marshall E. Tyler, CRA, FOPS. Stereoscopic fundus photography: principles and techniques. Journal of Ophthalmic Photography, 1996; 18(2): 68 - 81

[0016] NPL 2: Muller MS, Elsner AE. Confocal retinal imaging using a digital light projector with a near-infrared VCSEL source. Proc SPIE Int Soc Opt Eng. 2018; 10546: 105460G. doi: 10.1117 / 12.2290286

[0017] NPL 3: Elsner, AE., Petrig, BL. Laser scanning digital camera with simplified optics and potential for multiple scattered light imaging. U.S. Patent No. 7,831,106, 2007.

[0018] NPL 4: Espina MP, Arcinue CA, Ma F, Camacho N, Bartsch DU, Freeman WR. Analysis of a confocal scanning laser ophthalmoscope noncontact ultra-widefield lens system in retinal and choroidal diseases. Retina. 2015;35(12):2664-2668. Doi: 10.1097 / IAE.0000000000000899.

[0019] NPL 5: Huang, D; Swanson, EA; Lin, CP; Schuman, JS; Stinson, WG; Chang, W; Hee, MR; Flotte, T; et al. (1991). "Optical coherence tomography", Science. 254(5035):1178-81.

[0020] NPL 6: Hansung Kim, Shinwoo Choi, Kwanghoon Sohn, "Real-time depth reconstruction of stereo sequences", Proc. SPIE 6016, Three-Dimensional TV, Video, and Display IV, 60160E (November 15, 2005).

[0021] NPL 7: J. Gu, Y. Hitomi, T. Mitsunaga and S.K. Nayar, Encoded rolling shutter photography: Flexible spatio-temporal sampling. International Conference on Computational Photography (ICCP), March 2010.

[0022] NPL 8: Kafieh R, Rabbani H, Kermani S. A review of algorithms for optical coherence tomography image segmentation from the retina. J Med Signals Sens. 2013;3(1):45-60.

[0023] NPL 9: Chen TC. Spectral-domain optical coherence tomography of glaucoma: Qualitative and quantitative analysis of the optic nerve head and retinal nerve fiber layer (AOS thesis). Trans Am Ophthalmol Soc. 2009;107:254-281. SUMMARY OF THE INVENTION

[0024] The present Summary of the Invention is provided to introduce in a simplified form a selection of concepts that will be further described in the Detailed Description below. The Summary of the Invention is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Additionally, the claimed subject matter is not limited to limitations that solve any or all disadvantages noted in any part of this disclosure.

[0025] According to one aspect of the present disclosure, an ophthalmic imaging device includes: illumination optics including an illumination light source configured to output illumination light; a scanner configured to receive the illumination light from the illumination optics and redirect the illumination light toward a portion of an object to be imaged; an optical image capture device including a camera configured to receive backscattered light scattered from the illumination light by the object and capture a first image and a second image of the backscattered light; a control processor configured to control the scanner and the optical image capture device such that the optical image capture device captures the first image and the second image of the object, the first image and the second image being captured by the camera at different times and extracted from different portions of the backscattered light; and an image processor configured to generate a stereoscopic image from the first image and the second image of the object. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] When considered in conjunction with the accompanying drawings, a more complete appreciation of the invention and many of the attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description, in which:

[0027] Figure 1 : Figure 1 Schematically shows a block diagram of a stereoscopic color eye imaging device according to an embodiment of the present invention.

[0028] Figure 2A : Figure 2A Shows an example of a change in acquisition angle using an up-and-down view acquisition mechanism according to an embodiment of the present invention.

[0029] Figure 2B : Figure 2B Shows an example of a change in acquisition angle using a left-and-right view acquisition mechanism according to an embodiment of the present invention.

[0030] Figure 2C : Figure 2C Depicts an example of the operation of an illumination and detection mechanism according to an embodiment of the present invention.

[0031] Figure 2D : Figure 2D Shows an example of optical tracking according to an embodiment of the present invention to further show how the field of view angle and the stereoscopic viewing angle are differently related.

[0032] Figure 2E : Figure 2E Shows an example of the operation of an embodiment of the present invention having a dynamic aperture in the pupil plane.

[0033] Figure 2F : Figure 2F Shows an example of the operation of an embodiment of the present invention having a dynamic aperture change (split aperture case).

[0034] Figure 2G : Figure 2G Shows an example of using a dynamic aperture in an embodiment of the present invention.

[0035] Figure 2H : Figure 2H Shows an example of using an aperture stop setting in an embodiment of the present invention.

[0036] Figure 2I : Figure 2I Shows an example of using a split aperture in an embodiment of the present invention.

[0037] Figure 2J : Figure 2J Shows an example of a narrow viewing angle in a split aperture setting according to an embodiment of the present invention.

[0038] Figure 2K : Figure 2K Shows an example of a wide viewing angle in a split aperture setting according to an embodiment of the present invention.

[0039] Figure 3A : Figure 3A Shows an example of a viewing angle change with a narrow viewing angle in an aperture stop setting according to an embodiment of the present invention.

[0040] Figure 3B : Figure 3B Shows an example of a viewing angle change with a wide viewing angle according to an embodiment of the present invention.

[0041] Figure 4A : Figure 4A Shows an example of an optical field of view (FOV) change with a wide FOV in an aperture stop setting according to an embodiment of the present invention.

[0042] Figure 4B : Figure 4B Shows an example of an FOV change with a narrow FOV in an aperture stop setting according to an embodiment of the present invention.

[0043] Figure 4C : Figure 4C Shows an example of an increase in the imaging field of view by acquiring a series of images and mosaicking them.

[0044] Figure 4D : Figure 4D Shows an example of a change in the scanning direction.

[0045] Figure 5 : Figure 5 Shows an example of a stereoscopic color eye imaging device including a 2D camera with a rolling shutter mechanism plus a galvanometer scanner according to an embodiment of the present invention.

[0046] Figure 6A : Figure 6A Shows an example of a galvanometer and detector upper view control signal according to an embodiment of the present invention.

[0047] Figure 6B : Figure 6B Shows an example of a galvanometer and detector lower view control signal according to an embodiment of the present invention.

[0048] Figure 6C : Figure 6C Shows an example of the operation of an embodiment of the present invention to further show how the change in the galvanometer scanner control voltage affects the scanning angle (or illumination angle).

[0049] Figure 7A : Figure 7A Shows an example of an upper view control signal during a narrow-angle dynamic view change on an imaging device based on a galvanometer scanner according to an embodiment of the present invention.

[0050] Figure 7B : Figure 7B Shows an example of a lower view control signal during a narrow-angle dynamic view change on an imaging device based on a galvanometer scanner according to an embodiment of the present invention.

[0051] Figure 7C : Figure 7C Shows an example of an upper view control signal during a wide-angle dynamic view change on an imaging device based on a galvanometer scanner according to an embodiment of the present invention.

[0052] Figure 7D : Figure 7D Shows an example of a lower view control signal during a wide-angle dynamic view change on an imaging device based on a galvanometer scanner according to an embodiment of the present invention.

[0053] Figure 8 : Figure 8 Shows an example of a stereoscopic color eye imaging device including a 2D camera with a rolling shutter mechanism plus a DMD according to an embodiment of the present invention.

[0054] Figure 9A : Figure 9A Shows an example of a DMD and detector upper view control signal.

[0055] Figure 9B : Figure 9B Shows an example of a DMD and detector lower view control signal.

[0056] Figure 10A : Figure 10AShows an example of an upper view control signal during a narrow - view dynamic view change in DMD - based imaging according to an embodiment of the present invention.

[0057] Figure 10B : Figure 10B Shows an example of a lower view control signal during a narrow - view dynamic view change in DMD - based imaging according to an embodiment of the present invention.

[0058] Figure 10C : Figure 10C Shows an example of an upper view control signal during a wide - view dynamic view change in DMD - based imaging according to an embodiment of the present invention.

[0059] Figure 10D : Figure 10D Shows an example of a lower view control signal during a wide - view dynamic view change in DMD - based imaging according to an embodiment of the present invention.

[0060] Figure 11A : Figure 11A Shows an example of a stereoscopic color eye imaging device including a multi - scan camera plus a galvanometer scanner according to an embodiment of the present invention.

[0061] Figure 11B : Figure 11B Shows Figure 11A an example of the operation in the aperture stop setting of the embodiment in

[0062] Figure 12A : Figure 12A Shows an exemplary method of stereoscopic imaging using parallel stereo acquisition according to an embodiment of the present invention.

[0063] Figure 12B : Figure 12B Shows an exemplary method of stereoscopic imaging using convergent stereo acquisition according to an embodiment of the present invention.

[0064] Figure 12C : Figure 12C Shows an example of the operation of stereoscopic imaging according to an embodiment of the present invention.

[0065] Figure 13 : Figure 13 Shows an example of a method of stereoscopic imaging with 3D visualization and analysis processing performed by a processing unit according to an embodiment of the present invention.

[0066] Figure 14A : Figure 14A Shows an example of a stereoscopic color eye image according to an embodiment of the present invention.

[0067] Figure 14B :Figure 14B Shows a three-dimensional drawing of information extracted from a stereoscopic color eye image according to an embodiment of the present invention. Figure 14A from

[0068] Figure 15A : Figure 15A Shows a three-dimensional drawing including information on cup depth estimation performed according to an embodiment of the present invention.

[0069] Figure 15B : Figure 15B Shows a three-dimensional drawing including information on cup-to-disc ratio estimation performed according to an embodiment of the present invention.

[0070] Figure 16 : Figure 16 Shows an example of a computer architecture according to an embodiment of the present invention that can be used to implement parts of the present invention.

[0071] From the detailed description provided below, further applicable fields of the present disclosure will become apparent. It should be understood that the detailed description of the exemplary embodiments is for illustrative purposes only and thus is not necessarily intended to limit the scope of the present disclosure. Detailed Description

[0072] As used herein, an element or step recited in the singular and preceded by the word "a" or "an" should be understood to not exclude a plurality of elements or steps, unless such exclusion is explicitly recited. Additionally, a reference to "an embodiment" of the present invention is not intended to be construed as excluding the existence of additional embodiments that also incorporate the recited features.

[0073] The control methods and systems described herein can be implemented using computer programming or engineering techniques including computer software, firmware, hardware, or any combination or subset thereof.

[0074] Age-related macular degeneration (AMD) and glaucoma are the main causes of ophthalmic blindness. Many eye examinations such as clinical slit-lamp examinations and imaging have been developed to evaluate these types of diseases. Recently, OCT devices have been used to image and visualize these types of eye diseases in near real-time. Unfortunately, OCT devices are relatively expensive and require trained operators or technicians to operate the OCT devices. Therefore, examinations based on OCT devices are costly, which may partly explain why examinations based on OCT devices are not widely used in less developed countries or rural areas.

[0075] The present disclosure relates to a stereoscopic color eye imaging device, method, and computer-readable medium, which include reconstructing a 3D depth profile from the obtained stereoscopic color eye image and can overcome existing limitations. Any desired part of the eye, including the fundus of the eye, can be imaged. The present invention is also applicable to imaging other parts of the human anatomical structure. The terms "stereoscopic color eye imaging device" and "stereoscopic color fundus imaging device" used to describe embodiments of the present invention should be considered equivalent. The stereoscopic color fundus imaging device according to an embodiment of the present invention may include a light source unit, a sensor / detector unit, a scanner unit, imaging optics, and a control unit, and the light source unit, detector / sensor unit, scanner unit, imaging optics, and control unit operate to provide stereoscopic images and information useful for screening eye diseases in ophthalmology. Additionally, the stereoscopic color fundus imaging device and method described herein can provide images, optical resolutions, and features equivalent to those generated by conventional DLO, cSLO, or line-scan SLO (e.g., as described in US 6,758,564 B2), while avoiding or minimizing the disadvantages of those prior art methods.

[0076] Compared with OCT-based devices, embodiments of the present invention can result in a relatively low-cost system. Such embodiments can be partially or fully automated (e.g., as easy to operate as taking a selfie), or may require only minimal imaging training. People living in remote areas and less developed countries will be able to fully utilize the present invention.

[0077] Embodiments of the stereoscopic color fundus imaging device and method according to the present invention can stereoscopically analyze the retina and can provide imaging features equivalent to those of a conventional color fundus imager at a lower cost and with easier operation. According to an embodiment of the present invention, multiple scattering signals (backscattered light in the upper and lower directions) can be used to reconstruct a 3D depth profile (i.e., perform stereoscopic imaging) by computationally and dynamically calculating the disparity between a first image and a second image.

[0078] Figure 1 An exemplary schematic diagram of a stereoscopic color fundus imaging device that can obtain a stereoscopic color image of the fundus of the eye 116 is shown. The device in this embodiment includes imaging optics 102, an optical scanner unit 110, a control unit 104, a detector / sensor unit 112, a processing unit 106, a display unit 108, and a storage unit 114. Additionally, the device according to an alternative embodiment can capture a grayscale stereoscopic image of the fundus of the eye 116 instead of a color stereoscopic image of the fundus of the eye 116.

[0079] The imaging optical device 102 may include optical components such as a light source, optical lenses, a holder and an optical alignment tool, and an optical shutter. One or more halogen lamps and / or LEDs may be used as the light source. The optical scanner unit 110 may redirect the optical illumination light from the imaging optical device to a sample or object to be imaged (e.g., an eye) and at the same time receive the backscattered light scattered from the sample or object to be imaged. The operation of all electrical and mechanical components and units in the system may be controlled by a control unit 104 implemented by a general-purpose computer programmed for the following functions. To achieve a stereoscopic view, at least two or more images are acquired, with a sufficient angular difference in perspective between the two or more images. According to an embodiment of the present invention, the image acquisition angle may be arbitrarily set, but its scanning direction may advantageously be perpendicular to the detector closing direction, for example, to optimally utilize the rolling shutter camera mechanism according to an example described below regarding Figure 2A , Figure 2B , Figure 2C and Figure 2D . In addition, a fixed target from the control unit may be provided to assist in optical focusing / aligning and suppress eye movement.

[0080] The relative distance between the optical scanning position and the detection position may be dynamically changed to introduce and / or change the stereoscopic angular difference discussed below regarding Figure 3A and Figure 3B . To complete stereoscopic imaging, at least two images should be acquired at different perspectives (e.g., scan #1 208a and scan #2 208b in Figure 2A ; or scan #1 208c and scan #2 208d in Figure 2B ; or scan #1 308a and scan #2 308b in Figure 3A ; or scan #1 308c and scan #2 308d in Figure 3B ).

[0081] According to an embodiment of the present invention, stereoscopic imaging may be performed using: 1) split aperture operation and 2) aperture stop operation. According to an embodiment of the present invention, the optical aperture in the imaging optical device may be dynamically changed between the split aperture mode and the aperture stop mode. An example of such a dynamic change between split aperture and aperture stop operations during the imaging of the eye 276 is shown in Figure 2E . In the example of Figure 2E , illumination is performed through the central part of the eye and detection is annular. In the aperture stop mode, illumination is in the annular part of the eye and detection is through the central part of the eye. In Figure 2EIn the example, in the case of a split aperture with a horizontal aperture, the illumination 270 is ~φ2.0 mm, d 272 is ~0.5 mm, and the detection width is 274. For the case of a split aperture, illumination of the eye 276 using a vertical aperture is also shown. Figure 2E The case of an aperture stop with a detection 280 of ~φ2.0 mm, d 278 of ~0.5 mm, and an illumination width 282 of ~0.5 mm is also shown. For the case of a vertical aperture / aperture stop, illumination of the eye 276 is also shown. Thus, the parameters dynamically controlled by the control unit 104 include the aperture, the optical magnification, and the viewing angle.

[0082] Figure 2F More details regarding the operation during the split aperture mode are shown. According to an embodiment of the present invention, in the case of a split aperture, using a horizontal aperture (i.e., at the Figure 2F upper part), aperture switching can be performed to introduce a viewing angle by switching from a top view scan of the eye 284 (i.e., at the Figure 2F upper left corner of the Figure 2F to generate a first optical image 286) to a bottom view scan of the eye 284 (i.e., at the Figure 2F upper right corner of the Figure 2F to generate a second optical image 288). Alternatively, when using a vertical aperture, a viewing angle difference between a first optical image 290 and a second optical image 292 is created by switching between a left view scan (i.e., at the Figure 2A lower left corner of the Figure 2B and a right view scan (i.e., at the Figure 3A lower right corner of the Figure 3B In the split aperture mode, the illumination and detection positions can be the same. By switching from the upper aperture to the lower aperture in the pupil plane or from the left aperture to the right aperture in the pupil plane, a viewing angle difference can be introduced. Therefore, in split aperture mode stereoscopic imaging, no time delay between illumination and detection is required (different from the aperture stop mode), because the viewing angle difference is introduced by switching the aperture shape (top view aperture ←→ bottom view aperture). When the aperture setting is at the aperture stop, by changing the optical scan position (e.g., the illumination / scanning positions 204a and 202b, 204c and 202d in Figure 2A and Figure 2B and the detection positions 202a, 204b, 202c, and 204d in Figure 3A and Figure 3BThe relative distances between the detection positions 302a, 304b, 302c, and 304d) in [description] can increase or decrease the viewing angle difference between the scanned images. The optical scanning position is the position on the fundus of the eye onto which the optical scanning beam (i.e., the illumination light projected by the device onto the fundus of the eye) is projected, and the detection position is the corresponding position on the imaging plane (e.g., the camera sensor surface) onto which the optical backscattered light is projected. When the optical magnification factor is equal to 1x, the physical ratio between the imaging plane and the fundus plane should be the same. According to an embodiment of the present invention, the change in the viewing angle difference is achieved by changing the relative time interval (i.e., the leading or lagging time) between the optical scanner control signal for selecting the line of the fundus of the eye to capture the image and the image acquisition control signal (acquired line by line).

[0083] When the optical scanning beam is projected onto the fundus of the eye through a galvanometer scanner (e.g., galvanometer scanner 510), a galvanometer control signal is generated from the control unit 104 to control the galvanometer scanner. The galvanometer control signal is controlled by the control unit 104 based on parameters including optical magnification, camera frame rate, desired depth resolution, and FOV. These parameters can be predetermined and stored in a memory (e.g., storage unit 114), can be dynamically set by a user interface on the control unit 104 or the processing unit 106, or can be controlled by a remote operator using another computer connected to the control unit 104 or the processing unit 106. The galvanometer control signal is specifically designed to control the projection angle of the optical scanning beam relative to the imaging sample. Generally, the amount of change in the projection angle is linearly proportional to the analog voltage of the galvanometer control signal (excluding the DC voltage offset in the signal). For example, when wide FOV imaging is required, the voltage of the galvanometer control signal should be increased proportionally compared to narrow FOV imaging. In parallel, the backscattered light from the imaging sample will be reflected from the mirror surface of the galvanometer scanner and then digitized by a detector (i.e., a camera sensor). This digitization is performed using a 2D camera with a rolling shutter mechanism, such that each row of the detector array on the 2D camera will be sequentially triggered by a camera trigger control signal generated by the control unit 104 to digitize the incident light signal received through the corresponding pixel line on the camera sensor. In traditional SLO imaging, the detection position should be equal to the scanning beam position to achieve the optimal signal-to-noise ratio. However, in stereoscopic imaging according to an embodiment of the present invention, for a stereoscopic view, the detection position should be off-center relative to the scanning beam position. The eccentricity on the image plane (or camera) can be represented in terms of a time scale because the control unit 104 controls the line-by-line acquisition of the 2D camera and the incremental change in the voltage of the galvanometer control signal at a fixed rate (frequency). Therefore, the perspective in the dynamic stereoscopic image generated by the processing unit 106 can be simply controlled by the control unit 104 that introduces a relative phase shift between the 2D camera trigger control signal and the galvanometer control signal. The resulting stereoscopic perspective between two consecutive images (stereoscopic images) will determine the minimum resolvable depth resolution obtained by analyzing the images. Therefore, the control unit 104 can synchronize the operations of the optical scanner unit 110 and the detector / sensor unit 112 to achieve an ideal depth resolution. A polygon scanner can also be used instead of a galvanometer scanner.

[0084] The optical magnification of the stereoscopic image generated by an embodiment of the present invention can affect not only the optical field of view but also the resolvable stereoscopic depth resolution. The larger the optical magnification, the larger the resolvable stereoscopic depth resolution.

[0085] The detector / sensor unit 112 includes a digital converter / sensor for converting backscattered light into an electrical signal. The digital converter may include a charge-coupled device (CCD) camera with a rolling shutter, a multi-line scanning camera, or any other readily available two-dimensional image sensing device. The control unit 104 controls all system components including optical magnification control, execution of user control commands, auto-alignment / focusing, galvanometer / DMD control and safety mechanisms, real-time optical power / spectrum monitoring, and real-time eye tracking. The processing unit 106 includes processing circuitry configured by software, firmware, and / or hardware to compute, process, and extract stereoscopic information from the captured images. The processing unit 106 processes user applications such as image reconstruction analysis, cloud-based services, patient information, and clinical records. One or both of the processing unit 106 and the control unit 104 further include a user interface that allows a user to control parameters and operations of the device and / or method. The user interface is configured to provide instructions and / or control feedback (including visual and / or auditory cues) for instructing the user on the proper use of the device and / or method. One or both of the processing unit 106 and the control unit 104 include a communication interface that allows input of images from other imaging sources and allows electronic output of the processing results (e.g., sharing with a patient or healthcare professional), and allows portions or all of the control and processing functions described herein to be performed by a remotely networked and / or cloud-based computer. The display unit 108 includes a projector, monitor, display, or printer that can be used by a user to visualize the information generated by the processing unit 106. The storage unit 114 is a digital storage memory configured to store information such as captured images, 3D reconstruction data, clinical records, operating programs for the processing unit 106, control unit 104, and configuration data for the device or method.

[0086] Figure 2AShows an example of the acquisition angle change using the up / down view acquisition mechanism in the aperture stop mode according to an embodiment of the present invention. In this example, "up" and "down" refer to different directions of the viewing angle change. However, this is merely an exemplary pair of directions, and other directions such as 10 o'clock and 4 o'clock, or 11 o'clock and 5 o'clock, 7 o'clock and 1 o'clock, etc. are also within the scope of the present invention, so that the first image and the second image are received from the object at different scattering angles, where the scattering angles are spaced 180 degrees apart from each other. A sensor (such as a CCD image sensor) is configured to capture a detection line 202a (i.e., a pixel line from the sensor) on the fundus of the eye 208a focused by the sensor, while an illumination line 204a illuminates another part of the fundus of the eye 208a, and at the same time the detection line 202a and the illumination line 204a move along the fundus of the eye 208a in the scanning direction 206a (by the scanner described below) to capture the first (up view) color fundus image 210a. The position of the illumination line 204a (i.e., the part illuminated by the device) is separated from the position of the detection line 202a (i.e., the position of the area captured by the device in the image) by a distance that is predetermined or can be controlled by the control unit based on the processing discussed in more detail below. The down view is captured by the detection line 204b and the illumination line 202b, and the detection line 204b and the illumination line 202b are controlled to move across the fundus of the eye 208b (the same eye as in 208a) to capture the second (down view) color fundus image 210b.

[0087] Figure 2B Shows an example of the acquisition angle change using the left / right view acquisition mechanism in the aperture stop mode according to an embodiment of the present invention. This example operates similar to the example in Figure 2A ; however, in Figure 2B , the viewing angle difference is captured in the left / right direction opposite to the up / down direction in Figure 2A . Thus, according to the example in Figure 2B , the left view is performed by the detection line 202c and the illumination line 204c, and the detection line 202c and the illumination line 204c move across the fundus of the eye 208c in the scanning direction 206c to generate the first (left view) color fundus image 210c. According to this example, the right view is performed by the detection line 204d and the illumination line 202d, and the detection line 204d and the illumination line 202d move across the fundus of the eye 208d in the scanning direction 206d to capture the second (right view) color fundus image 210d.

[0088] Figure 2C Depicts an example of the operation of the illumination and detection mechanism according to an embodiment of the present invention. As in Figure 2CAs shown, the illumination light 236 illuminates the fundus 220 through the center of the pupil plane. The backscattered light 230 from the illuminated eye 220 propagates at a backscattering angle 238 and is received by the focusing lens 232 and the beam splitter 234 (which is included in the imaging optics 102). The galvanometer scanner 224 redirects the backscattered light 230 to the rolling shutter camera 222. A dynamic aperture assembly (e.g., an aperture wheel that can be rotated to select different fixed-size apertures or a controllable aperture with a variable aperture opening size) can be used to switch between different apertures such as the split-aperture mode and the aperture diaphragm mode. A portion of the backscattered light 230 scattered in the first direction 226 is used to capture a first color fundus image from a first viewing angle (e.g., the upper viewing angle for capturing the color fundus image 210a in Figure 2A or the left viewing angle for capturing the color fundus image 210c in Figure 2B ), and another portion of the backscattered light 230 scattered in the second direction 228 (where the first direction 226 and the second direction 228 are separated by 180 degrees) is used to capture a second image from a second viewing angle (e.g., the lower viewing angle for capturing the color fundus image 210b in Figure 2A or the right viewing angle for capturing the color fundus image 210d in Figure 2B ). Thus, the backscattered light 230 in this example is digitized by the detector array to generate a pair of images with separated viewing angles (i.e., upper / lower images or left / right images). The same sensor can capture the first image and the second image at different timings (e.g., before or after the timing of capturing the first color fundus image) using, for example, the illumination line 202b and the detection line 204b, and the illumination line 202b and the detection line 204b move in the scanning direction 206b to capture the second color fundus image 210b of the fundus of the eye 208b.

[0089] Figure 2D An example of optical tracking according to an embodiment of the present invention is shown to further illustrate how the FOV and the stereoscopic viewing angle are related. In Figure 2DIn the example, the illumination light 236 illuminates the eye 220, and the eye 220 scatters the backscattered light 230 at a backscattering angle 238 corresponding to the angular difference in perspective between the first and second (e.g., upper and lower) captured images. The backscattered light 230 is received by the focusing lens 232, the scanner DMD 240, the dynamic aperture in the aperture stop mode, and the rolling shutter camera 244 to generate an upper view from the backscattered portion 226 and a lower view from the backscattered portion 228. When the scan in the scan direction 262 passes through the fundus of the eye 248 to generate an upper view scan (i.e., the first captured image), the upper view portion 226 corresponds to the detection position 258 and the associated illumination line 260. When the detection position 266 and the associated illumination line 264 move through the fundus of the eye 250 in the scan direction 268 to capture a second image, the lower view portion 228 corresponds to the detection position 266 and the associated illumination line 264. The distance between the detection line 258 and the detection line 266 corresponds to the baseline distance and corresponds to a time delay (e.g., the time delay 608a described below). The area above the captured fundus of the eye 250 corresponds to the horizontal FOV 252 and the vertical FOV 256. The first (upper) scan and the second (lower) scan can be captured at an interval of 40 ms as indicated by the elapsed time 246. The narrow / wide view angle discussed further in the examples below is related to the field of view (FOV) of imaging. The FOV is the maximum imaging window size, while the view angle refers to the stereo view angle. Thus, in Figure 3A and Figure 3B the example, the stereo imaging pair (i.e., the first captured image and the second captured image) is captured while the view angle changes from a narrow angle to a wide angle. The wide / narrow view angle changes under the control of the control unit 104 while using actuators / motors included in the imaging optics 102 or the optical scanner 110 or the detector / sensor unit 112 to change the wide / narrow view angle.

[0090] Figure 2G shows an example of an embodiment of the present invention including many elements common to the embodiment shown in Figure 2C but adding a dynamic aperture assembly 294 (e.g., an aperture wheel or a controllable aperture) configured to operate according to a dynamic aperture switching mode.

[0091] Figure 2H and Figure 2I shows an example of the change in the view angle between the captured images in the aperture stop mode ( Figure 2H ) and the captured images in the split aperture mode ( Figure 2I ). In these examples, the eye 2102 is illuminated by the illumination light 2110, and the backscattered light 2104 propagating at the view angle 2106 is received by the focusing lens 2108 and the galvanometer / DMD 2118. The backscattered light is further received by the aperture stop 2112 (inFigure 2H in the example of) or the split aperture 2114 (in Figure 2I the example of), the optical amplifier 2116, and the rolling shutter camera 2120 having the detector surface 2126 receive. In Figure 2H the example of, the aperture stop 2112 is used in the aperture stop mode. As shown in the superimposed intensity plot 2128, the point spread function (PSF) at the detection surface 2126 of the camera 2120 includes the top view 2132 and the bottom view 2130. In Figure 2I the example of, the split aperture 2114 is used in the split aperture mode. As shown in the superimposed intensity plot 2138, the PSF at the detection surface 2126 of the camera 2120 includes the top / bottom views 2134 / 2136. In the split aperture mode, the illumination and detection positions should be the same. As Figure 2F shown, the perspective difference in the split aperture mode is obtained by switching from the upper aperture to the lower aperture or from the left aperture to the right aperture at the pupil plane.

[0092] Figure 2J shows an example of narrow field of view capture in the split aperture setting mode. In this example, a horizontal aperture is used. A top view scan is performed on the eye 2204 to capture the first optical image 2206, and after the aperture switch, a bottom view scan is performed on the eye 2204 to obtain the second optical image 2208. The offset d 2202 is ~0.2 mm.

[0093] Figure 2K shows an example of wide field of view capture in the split aperture setting mode. In this example, a horizontal aperture is used. A top view scan is performed on the eye 2204 to capture the first optical image 2210, and after the aperture switch, a bottom view scan is performed on the eye 2204 to obtain the second optical image 2212. The offset d 2214 is ~0.6 mm. Alternatively, in Figure 2J and Figure 2K the example of a vertical aperture can be used.

[0094] Figure 3AAn example of the perspective change with a narrow viewing angle in the aperture stop setting according to an embodiment of the present invention is shown. In this example, the camera sensor is configured to capture an upper view with a narrow field of view using the detection line 302a and the illumination line 304a. The detection line 302a and the illumination line 304a are controlled to move along the fundus of the eye 308a in the scanning direction 306a to capture the first (upper view / narrow field of view) color fundus image 310a. At different times before or after the above capture, the same sensor uses the detection line 304b and the illumination line 302b to capture a lower view with a narrow FOV. The detection line 304b and the illumination line 302b are controlled to move across the fundus of the eye 308b to capture the second (lower view / narrow field of view) color fundus image 310b. To reconstruct three dimensions, at least two images with different perspectives are required in stereoscopic imaging. The imaging field of view, which is usually determined by the user, needs to be the same between these images.

[0095] Figure 3B An example of the view change with a wide field of view in the aperture stop setting according to an embodiment of the present invention is shown. In this example, the camera sensor is configured to capture an upper view with a wide field of view using the detection line 302c and the illumination line 304c. The detection line 302c and the illumination line 304c are controlled to move along the fundus of the eye 308c in the scanning direction 306c to capture the first (upper view / wide field of view) color fundus image 310c. The same sensor uses the detection line 304d and the illumination line 302d to capture a lower view of the same eye with a wide field of view. The detection line 304d and the illumination line 302d are controlled to move along the fundus of the eye 308d in the scanning direction 306d to capture the second (lower view / wide field of view) color fundus image 310d. Capturing the first and second images from different perspectives with a constant narrow or wide field of view provides the ability to view different regions of the fundus of the eye in three dimensions and with a controlled resolution and / or to depict larger or smaller regions of interest and provides different illumination angles.

[0096] Figure 4A An example of the change in the optical field of view (FOV) with a wide FOV in the aperture stop mode according to an embodiment of the present invention is shown. Figure 4B An example of the FOV change with a narrow FOV according to an embodiment of the present invention is shown. Except Figure 3A and Figure 3B showing the perspective change from the first pair of captured images to the second pair of captured images, Figure 4A and Figure 4B the examples are similar to the examples in Figure 3A and Figure 3B respectively. When the aperture is set to the split aperture mode, except for the differences in the acquisition mechanism as described in Figure 2F above, the FOV change is the same. Figure 4A andFigure 4B shows the change in the field of view angle and the change in the scanning direction from the first pair of captured images to the second pair of captured images. Thus, in Figure 4A and Figure 4B 's example, the stereoscopic imaging pairs are captured with a wide field of view and a narrow field of view. The wide / narrow FOV changes under the control of the control unit 104 while using an actuator / motor included in one or more of the imaging optics 102, the optical scanner 110, and the detector / sensor unit 112 to change the wide / narrow FOV. Thus, according to Figure 4A 's example, the first scan of the upper view with a wide FOV angle is captured using the detection line 402a and the illumination line 404a. The detection line 402a and the illumination line 404a are controlled to move across the fundus of the eye 408a in the scanning direction 406a to capture the first (upper view / wide FOV angle) color fundus image 410a. The same sensor can use the detection line 404b and the illumination line 402b to capture a scan of the lower view with a wide FOV angle. The detection line 404b and the illumination line 402b are controlled to move across the fundus of the eye 408b in the scanning direction 406b to capture the second (lower view / wide FOV angle) color fundus image 410b.

[0097] According to Figure 4B 's example, the first scan of the left view with a narrow FOV angle is captured using the detection line 402c and the illumination line 404c. The detection line 402c and the illumination line 404c are controlled to move across the fundus of the eye 408c in the scanning direction 406c to capture the first (left view / narrow FOV angle) color fundus image 410c. The same sensor can use the detection line 404d and the illumination line 402d to capture a scan of the right view with a narrow FOV angle. The detection line 404d and the illumination line 402d are controlled to move across the fundus of the eye 408d in the scanning direction 406d to capture the second (right view / narrow FOV angle) color fundus image 410d.

[0098] According to an embodiment of the present invention, the time between the first image and the second image can advantageously be less than 100 ms and preferably 40 ms, thereby reducing the possibility of eye movement between the first image and the second image, and as a result improving the accuracy of the three-dimensional image created based on the first image and the second image. As the acquisition speed of the detector increases, the time between the first image and the second image may decrease.

[0099] As Figure 4CAs shown in the example of , the imaging field of view (or region of interest) of each image can also be offset at the center position to obtain a series of images 412, 414, 416, and 418, mosaic them, and combine the individual images to form a mosaic image with a larger field of view than any single image, thereby increasing the imaging field of view. The dynamic change of the region of interest can be carried out through the user interface or the control unit.

[0100] In addition, as Figure 4D shown in the example of , the scanning direction can be changed according to user preferences. In Figure 4D the example of , the fundus 428 can be imaged in the scanning direction 426 using the illumination line 422 and the detection line 424 to capture the first scan as shown in the upper left corner of Figure 4D (e.g., a lower view color fundus image). The fundus 428 can be imaged in the scanning direction 434 (which is the same direction as the scanning direction 426) using the illumination line 432 and the detection line 430 to capture the second scan as shown in the lower left corner of Figure 4D (e.g., an upper view color fundus image). Alternatively, the scanning direction can be reversed, and the fundus 428 can be imaged in the scanning direction 440 (which is the direction opposite to the scanning directions 426 and 434) using the illumination line 438 and the detection line 436 to capture the first scan as shown in the upper right corner of Figure 4D (e.g., an upper view color fundus image). The fundus 428 can be imaged in the scanning direction 446 (which is the same direction as the scanning direction 440) using the illumination line 442 and the detection line 444 to capture the second scan as shown in the lower right corner of Figure 4D (e.g., a lower view color fundus image). In the split-aperture mode, for example, as Figure 2E , Figure 2F , Figure 2H and Figure 2I shown, the scanning direction can be changed by switching the aperture shape (upper view aperture ←→ lower view aperture).

[0101] Figure 5Shows an exemplary schematic diagram of a stereoscopic color fundus imaging device according to an embodiment, the stereoscopic color fundus imaging device including a detector / sensor unit implemented as a 2D camera with a rolling shutter mechanism. This embodiment includes an imaging assembly 502, a detector / sensor unit 504, projection optics 508, a scanner 510, and illumination optics 512. The imaging assembly 502, which is part of the imaging optics 102, includes a lens (e.g., a relay lens) 502a, a dynamic aperture mirror 502b, and an objective lens 502c. The detector / sensor unit 504 includes a 2D camera with a rolling shutter mechanism 504a. The projection optics 508, which is an additional part of the imaging optics 102, includes a lens 508c, a black dot mask 508a, and a mirror 508b. The illumination optics 512 includes a lens 512a (e.g., a relay lens), a slit 512b, an iris aperture 512c, a cylindrical lens 512d, and an illumination light source 512e. Also included in this embodiment are a control unit 104, a processing unit 106, a display unit 108, and a storage unit 114. The illumination light source 512e may include an LED, a halogen lamp, or other known light sources.

[0102] According to this embodiment, the illumination optics 512 outputs the illumination light generated by the illumination light source 512e. The scanner 510 receives the illumination light from the illumination optics 512, redirects the optical axis of the illumination light, and outputs the redirected illumination light. The projection optics 508 includes a mirror 508b that can be implemented using a digital light projector (DLP), and the projection optics 508 is configured to receive the redirected illumination light from the scanner 510 and output the projected light. The scanner can be implemented using a galvanometer mechanical scanner. A galvanometer scanner is a galvanometer that indicates that it has sensed a current by deflecting a light beam using a mirror. Generally, a galvanometer scanner is used to project a scanning beam (i.e., an illumination line) onto a given sample / eye region. The imaging optics 502 includes a dynamic aperture mirror 502b that receives and redirects the projected light passing through the objective lens 502e towards the eye 506. The backward-scattered and reflected light from the eye 506 is received by the imaging optics 502 and provided to the detector / sensor unit 504, where the light is captured by the 2D camera with the rolling shutter mechanism 504a.

[0103] The control unit 104 controls the operation and timing of each of the various parameters in the device, the various parameters including aperture size / pattern, exposure timing, sensor gain / sensitivity, focus of the imaging optics, focus of the illumination optics, focus of the projection optics, direction and timing of the scanner, capture timing of the camera / sensor, orientation of the DLP / mirror 508b, intensity and timing of the illumination light, and the optical magnification of each element in the device. The processing unit 106 receives the images captured by the 2D camera having the rolling shutter mechanism 504a and performs the processing discussed further below to generate a stereoscopic color fundus image.

[0104] To obtain a stereoscopic image, according to this embodiment, an optical scan with a controlled and variable viewing angle is achieved using a mechanical galvanometer scanner. Figure 5 The embodiment in is configured to operate according to Figure 2A , Figure 2B , Figure 3A , Figure 3B , Figure 4A and Figure 4C each example shown in. The operation is controlled according to a predetermined command stored in the storage unit 114, according to a command provided by a user interface on the control unit 104 or the processing unit 106, or according to a command received by the control unit 104 or the processing unit 106 from a remote location.

[0105] Figure 6AAn example of a galvanometer scanner and detector upper view control signal according to an embodiment of the present invention is shown. In this example, the galvanometer scanner control signal 604a corresponding to the voltage applied to the galvanometer scanner mirror is plotted as the applied scanner voltage 602a (e.g., 0 - 5 volts) on the y-axis with respect to time 606a on the x-axis (e.g., 0 - 50 ms). The voltage of the galvanometer scanner control signal determines the position of the scanned beam (i.e., the illumination line) on the fundus of the eye. When the dynamic aperture setting is the split aperture mode, the constant time delay (608a, 608c, 638, 708a, 708b, 708c, and 708d) between illumination and detection is zero because a perspective difference is introduced and controlled by switching the upper and lower view apertures. Therefore, the stereoscopic image acquisition speed is relatively faster than that of the aperture diaphragm-based stereoscopic imaging mode where the time delay between illumination and detection is greater than zero. When the dynamic aperture setting is in the aperture diaphragm mode, a perspective must be introduced with respect to time. For example, arrow 608a shows a constant time delay (corresponding to the perspective). The corresponding detector / sensor control signal for turning on / off sensor capture is also plotted with the same time scale on the x-axis 606a and the voltage of the detector / sensor control signal 602b on the y-axis. The "turn-on" timing of each of the detector / sensor control signals 610b, 612b, and 614b is delayed by the time delay 608a and corresponds to the capture of lines of the image by consecutive detector rows (i.e., row 1 captured by the detector / sensor control signal 610b, row 2 captured by the detector / sensor control signal 612b, and row 3 captured by the detector / sensor control signal 614b). As shown in this example, consecutive rows are captured while the voltage of the galvanometer scanner control signal increases the voltage 602a (which corresponds to a change in the galvanometer scanner angle).

[0106] Figure 6B An example of a galvanometer and detector lower view control signal according to an embodiment of the present invention is shown. Figure 6B The example of Figure 6A is similar to Figure 6B except that Figure 6A it shows a lower view control signal timing opposite to the upper view control signal timing in Figure 6BIn this case, the galvanometer scanner control signal is controlled to have an increasing voltage 602c over time 606c after a constant time delay 608c corresponding to the desired angular difference between the upper and lower views. The desired viewing angle can be predetermined and stored in the storage unit 114, set according to user interface instructions, or set remotely. Image capture is controlled by the control unit 104 to control the capture through sensor row numbers 1-6 (i.e., 610d, 612d, 614d, 616d, 618d, and 620d), and the control unit 104 outputs a control signal voltage 602d that varies over time 606c without waiting for the time delay 608c. Thus, in this example, the time delay corresponds to Figure 6A the upper view scan in Figure 6B and the angular difference between the lower view scans in

[0107] Figure 6C An operating example of an embodiment of the present invention is shown to further illustrate how the change in the galvanometer scanner control voltage affects the scanning angle (or illumination angle). According to this example, the voltage change of the galvanometer scanner control signal corresponds to the scanning angle of the illumination light (i.e., the illumination angle), and the scanning angle corresponds to the position of the illumination line on the fundus of the eye. For example, at a galvanometer scanner voltage of 1.0 volts, the scanning angle can be 20 degrees. According to Figure 6C the example, the galvanometer scanner control signal 630 is controlled by the control unit 104 to have an increasing voltage 632 over time 636 without waiting for the time delay 638. The varying voltage of the galvanometer scanner control signal 630 directly corresponds to the change in the scanning angle 634 (also known as: illumination angle) over time 636, and the scanning angle 634 determines the position of the illumination line (e.g., illumination line 204a). The detector / sensor control signals 642, 644, and 646 that start after the time delay 638 operate to capture consecutive rows of the camera sensor (e.g., rows 1, 2, and 3 respectively) while the scanning angle changes during time 636. The maximum value 648 of the galvanometer scanner control signal 630 (e.g., 1 volt) corresponds to the maximum value of the scanning angle 634 (e.g., 20 degrees), and the zero voltage value 650 of the galvanometer scanner control signal 630 corresponds to a 0-degree scanning angle.

[0108] Figure 7A An example of the upper view control signal during narrow-angle dynamic viewing angle change on a galvanometer scanner-based imaging device according to an embodiment of the present invention is shown. Figure 7B An example of the lower view control signal during narrow-angle dynamic viewing angle change on a galvanometer scanner-based imaging device according to an embodiment of the present invention is shown. Except that in Figure 7A and Figure 7B the images are captured at a narrow viewing angle, Figure 7A and Figure 7BThe examples of Figure 6A and Figure 6B are respectively similar to the examples of Figure 7A . Thus, in the example of Figure 7B , the galvanometer scanner control signal 704a causes the voltage 702a to increase with time 706a without waiting for the time delay 708a, while the detector row number control signals 710b, 712b, and 714b operate after a constant time delay 708a to capture consecutive rows (e.g., rows 1, 2, and 3 respectively) in the camera to capture an upper view / narrow field of view image. In the example of

[0109] Figure 7C shows an example of an upper view control signal during a wide field of view dynamic view change on a galvanometer scanner-based imaging device according to an embodiment of the present invention. Figure 7D shows an example of a lower view control signal during a wide field of view dynamic view change on a galvanometer scanner-based imaging device according to an embodiment of the present invention. Except that in Figure 7C and Figure 7D , a wide field of view image opposite to the narrow field of view image captured in Figure 7A and Figure 7B is captured, the examples of Figure 7C and Figure 7D are respectively similar to the examples of Figure 7A and Figure 7B . According to the example of Figure 7C , the galvanometer scanner control signal 704d is controlled by the control unit 104 to have an increasing voltage 702e with time 706d, and the detector / sensor control signals 710d, 712d, and 714d capture consecutive image rows (e.g., rows 1, 2, and 3 respectively) after a delay time 708b to obtain an upper view / wide field of view image. According to the example in Figure 7D , the galvanometer scanner control signal 704e causes the voltage 702g to increase after a time delay 708d, and the detector / sensor control signals 710e, 712e, 714e, 716e, 718e, and 720e are output to capture consecutive image rows (e.g., rows 1 - 6 respectively) to obtain a lower view / wide field of view image.

[0110] The control unit 104 is configured to capture the narrow field of view images in Figure 7A and Figure 7B and the images captured according toFigure 7C and Figure 7D dynamically switch between the captured images with a wide viewing angle. According to an exemplary embodiment, the control unit 104 can alternately capture narrow-view images and wide-view images after each pair of captured images.

[0111] As Figure 2C shown in the example of Figure 2C , the illumination light is projected through the center of the pupil plane, and then the backscattered light is digitized by an off-center detector array (up / down or left / right). Narrow / wide is related to the field of view (FOV) of the system and determines how wide the imaging area is. However, the viewing angle determines the stereoscopic viewing angle related to the depth resolution in stereoscopic imaging.

[0112] Figure 6A , Figure 6B , Figure 7A , Figure 7B , Figure 7C and Figure 7D The examples of Figure 7C show how to dynamically adjust the viewing angle in stereoscopic imaging by shortening or lengthening the time interval (or the distance on the 2D detector) from the current illumination position. The dynamic viewing angle change can be described by the following equation:

[0113]

[0114] where θ view is the viewing angle in stereoscopic imaging, T is the relative time delay between two consecutive images (e.g., the up / down view images) (or Figure 2D the baseline distance in Figure 2D ), and M is the optical magnification coefficient. The control unit 104 adjusts the time delay T and the optical magnification coefficient M. For example, as the constant time delay between the galvanometer control signal and the detector control signal increases, the resulting viewing angle in stereoscopic imaging also increases at a given optical magnification coefficient. The viewing angle change also causes the depth resolution of the stereoscopic image to increase or decrease. The narrower the viewing angle, the greater the depth resolution of the stereoscopic image. Alternatively, the resulting viewing angle can be decreased or increased by simply increasing or decreasing the optical magnification coefficient (e.g., 1x ←→ 10x). As further described below, according to an embodiment of the present invention, the viewing angle value is used for the 3D depth reconstruction process performed by the processing unit 106. M, T, and θ viewManaged to achieve dynamic stereoscopic imaging (i.e., process perspective change, stereoscopic imaging, 3D reconstruction / analysis / visualization, optical magnification, recording, and clinical analysis / diagnosis such as comparison with a standard database and near-real-time disease progression detection). In real-time eye imaging, eye movement degrades the quality of stereoscopic images. Dynamic stereoscopic imaging and analysis without eye movement artifacts can improve clinical disease assessment. During stereoscopic imaging, the control unit can perform eye tracking. Additionally, the perspective can be changed not only by the optical magnification factor but also by the timing (or phase shift) between the galvanometer control signal and the detection control signal.

[0115] Figure 8 An example of a stereoscopic color fundus imaging device according to an embodiment of the present invention including a 2D camera having a rolling shutter mechanism plus a DMD for optical switching is shown. According to Figure 8 the embodiment, the illumination light from the illumination optical device 814 (a part of the imaging optical device 102 and also implementing the optical scanner unit 110) is output to the projection optical device 808 (another part of the imaging optical device 102) and is output from the projection optical device 808 toward the eye 806 to the imaging optical device 802 (a part of the imaging optical device 102). The backscattered light from the eye 806 is received by the objective lens 802c, the dynamic aperture mirror 802b, and the relay lens 802a in the imaging optical device 802. The backscattered light from the eye is directed by the imaging optical device 802 to the sensor array 804a in the camera 804 (i.e., the detector / sensor unit 112). The illumination optical device 804 includes a lens 814a, a DMD 814b, a controllable (by the control unit 104) iris aperture 814c, and an illumination light source 814d. Also included in this embodiment are: a control unit 104, a processing unit 106, a display unit 108, and a storage unit 114. As described above, the DMD 814b can be digitally (i.e., optically switched) opened or closed under the control of the control unit 104. In this embodiment, the DMD 814d is used to control the part of the eye illuminated at a given time, instead of the mechanical galvanometer scanner used in the previously discussed embodiments. As described above, compared to the galvanometer scanner, the DMD has some advantages such as 1) fast response speed, 2) mechanically more stable, 3) flexible illumination pattern, etc. Figure 8 The embodiment in is configured to be in accordance with Figure 2A 、 Figure 2B 、 Figure 3A 、 Figure 3B 、 Figure 4A and Figure 4COperate on each example shown in the figure. Control the operation according to a predetermined command stored in the storage unit 114, according to a command provided through the user interface on the control unit 104 or the processing unit 106, or according to a command received by the control unit 104 or the processing unit 106 from a remote location.

[0116] Figure 9A Examples of the view control signals on the DMD and the detector are shown, and Figure 9B Examples of the view control signals on the DMD and the detector are shown. When the dynamic aperture setting is in the split aperture mode, the constant time delay (e.g., time delays 910a, 910b, 1010a, 1010b, 1010c, and 1010d) between illumination and detection is zero because a perspective difference is introduced and controlled by switching the upper and lower view apertures. Therefore, the stereoscopic image acquisition speed is relatively faster than that of the aperture diaphragm-based stereoscopic imaging mode with a time delay greater than zero. As Figure 9A shown, the upper view is captured using the DMD control signals 912a, 914a, 916a, 918a, 920a, and 922a that vary the voltage 902a with time 904a without waiting for the constant time delay (i.e., perspective) 910a, while the detector / sensor control signals 924a, 926a, 928a, 930a, and 932a vary the voltage 906a with time 904a after waiting for the time delay 910a. As Figure 9B shown, the lower view is captured using the DMD control signals 912b, 914b, 916b, 918b, and 920b that vary the voltage 902b with time 904b after waiting for the constant time delay 910b, while the detector / sensor control signals 922b, 924b, 926b, 928b, 930b, and 932b vary the voltage 906b with time 904b without waiting for the time delay 910b.

[0117] Figures 10A to 10D Shows how an embodiment of the present invention can be dynamically switched between narrow view capture (in Figure 10A and Figure 10B ) and wide view capture (in Figure 10C and Figure 10D ). Figure 10AShows an example of an upper view control signal using DMD control signals 1012a, 1014a, 1016a, 1018a, 1020a, and 1022a and detector / sensor control signals 1024a, 1026a, 1028a, 1030a, and 1032a during a narrow - angle dynamic view change in DMD - based imaging according to an embodiment of the present invention. The DMD control signals 1012a, 1014a, 1016a, 1018a, 1020a, and 1022a cause the voltage 1002a to vary with time 1004a without waiting for the time delay 1010a, and the detector / sensor control signals 1024a, 1026a, 1028a, 1030a, and 1032a cause the voltage 1006a to vary with time 1004a after the waiting time delay (view angle) 1010a. Figure 10B Shows an example of a lower view control signal using DMD control signals 1012b, 1014b, 1016b, 1018b, and 1020b and detector / sensor control signals 1022b, 1024b, 1026b, 1028b, 1030b, and 1032b during a narrow - angle dynamic view change in DMD - based imaging according to an embodiment of the present invention. The DMD control signals 1012b, 1014b, 1016b, 1018b, and 1020b cause the voltage 1002b to vary with time 1004b after the waiting time delay 1010b, and the detector / sensor control signals 1022b, 1024b, 1026b, 1028b, 1030b, and 1032b cause the voltage 1006b to vary with time 1004b without waiting for the time delay 1010b.

[0118] Figure 10C Shows an example of an upper view control signal using DMD control signals 1012c, 1014c, 1016c, 1018c, 1020c, and 1022c and detector / sensor control signals 1024c, 1026c, and 1028c during a wide - angle dynamic view change in DMD - based imaging according to an embodiment of the present invention. The DMD control signals 1012c, 1014c, 1016c, 1018c, 1020c, and 1022c cause the voltage 1002c to vary with time 1004c without waiting for the time delay 1010c, and the detector / sensor control signals 1024c, 1026c, and 1028c cause the voltage 1006c to vary with time 1004c after the waiting time delay 1010c (which is greater than the time delays 1010a and 1010b, thus providing a wider view angle). Figure 10DShows an example of the lower view control signal using DMD control signals 1012d, 1014d, and 1016d and detector / sensor control signals 1018d, 1020d, 1022d, 1024d, 1026d, and 1028d during wide-angle dynamic view change in DMD-based imaging according to an embodiment of the present invention. The DMD control signals 1012d, 1014d, and 1016d cause the voltage 1002d to vary with time 1004d after a waiting time delay 1010d, and the detector / sensor control signals 1018d, 1020d, 1022d, 1024d, 1026d, and 1028d cause the voltage 1006d to vary with time 1004d without a waiting time delay 1010d. These figures show how to dynamically adjust the viewing angle in stereoscopic imaging by shortening or lengthening the time interval (or distance on the 2D detector) from the current illumination position. In Figure 10A and Figure 10B the time delays 1010a and 1010b are the same as each other. In Figure 10C and Figure 10D the time delays 1010c and 1010d are the same as each other. The time delays 1010c and 1010d are longer than the time delays 1010a and 1010b, resulting in Figure 10C and Figure 10D the viewing angle of the captured images in being wider than the viewing angle of the captured images in Figure 10A and Figure 10B The acquisition and 3D reconstruction speed can be close to real-time.

[0119] Figure 11A Shows an example of a stereoscopic color fundus imaging device including a multi-scan camera plus a galvanometer scanner according to an embodiment of the present invention, and Figure 11B shows Figure 11A an example of the operation of the embodiment in Figure 11AIn an embodiment, the illumination optical device 1112 outputs illumination light, which is further directed towards the eye 1106 by the dynamic aperture mirror 1116. The backscattered light from the eye 1106 is received by the objective lens 1112 and directed by the mirror 1110 to the projection optical device 1108, which redirects the backscattered light to the scanner 1118 (a part of the optical scanner unit 110), the mirror 1120, the lens 1104, the dynamic aperture mirror 1116, the relay lens 1114 and finally to the multi-line sensor camera 1102 including one or more multi-line sensor arrays 1102a. The multi-line sensor camera 1102 is a part of the detector / sensor unit 112. According to this example, the projection optical device 1108 forming a part of the imaging optical device 102 includes a lens 1108c, a mirror or DLP 1108b, and a black dot mask 1108a. The control unit 104, the processing unit 106, the display unit 108, and the storage unit 114 are also included in this embodiment.

[0120] As Figure 11B shown in the example of Figure 11A the embodiment, the first scan (top view) can be performed using the detection line 1120 and the illumination line 1122 to capture the first (top view angle) color fundus image 1136. While scanning the top view, the same camera can perform a second scan (bottom view) using the detection line 1134 and the illumination line 1122 to capture the second (bottom view angle) color fundus image 1138.

[0121] The multi-line scanning camera is a single camera including two or more line scanning detector arrays arranged in parallel. For example, a multi-line scanning camera including three rows of detector arrays can digitize the incident backscattered light 230 each time the camera is triggered (i.e., for each “on / off” of Figures 6A - 6B , Figures 7A - 7D , Figures 9A - 9B and Figures 10A - 10C ). Therefore, the overall acquisition speed may be much faster than that of the first two embodiments of Figure 5 and Figure 8 . However, in this embodiment, the viewing angle is fixed by the camera design. The multi-line scanning camera can be used to create a baseline distance (or viewing angle difference), which enables stereoscopic imaging (or top-bottom view, left-right view). In this method, the images obtained from at least two line scanners in the multi-scan camera should be divided into two images ( Figure 11B ). According to this embodiment, the viewing angle is fixed by the design of the multi-scan camera. The overall acquisition speed is much faster than the first two embodiments ( Figure 5 and [[ID ). Its control mechanism and synchronization are still as important as in the previous embodiments.

[0122] ​ , ​ and ​ illustrate exemplary methods of stereoscopic imaging. Two or more images acquired at specific different viewpoints can be used to obtain a stereoscopic image. Two different methods are illustrated: ​ parallel stereo acquisition in ​ and converging stereo acquisition in ​ These two methods provide comparable performance in human depth perception. In binocular stereo where the two camera axes are parallel, depth can be calculated given a disparity (the position offset of corresponding points between images). If the focal length of the two cameras is f, the baseline is b, and the disparity is d, then the depth z is given by z = (f × b) / d. In multi-baseline stereo, more than two cameras or camera positions are employed.

[0123] ​An example of a method for stereoscopic imaging with 3D visualization and analysis processing performed using a processing unit according to an embodiment of the present invention is shown. According to this example, the method starts in step S1302 and a pair of images (e.g., an upper and lower view or a left and right view) obtained by the detector / sensor unit 112 are provided to the processing unit 106 in step S1304. In step S1306, the disparity between the two images is estimated by registering the images point-to-point and recording the pixel-level differences. In step S1310, the disparity of the pixels is converted to an object distance (e.g., the actual distance on the fundus in millimeters). In step S1308, the desired / current viewing angle is set by control unit or user interface control. The desired viewing angle can be set at any time before or during imaging, and the control unit can adjust the viewing angle to meet the depth resolution requirements. Then, in step S1312, the actual depth (i.e., in three dimensions different from the two dimensions directly captured by the detector / sensor unit 112) is calculated using Equation (1) when using parallel views or Equation (2) when using convergent views:

[0124]

[0125]

[0126] where ΔZ parallel is the depth in parallel views, f is the focal length, b is the baseline distance between two virtual parallel cameras (i.e., the cameras with changed viewing angles generated by the embodiments of the present invention), d is the disparity measured by point-to-point registration, the viewing distance in parallel views, ΔZ converging is the depth calculated by convergent views, and θ is the viewing angle difference between two individual images. In step S1314, clinically relevant parameters or features can be extracted from the calculated depth profile and highlighted in the calculated depth profile. For example, glaucoma and age-related macular degeneration are common causes of blindness. Volume analysis such as the cup volume at the optic nerve head is directly related to glaucoma damage. Additionally, the change in volume over time can be used to correlate with the efficacy of clinical treatment. These types of analyses can be implemented on the processing unit. Next, before stopping in step S1322, the calculated depth information can be converted into a 3D map, a plot, or an image, which can be directly visualized in step S1316, transmitted elsewhere, or stored by the storage unit 114 in step S1320. If not completed in step S1318, the operation can be repeated.

[0127] ​ An example of stereoscopic color fundus images 1402 and 1404 captured according to an embodiment of the present invention is shown, and ​Shows a three-dimensional plot of information drawn in three dimensions 1406, 1408, and 1410 extracted from a stereoscopic color fundus image according to an embodiment of the present invention, for example, with a field of view of approximately 3 mm. ​ For example, the field of view is about 3 mm.

[0128] ​ Shows a three-dimensional scalar plot of information on three axes 1502, 1504, and 1506 for depicting the estimated cup depth 1510 resulting from cup depth estimation performed according to an embodiment of the present invention. ​ Shows a scalar plot of information in three dimensions 1518, 1520, and 1522 depicting the cup boundary 1512 and the disc boundary 1514 of the cup-to-disc ratio estimation performed according to an embodiment of the present invention. To perform cup depth estimation and cup-to-disc ratio estimation according to an embodiment of the present invention, first, a pair of stereoscopic images (e.g., top view and bottom view images) are acquired using the methods discussed herein. Next, point-to-point image registration is performed to calculate the disparity (or viewing angle) between two points. Then, this disparity (or angle) is used to estimate the depth of each point. The resulting depth profile (Z-axis) and the 2D color fundus image (X-axis and Y-axis) are rendered to visualize the depth information of the optic nerve head.

[0129] There are usually some difficulties in stereoscopic imaging, such as 1) eye movement during image acquisition, 2) artifacts caused by imaging angle changes, 3) lack of similarity between two or more images from different views (e.g., due to the movement of floaters in the eye), 4) long imaging time intervals between two or more consecutive acquisitions (e.g., the optical alignment and refocusing required between imaging), etc. These problems can be minimized by the embodiments disclosed herein because the stereoscopic acquisition in the present invention is acquired using a single camera and almost in real time. This will result in low motion artifacts for the stereoscopic color fundus images described herein. Additionally, due to the higher acquisition speed, the stereoscopic color fundus imaging method described herein can provide more reliable depth information.

[0130] Taking the above into consideration, the methods and systems described herein improve the existing stereoscopic color fundus imaging techniques at least in the following aspects: 1) the stereoscopic image acquisition speed; 2) the quality of the depth profile of a given sample is improved due to fewer eye movement artifacts; and 3) prognostic information such as cup depth, cup-to-disc ratio, and suspicious lesion volume is provided for eye diseases.

[0131] ​ Shows a block diagram of a computer that can implement the control unit 104 and / or the processing unit 106 according to the embodiments described herein.

[0132] Aspects of the present disclosure may be embodied as a system, a method, and / or a computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions recorded thereon, the computer-readable program instructions being configured to cause one or more processors to perform aspects of the embodiments.

[0133] A computer-readable storage medium may be a tangible device that can store instructions for use by an instruction execution device (processor). A computer-readable storage medium may be, for example, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of these devices, without limitation. A non-exhaustive list of more specific examples of computer-readable storage media includes each of the following (and suitable combinations): a floppy disk, a hard disk, a solid state drive (SSD), a random access memory (RAM), a read only memory (ROM), an erasable programmable read only memory (EPROM or flash memory), a static random access memory (SRAM), a compact disc (CD or CD-ROM), a digital versatile disc (DVD), and a memory card or stick. A computer-readable storage medium as used in the present disclosure is not to be construed as a transient signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses through an optical fiber cable), or electrical signals transmitted through a wire.

[0134] The computer-readable program instructions described in the present disclosure may be downloaded from a computer-readable storage medium to a suitable computing or processing device, or downloaded to an external computer or external storage device via a global network, i.e., the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission lines, optical communication fibers, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing or processing device may receive the computer-readable program instructions from the network and forward the computer-readable program instructions for storage in a computer-readable storage medium within the computing or processing device.

[0135] The computer-readable program instructions for performing the operations of the present disclosure may include machine language instructions and / or microcode, which may be compiled or interpreted from source code written in any combination of one or more programming languages including assembly language, Basic, Fortran, Java, Python, R, C, C++, C#, or similar programming languages. The computer-readable program instructions may be executed entirely on a user's personal computer, laptop computer, tablet computer, or smart phone, may be executed entirely on a remote computer or computer server, or may be executed on any combination of these computing devices. The remote computer or computer server may be connected to one or more of the user's devices via a computer network, including a local area network or a wide area network, or a global network, i.e., the Internet. In some embodiments, an electronic circuit, including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may execute the computer-readable program instructions by using information from the computer-readable program instructions to configure or customize the electronic circuit to perform aspects of the present disclosure.

[0136] Aspects of the present disclosure are described herein with reference to the flowcharts and block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It will be understood by those skilled in the art that each block of the flowcharts and block diagrams, and combinations of blocks in the flowcharts and block diagrams, can be implemented by computer-readable program instructions.

[0137] The computer-readable program instructions that can implement the systems and methods described in the present disclosure can be provided to one or more processors (and / or one or more cores within the processor) of a general-purpose computer, a special-purpose computer, or other programmable device to generate a machine such that the instructions executed by the processor of the computer or other programmable device create a system for implementing the functions specified in the flowcharts and block diagrams of the present disclosure. These computer-readable program instructions can also be stored in a computer-readable storage medium, which can direct a computer, a programmable device, and / or other devices to operate in a specific manner, such that the computer-readable storage medium having the instructions stored thereon is an article of manufacture including instructions for implementing aspects of the functions specified in the flowcharts and block diagrams of the present disclosure.

[0138] The computer-readable program instructions can also be loaded onto a computer, other programmable device, or other device such that a series of operational steps are executed on the computer, other programmable device, or other device to generate a computer-implemented process such that the instructions executed on the computer, other programmable device, or other device implement the functions specified in the flowcharts and block diagrams of the present disclosure.

[0139] ​Further shown is a networking system of one or more networked computers and servers. In an embodiment, ​ the hardware and software environment shown in can provide an exemplary platform for implementing the software and / or methods according to the present disclosure.

[0140] Referring ​ , the networking system can include a computer 1605, a network 1610, a remote computer 1615, a network server 1620, a cloud storage server 1625, and a computer server 1630, but is not limited thereto. In some embodiments, multiple examples of one or more functional blocks shown in ​ can be used.

[0141] In ​ , additional details of the computer 1605 are shown. The functional blocks shown within the computer 1605 are provided only to establish example functionality and are not intended to be exhaustive. Although details of the remote computer 1615, the network server 1620, the cloud storage server 1625, and the computer server 1630 are not provided, these other computers and devices can include functionality similar to that shown for the computer 1605.

[0142] The computer 1605 can be a personal computer (PC), a desktop computer, a laptop computer, a tablet computer, a netbook computer, a personal digital assistant (PDA), a smart phone, or any other programmable electronic device capable of communicating with other devices on the network 1610.

[0143] The computer 1605 can include a processor 1635, a bus 1637, a memory 1640, a non-volatile storage 1645, a network interface 1650, a peripheral interface 1655, and a display interface 1665. In some embodiments, each of these functions can be implemented as a separate electronic subsystem (an integrated circuit chip or a combination of a chip and related devices), or in other embodiments, some combination of the functions can be implemented on a single chip (sometimes referred to as a system-on-chip or SoC).

[0144] The processor 1635 can be one or more single-chip or multi-chip microprocessors such as those designed and / or manufactured by Intel Corporation, Advanced Micro Devices, Inc. (AMD), Arm Holdings (Arm), Apple Computer, etc. Examples of microprocessors include Celeron, Pentium, Core i3, Core i5, and Core i7 from Intel Corporation; Opteron, Phenom, Athlon, Turion, and Ryzen from AMD; and Cortex-A, Cortex-R, and Cortex-M from Arm.

[0145] The bus 1637 can be a proprietary or industry-standard high-speed parallel or serial peripheral interconnect bus such as ISA, PCI, PCI Express (PCI-e), and AGP.

[0146] The memory 1640 and the non-volatile storage 1645 can be computer-readable storage media. The memory 1640 can include any suitable volatile storage device such as dynamic random access memory (DRAM) and static random access memory (SRAM). The non-volatile storage 1645 can include one or more of the following: floppy disk, hard disk, solid state drive (SSD), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical disc (CD or CD-ROM), digital versatile disc (DVD), and memory card or memory stick.

[0147] The program 1648 can be a collection of machine-readable instructions and / or data stored in the non-volatile storage 1645 and used to create, manage, and control certain software functions discussed in detail elsewhere in the present disclosure and shown in the figures. In some embodiments, the memory 1640 can be much faster than the non-volatile storage 1645. In these embodiments, the program 1648 can be transferred from the non-volatile storage 1645 to the memory 1640 before being executed by the processor 1635.

[0148] The computer 1605 can communicate and interact with other computers via the network interface 1650 over the network 1610. For example, the network 1610 can be a local area network (LAN), a wide area network (WAN) such as the Internet, or a combination of both, and can include wired, wireless, or fiber optic connections. Generally, the network 1610 can be any combination of connections and protocols that support communication between two or more computers and associated devices.

[0149] The peripheral interface 1655 may permit input and output of data with other devices that may be locally connected to the computer 1605. For example, the peripheral interface 1655 may provide a connection to an external device 1660. The external device 1660 may include devices such as a keyboard, a mouse, a keypad, a touch screen, and / or other suitable input devices. The external device 1660 may also include portable computer-readable storage media such as, for example, a thumb drive, a portable compact disc or disk, and a memory card. Software and data for practicing embodiments of the present disclosure, such as program 1648, may be stored on such portable computer-readable storage media. In such embodiments, the software may be loaded onto the non-volatile storage 1645, or alternatively, directly loaded into the memory 1640 via the peripheral interface 1655. The peripheral interface 1655 may use industry standard connections such as RS-232 or Universal Serial Bus (USB) to connect to the external device 1660.

[0150] The display interface 1665 may enable the computer 1605 to be connected to a display 1670. In some embodiments, the display 1670 may be used to present a command line or a graphical user interface to a user of the computer 1605. The display interface 1665 may connect to the display 1670 using one or more proprietary or industry standard connections (such as, VGA, DVI, DisplayPort, and HDMI).

[0151] As described above, the network interface 1650 provides communication with other computing and storage systems or devices external to the computer 1605. The software programs and data discussed herein may be downloaded from, for example, a remote computer 1615, a network server 1620, a cloud storage server 1625, and a computer server 1630 to the non-volatile storage 1645 via the network interface 1650 and the network 1610. Additionally, the systems and methods described in the present disclosure may be executed by one or more computers connected to the computer 1605 via the network interface 1650 and the network 1610. For example, in some embodiments, the systems and methods described in the present disclosure may be executed by a remote computer 1615, a computer server 1630, or a combination of interconnected computers on the network 1610.

[0152] In embodiments of the systems and methods described in the present disclosure, the data, data sets, and / or databases employed may be stored and / or downloaded from a remote computer 1615, a network server 1620, a cloud storage server 1625, and a computer server 1630.

[0153] Obviously, many modifications and variations of the present invention are possible in light of the above teachings. Thus, it is to be understood that within the scope of the appended claims, the invention may be practiced otherwise than as specifically described herein.

Claims

1. An ophthalmic imaging device, comprising: Illumination optics including an illumination light source configured to output illumination light; A scanner configured to receive the illumination light from the illumination optics and redirect the illumination light toward a portion of an object to be imaged; An optical image capture device including a camera configured to receive backscattered light scattered by the object from the illumination light and capture a first image and a second image of the backscattered light; A control processor configured to control the scanner and the optical image capture device such that the optical image capture device captures the first image and the second image of the object, the first image and the second image being captured by the camera at different times and extracted from different portions of the backscattered light; And An image processor configured to generate a stereoscopic image from the first image and the second image of the object, wherein the control processor is configured to control the scanner and the optical image capture device such that the optical image capture device captures the first image to include a first portion of the backscattered light and captures the second image to include a second portion of the backscattered light, the optical image capture device receives the first portion of the backscattered light along a first backscattering angle, and the optical image capture device receives the second portion of the backscattered light along a second backscattering angle that differs from the first backscattering angle by a viewing angle difference.

2. The device according to claim 1, wherein, The scanner includes one of a galvanometer mechanical scanner, a digital micromirror device, and a polygon scanner.

3. The device according to claim 1, wherein, The control processor is configured to control the timing of a line number capture signal and a galvanometer scanner voltage control signal such that the first image and the second image are captured.

4. The apparatus according to claim 3, wherein, The line number capture signal indicates the capture timing of a specified one or more rows of pixels in a sensor array in the camera.

5. The device according to claim 3, wherein, The galvanometer scanner voltage control signal controls the deflection angle of light deflected by a galvanometer scanner mirror, and the voltage of the galvanometer scanner voltage control signal is controlled to change from a negative voltage to a positive voltage.

6. The device according to claim 1, wherein, the optical image capture device is controllable to change an optical magnification coefficient for capturing the first image and the second image; And the control processor changes the viewing angle difference by controlling the optical magnification coefficient of the optical image capture device and the timing of the line number capture signal such that the first image and the second image are captured.

7. The device according to claim 6, wherein, the control processor controls the scanner and the optical image capture device such that the optical image capture device captures a third image and a fourth image of the object; the control processor changes the viewing angle difference between the third image and the fourth image by controlling the optical magnification coefficient of the optical image capture device; and the control processor controls the viewing angle difference between the first image and the second image to be the same as the viewing angle difference between the third image and the fourth image.

8. The apparatus according to claim 7, wherein, the optical image capture device includes an actuator that controls the position of an optical amplifier to vary the optical magnification factor of the first and second images for capture.

9. The device according to claim 1, wherein, the object is the fundus of an eye.

10. The apparatus according to claim 1, wherein, the control processor is configured to control the scanner and the optical image capture device such that the optical image capture device captures the first image with a first viewing angle, and the control processor controls the first viewing angle based on a time delay between a control signal of the optical image capture device and a control signal of the scanner.

11. The device according to claim 6, wherein, a first portion of the backscattered light is received from the object at a first backscattering angle, and a second portion of the backscattered light is received from the object at a second backscattering angle that is separated from the first backscattering angle by 180 degrees.

12. The device according to claim 1, wherein, the control processor is configured to vary the capture order of the first and second images.

13. The device according to claim 1, wherein the control processor is configured to control the scanner and the optical image capture device such that the optical image capture device captures the first image of the object at a first viewing angle and captures the second image of the object at a second viewing angle that is different from the first viewing angle.

14. The device according to claim 1, wherein the control processor is configured to control at least one of the illumination optics, the scanner, and the optical image capture device to capture the first and second images of the object such that a field of view angle in the first image is the same as a field of view angle in the second image.

15. The device according to claim 1, wherein, the control processor is configured to control at least one of the illumination optics, the scanner, and the optical image capture device to capture a third image and a fourth image that have different viewing angles from each other, the same field of view angle from each other, and the field of view angle of the third and fourth images is different from the field of view angle of the first and second images.

16. The device according to claim 1, wherein, the image processor is configured to generate information for 3D visualization, depth profiling, volume quantification, disease progression, and information for a standard database for critical diagnosis.

17. The device according to claim 1, wherein, the control processor is configured to control the scanner and the optical image capture device such that the optical image capture device captures the first and second images of the object based on different configured apertures being used during capture of each of the first and second images to extract different portions of the backscattered light.

18. The apparatus according to claim 17, further comprising: a horizontal aperture in the optical path of the backscattered light; and the control processor is configured to dynamically switch the horizontal aperture between illuminating an upper portion of the object when capturing one of the first and second images of the object and illuminating a lower portion of the object when capturing the other of the first and second images of the object.

19. The apparatus according to claim 17, further comprising: A vertical aperture in the optical path of the backscattered light; And The control processor is configured to dynamically switch the vertical aperture between illuminating the left part of the object when capturing one of the first image and the second image of the object and illuminating the right part of the object when capturing the other of the first image and the second image of the object.

20. A method of operating an ophthalmic imaging device, the method comprising: Outputting illumination light; Receiving the illumination light; Redirecting an optical axis of the illumination light toward a portion of an object to be imaged; Receiving backscattered light scattered from the illumination light by the object; Capturing a first image and a second image of the object, the first image and the second image being captured at different times by the same optical image capture device and extracted from different portions of the backscattered light; And Generating a stereoscopic image from the first image and the second image of the object, Wherein, the method further comprises: Capturing the first image to include a first portion of the backscattered light; Capturing the second image to include a second portion of the backscattered light; Receiving the first portion of the backscattered light by the optical image capture device along a first backscattering angle; Receiving the second portion of the backscattered light by the optical image capture device along a second backscattering angle that differs from the first backscattering angle by a viewing angle difference.

21. The method according to claim 20, further comprising: Redirecting the optical axis of the illumination light using one of a galvanometric mechanical scanner, a digital micromirror device, and a polygon scanner.

22. The method according to claim 20, further comprising: Controlling the timing of a line number capture signal and a galvanometric scanner voltage control signal such that the first image and the second image are captured.

23. The method according to claim 22, further comprising: Controlling the line number capture signal to indicate the timing of capture of a specified one or more rows of pixels among multiple rows of pixels in a sensor array in the optical image capture device.

24. The method according to claim 22, further comprising: Controlling the galvanometric scanner voltage control signal to change a deflection angle of light deflected by a galvanometric scanner mirror, the voltage of the galvanometric scanner voltage control signal changing from a negative voltage to a positive voltage.

25. The method according to claim 20, further comprising: Controlling an optical magnification coefficient for capturing the first image and the second image; And Changing the viewing angle difference such that the first image and the second image are captured by controlling the optical magnification coefficient of the optical image capture device and the timing of the line number capture signal.

26. The method according to claim 25, further comprising: Capturing a third image of the object; Capturing a fourth image of the object; Changing a viewing angle difference between the third image and the fourth image by controlling the optical magnification coefficient of the optical image capture device; And Controlling the viewing angle difference between the first image and the second image to be the same as the viewing angle difference between the third image and the fourth image.

27. The method according to claim 26, further comprising: Controlling an actuator to change the position of the optical amplifier to change the optical magnification factor of the first and second images for capture.

28. The method according to claim 20, wherein, The object is the fundus of an eye.

29. The method according to claim 20, further comprising: Capturing the first image to have a first viewing angle; And Controlling the first viewing angle based on a time delay between a control signal for the optical image capture device and a control signal for the scanner.

30. The method according to claim 25, wherein Receiving the first portion of the backscattered light from the object at a first backscattering angle, and receiving the second portion of the backscattered light from the object at a second backscattering angle separated from the first backscattering angle by 180 degrees.

31. The method according to claim 20, further comprising: Changing the capture order of the first and second images.

32. The method according to claim 20, further comprising: Capturing the first image of the object at a first viewing angle and capturing the second image of the object at a second viewing angle different from the first viewing angle.

33. The method according to claim 20, further comprising: Capturing the first and second images of the object such that the field of view angle in the first image is the same as the field of view angle in the second image.

34. The method according to claim 20, further comprising: Capturing a third image and a fourth image, the third and fourth images having different viewing angles from each other, the same field of view angle from each other, and the field of view angle of the third and fourth images being different from the field of view angle of the first and second images.

35. The method according to claim 20, further comprising: Generating information for 3D visualization, depth contour, volume quantification, disease progression, and information for a standard database for key diagnosis.

36. The method according to claim 20, further comprising: Capturing the first and second images of the object to be extracted from different portions of the backscattered light based on using different configured apertures during capture of each of the first and second images.

37. The method according to claim 20, further comprising: Arranging a horizontal aperture in the optical path of the backscattered light; And Dynamically switching the horizontal aperture between illuminating the upper part of the object when capturing one of the first and second images of the object and illuminating the lower part of the object when capturing the other of the first and second images of the object.

38. The method according to claim 20, further comprising: Arranging a vertical aperture in the optical path of the backscattered light; And Dynamically switching the vertical aperture between illuminating the left part of the object when capturing one of the first and second images of the object and illuminating the right part of the object when capturing the other of the first and second images of the object.

39. A non - tangible computer - readable medium storing a computer program, which, when executed by a computer, causes the computer to perform steps of operating an ophthalmic imaging device, the steps including: Outputting illumination light; Receiving the illumination light; Redirecting the illumination light towards a portion of an object to be imaged; Receiving backscattered light scattered from the illumination light by the object; Capturing a first image and a second image of the backscattered light at different times by the same image - capturing device and extracting from different portions of the backscattered light; And Generating a stereoscopic image from the first image and the second image of the object, wherein the steps further include: Capturing the first image to include a first portion of the backscattered light; Capturing the second image to include a second portion of the backscattered light; Receiving the first portion of the backscattered light by the image - capturing device along a first backscattering angle; Receiving the second portion of the backscattered light by the image - capturing device along a second backscattering angle that differs from the first backscattering angle by a viewing - angle difference.

Images