Eye visualization, edge detection, and visible-IR image merging using a spectrally independent imager

Abstract

A visualization system for an ophthalmic procedure on a target eye, such as lens replacement surgery, includes a first light source and a second light source operable to direct visible light and NIR light toward the target eye, respectively. A hot mirror directs reflected light from the eye along two optical paths: a visible optical path for reflected visible light and an NIR optical path for reflected NIR light. A first camera and a second camera are disposed in the visible optical path and the NIR optical path, respectively, to detect reflected visible light or NIR light and output a visible image or NIR image. An electronic control unit (ECU) executes a method for detecting peripheral edges of an imaged portion of the target eye in the NIR image using edge detection logic, merges the visible image with the NIR image to construct a composite image, and indicates the peripheral edges in the composite image.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 384,694, filed November 22, 2022, which is incorporated herein by reference in its entirety. [Background technology]

[0002] introduction

[0001] The present disclosure relates to automated systems and methods for observing implantable medical devices and surrounding ocular anatomical structures during ophthalmic procedures. As is understood in the art, ophthalmic surgeries often require the attending surgeon or medical team to illuminate the lens, retina, vitreous, and surrounding tissues within a patient's eye. Visualization of the ocular anatomical structures, and potentially ocular implantable devices, is essential in many ophthalmic surgeries, including, but not limited to, cataract surgery, refractive lens exchange (REL), and other lens exchange procedures.

[0003] Specifically, with regard to lens replacement surgery, surgeons first use an ultrasound probe to destroy the patient's natural lens. After removing the resulting lens fragment through a small corneal incision, surgeons insert a replacement lens behind the patient's iris and pupil. The replacement lens, referred to in the art as an intraocular lens (IOL), then functions in place of the patient's natural lens. During cataract surgery, the patient's "red reflex," generated by the reflection of coaxial light from the retina to the observer, provides a contrast-rich background against which to view the structure of the patient's natural lens and the structure of the replacement lens. Different microscopes and lighting settings will affect the intensity and contrast of the red reflex, and the stability and intensity of the red reflex are important characteristics for microscopes during ophthalmic surgery. Summary of the Invention [Means for solving the problem]

[0004] Disclosed herein are systems and associated methods for automated visualization of a patient's target eye during an ophthalmic procedure. Implantable ocular devices, such as, but not limited to, intraocular lenses (IOLs), can be difficult to observe during lens replacement surgery. Ocular tissues, such as the internal limiting membrane (ILM), located between the retina and the vitreous, present similar challenges. Accordingly, the solutions disclosed herein are directed to improving a surgeon's ability to visualize implantable ocular devices and ocular anatomical structures during ophthalmic procedures, including, but not limited to, cataract surgery, refractive lens exchange (REL), vitrectomy, or other vitreoretinal surgery.

[0005] Current office-based and surgical visualization tools generally fail to fully take advantage of the numerous potential benefits of infrared (IR) imaging. This is particularly true during lens replacement surgery, but also when diagnosing ocular conditions such as capsular tears or visualizing similar thin structures such as the ILM described above. The human eye cannot visualize light in the IR spectrum. Nevertheless, IR imaging can be used to enhance traditional visible spectrum imaging in a properly equipped operating room. In general, the technical solutions described in detail below utilize different image sensors to simultaneously collect two different optical path images, providing the ability to collect and enhance image data of specific layers on the lens of a patient's eye.

[0006] In a possible embodiment, the method begins by illuminating the target eye with separate visible and near-infrared (NIR) light, i.e., from separate, spectrally specific light sources. The different spectra of reflected light from the target eye are similarly directed to separate, wavelength-tuned imagers or cameras. The cameras, which in one or more embodiments may be embodied as visible and NIR CMOS imagers, are configured to detect the visible and NIR spectrum, respectively.

[0007] The NIR image from the reflected NIR light is processed through edge detection logic in an electronic control unit (ECU) to detect edges in the image, such as the peripheral edge of an IOL. The ECU combines the visible and NIR images into a composite image and outputs a data set describing the corresponding locations of the peripheral edges. From this data set, the ECU can generate a two-dimensional (2D) or three-dimensional (3D) overlay graphic that is ultimately superimposed on the composite image in one or more embodiments.

[0008] A possible embodiment of the visualization system includes a first light source and a second light source, a hot mirror, first and second complementary metal-oxide semiconductor (CMOS) image sensors, and an ECU. The first light source in this embodiment is operable to direct visible light toward the target eye, and the first light source includes an array of red, green, and blue (RGB) laser diodes. The second light source directs NIR light toward the target eye and includes at least one NIR laser diode. The hot mirror is configured to direct reflected light from the target eye along two optical paths, including a visible optical path and an NIR optical path. The reflected light includes reflected visible light and reflected NIR light.

[0009] As part of this exemplary embodiment, a first CMOS image sensor is disposed in the visible light path and configured to detect reflected visible light and output a visible image composed of RGB pixels. A second CMOS image sensor is disposed in the NIR light path and configured to detect reflected NIR light and output an NIR image composed of NIR pixels. The ECU is programmed to detect a peripheral edge of an intraocular lens (IOL) in the NIR image using edge detection logic, merge the visible image with the NIR image to construct a composite image, and apply an overlay graphic on the composite image to indicate the peripheral edge of the IOL.

[0010] According to another embodiment, the visualization system includes a first light source operable to direct visible light toward the target eye and a second light source operable to direct NIR light toward the target eye. A hot mirror is configured to direct reflected light from the target eye along two optical paths, including a visible optical path and an NIR optical path, where the reflected light includes reflected visible light and reflected NIR light. A first camera is disposed in the visible optical path and detects the reflected visible light to output a visible image. A second camera is disposed in the NIR optical path and detects the reflected NIR light to output an NIR image. An electronic control unit (ECU) is programmed to detect peripheral edges of the imaged portion of the target eye in the NIR image using edge detection logic, merge the visible image with the NIR image to construct a composite image, and indicate the peripheral edges in the composite image.

[0011] Also disclosed herein is a method for use during an ophthalmic procedure on a target eye. The method may include directing visible light from a first light source toward the target eye and directing NIR light from a second light source toward the target eye. The method further includes using a hot mirror to direct reflected visible light and reflected NIR light from the target eye along a visible light path and an NIR light path, respectively. As part of this exemplary embodiment, the method includes detecting reflected visible light via a first camera positioned in the visible light path and responsively outputting a visible image, and detecting reflected NIR light via a second camera positioned in the NIR light path and responsively outputting an NIR image. Additionally, the ECU uses edge detection logic to detect peripheral edges of the imaged portion of the target eye in the NIR image and merges the visible image with the NIR image to construct a composite image, which then indicates the peripheral edges in the composite image.

[0012] The above features and advantages of the present disclosure, as well as other possible features and advantages, will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0013] [Figure 1]1 is a schematic diagram of an exemplary operating room configured with the visualization system described in detail herein; [Figure 2A] FIG. 2 is a front view of the target eye as it may be visualized in the operating room shown in FIG. 1. [Figure 2B] FIG. 2C is a cross-sectional side view of the target eye depicted in FIG. 2B. [Figure 3] 1 is a diagram of a representative combined visible and infrared pixel grid illustrating a possible implementation of the present teachings. [Figure 4] FIG. 2C is a front view of the target eye of FIG. 2B with an overlay graphic thereon to show the peripheral edge of the IOL. [Figure 5] 2C is a flowchart illustrating one embodiment of a method for detecting lens edges in the target eye images of FIGS. 2A and 2B using the visualization system of FIG. 1. DETAILED DESCRIPTION OF THE INVENTION

[0014] The above and other features of the present disclosure will become more fully apparent from the following description and appended claims, taken in conjunction with the accompanying drawings.

[0015] Embodiments of the present disclosure are described herein. However, it will be understood that the disclosed embodiments are merely exemplary and that other embodiments may take various alternative forms. The figures are not necessarily drawn to scale. Some features may be exaggerated or minimized to show details of particular components. Therefore, specific structural and functional details disclosed herein should not be construed as limiting, but merely as a representative basis for teaching those skilled in the art how to employ the present disclosure in various ways. As will be understood by those skilled in the art, various features shown and described with reference to any one of the drawings can be combined with features shown in one or more other drawings to create embodiments not explicitly shown or described. The combinations of features shown provide representative embodiments for typical applications. However, various combinations and modifications of features consistent with the teachings of the present disclosure may be desired for particular applications or implementations.

[0016] In the following description, certain terms may be used for reference purposes only and, therefore, are not intended to be limiting. For example, terms such as "upper" and "lower" refer to directions within the referenced drawings. Terms such as "front," "rear," "forward," "rearward," "left," "right," "rear," and "side" describe the orientation and / or location of a component or portion of an element within a consistent but arbitrary frame of reference that becomes clear by reference to the text and associated drawings that describe the component or element being discussed. Furthermore, terms such as "first," "second," and "third" may be used to describe separate components. Such terms may include those specifically mentioned above, derivatives thereof, and words of similar import.

[0017] Referring to the drawings, in which like reference numbers refer to like components, a representative operating room 10 is depicted generally in FIG. 1. The operating room 10 may be equipped with a multi-axis surgical robot (not shown), a surgical platform 12, such as an adjustable table or chair, and a visualization system 14 configured as described herein. The operating room 10 may be used in performing a surgical or diagnostic procedure on an eye 16 of a patient 18. The eye 16 that is a particular target surgical site according to the following disclosure will therefore be referred to hereinafter as the target eye 16 for clarity.

[0018] As contemplated herein, exemplary ophthalmic procedures that can be performed in the operating room 10 of Figure 1 include lens replacement surgery, e.g., cataract surgery or refractive lens exchange (REL), diagnosis or treatment of a condition of the target eye 16, such as a capsular tear, or visualization of the internal limiting membrane (ILM) (not shown) or other ocular anatomical structures of the target eye 16. During such procedures, surgeons may have difficulty visualizing implantable devices and / or ocular anatomical structures. While lens replacement surgery is described in the examples provided below, those skilled in the art will understand that other ophthalmic surgeries or in-office procedures may likewise benefit from the present teachings.

[0019] 1 in one or more embodiments may be connected to or in communication with an ophthalmic microscope 20 through which the surgeon can view the target eye 16. Alternatively, the visualization system 14 may be partially or fully integrated with the hardware and software of the ophthalmic microscope 20. Using the visualization system 14, the surgeon can view one or more composite images 22 of the target eye 16, which can be viewed within the operating room 10 via a corresponding high-resolution medical display screen 24 and, optionally, through an eyepiece (not shown) of the ophthalmic microscope 20.

[0020] An electronic control unit (ECU) 25 is also present in the exemplary operating room 10 of Figure 1. Within the scope of the present disclosure, the ECU 25 used in conjunction with or as an integral part of the visualization system 14 is programmed with software and equipped with hardware, i.e., configured to execute computer-readable instructions embodying the method 500, a representative implementation of which is described below with reference to Figure 5. Execution of the method 500, in turn, allows the surgeon to better visualize certain features of the target eye 16 when diagnosing or treating the target eye 16, as described above.

[0021] 2A and 2B, the target eye 16 includes an iris 27 surrounded by a sclera 26. A pupil 28 is centrally located within / surrounded by the iris 27. As shown in FIG. 2B, the target eye 16 also includes a cornea 30 that spans and protects the iris 27 and pupil 28. Light entering through the pupil 28 passes through a natural lens 32 that is connected to the surrounding anatomical structures of the target eye 16 via a ciliary muscle 34. FIG. 2B also shows a vitreous cavity 35 filled with vitreous humor (not shown), a retina 36 covering the posterior portion of the vitreous cavity 35, and an optic nerve 39 located posterior to the vitreous cavity 35 on the opposite side from the lens 32.

[0022] Upon executing the above-described set of instructions embodying method 500 or variations thereof, ECU 25 of FIG. 1 is enabled to support real-time visualization of target eye 16. To this end, visualization system 14 facilitates automated detection and tracking of peripheral edge 45 of intraocular lens (IOL) 320 during exemplary cataract surgery or refractive lens exchange (REL). The exemplary IOL 320 of FIG. 2B may be variously embodied as a monofocal, astigmatism-corrected, extended depth of focus, toric, multifocal, or, in different embodiments, accommodative IOL 320. Such IOL 320 includes an optical zone 42 configured to focus light incident through pupil 28 onto retina 38. The IOL 320 may also include arms or haptics 44 shaped and sized in a manner suitable for stabilizing and anchoring the IOL 320 within the target eye 16, as will be appreciated by those skilled in the art.

[0023] 1, ECU 25 is configured to identify peripheral edge 45 of FIG. 2B or an edge of another object, such as an ILM (not shown). ECU 25 then merges the different visible and near-infrared (NIR) light spectra to construct a composite image 22 of target eye 16, and optionally generates an overlay graphic 450 (see FIG. 4) indicating the current location of peripheral edge 45. After this generation, display screen 24 controls, for example, via electronic display control signals (arrows CC 24 ), is instructed by ECU 25 to superimpose and display overlay graphic 450 on top of composite image 22.

[0024] 1 as a monolithic box for illustrative clarity and simplicity, an ECU 25 within the scope of the present disclosure may include one or more networked devices, each having a central processing unit or other processor (P) 52 and a sufficient amount of memory (M) 54, including non-transitory (e.g., tangible) media that participates in providing data / instructions that can be read by processor 52. Instructions embodying an edge detection algorithm 55 may be stored in memory 54 and executed by processor 52 to perform the various functions described herein, thereby enabling the present method 500, illustrated in FIG.

[0025] The memory 54 may take many forms, including, but not limited to, non-volatile and volatile media. Non-volatile media may include optical and / or magnetic disks or other persistent memory, while volatile media may include dynamic random access memory (DRAM), static RAM (SRAM), etc., any or all of which may constitute the main memory of the ECU 25. The input / output (I / O) circuitry 56 may be used to facilitate connection to and communication with various peripheral devices used during an ophthalmic procedure, including the various hardware of the visualization system 14 of FIG. 1.

[0026] Other hardware not shown but commonly used in the art may be included as part of ECU 25, including but not limited to a local oscillator or high speed clock, signal buffers, filters, etc. A human machine interface (HMI) 15 is included within the structure of visualization system 14 to allow the surgeon to, for example, control the operation of the visualization system 14 via input signals (arrow CC 25 ) to the ECU 25. The ECU 25 may also communicate with the microscope via, for example, microscope control signals (arrow CC 20 ), or in a different embodiment via an input signal (arrow CC 25), which may directly control the ophthalmic microscope 20. Various implementations of the HMI 15 may be used within the scope of this disclosure, including, but not limited to, a foot switch, a touch screen, buttons, control knobs, a voice-activated speaker, etc. The ECU 25 of FIG. 1 may be configured to communicate over a network (not shown), such as a serial bus, a local area network, a controller area network, a controller area network with flexible data rates, or over Ethernet, Wi-Fi, Bluetooth, near field communications, and / or other forms of wired or wireless data connection.

[0027] Still referring to FIG. 1 , the visualization system 14 contemplated herein includes a first camera 60, labeled camera (VIS) for clarity, and a second camera (camera (NIR)) 62. Each of the first camera 60 and second camera 62 is configured to detect light in a specific portion of the electromagnetic spectrum. In particular, the first camera 60 is configured or “tuned” to detect incident reflected light 65R in the human visible spectrum, typically defined as corresponding to wavelengths from about 380 nanometers (nm) to about 750 nm. The second camera 62 is configured to detect reflected light 67R, a portion of which is in the NIR range, which, for purposes of implementing the present strategy, is typically defined as “eye-safe” wavelengths from about 780 nm to about 1.4 micrometers (μm).

[0028] In a possible configuration, first camera 60 and second camera 62 may be embodied as complementary metal-oxide semiconductor (CMOS) image sensors, such as CMOS imagers available from Teledyne Technologies of Thousand Oaks, CA. As recognized herein, if one were to attempt to simultaneously detect both visible and NIR light using a single CMOS imager, the resulting images would be suboptimal, at least in terms of their sharpness or color. Suboptimal images would be obtained from a CMOS imager with a broad sensitivity spectrum. As described herein, focusing NIR and visible light independently of one another ensures optimal sharpness and color across both relevant spectral ranges.

[0029] The visualization system 14 shown in FIG. 1 also includes a first light source 65 and a second light source 67, where the labels "V" and "NIR" correspond to visible light and NIR light, respectively. That is, the first light source 65 and the second light source 67 are configured to emit light toward the target eye 16 in designated portions of the electromagnetic spectrum. Specifically, the first (visible) light source 65 emits visible light 65L, i.e., human-visible light. The second light source 67 separately emits NIR light 67L. Thus, the visualization system 14 uses the first light source 65 and the second light source 67 as spectrally specific light sources, and similarly, the first camera 60 and the second camera 62 as spectrally specific imagers within the scope of the present disclosure.

[0030] Various solutions can be used to implement the respective first and second light sources 65, 67. For example, the first light source 65 used to generate the visible light 65L can include a red (R), a green (G), and a blue (B) laser diode as an RGB laser diode array configured to generate the visible light 65L as white light. For this purpose, commercially available highly compact RGB laser modules can be used, such as the Veglas™ RGB laser module from ams OSRAM AG. Similarly, the NIR light source 67 can be embodied as one or more commercially available NIR laser diodes.

[0031] During the illustrated surgical procedure, visible light 65L and near-IR light 67L reflect off the target eye 16 at an angle θ. The reflected visible light 65R and reflected NIR light 67R are directed along an optical axis AA that extends along the axis of the pupil 28 in FIG. 2A and includes a suitable optical target (“target”) 61. The optical target 61 may be stationary or may be directed by the ECU 25 via a target control signal (indicated by arrow CC). 61 ) may have one or more parameters, such as size, font, appearance, etc., that may be adjusted. The reflected NIR light 67R is then reflected from a hot mirror 68, which is typically embodied as a dielectric mirror and dichroic filter, as understood in the art, while passing the reflected visible light 65R. The hot mirror 68 may be positioned at approximately 45° relative to the second camera 62, as shown, so that the optical paths of the reflected visible light 65R and the reflected NIR light 67R are positioned orthogonal (90°) relative to one another.

[0032] Thus, the reflected NIR light 67R is directed towards the second camera 62, and optionally passes through a focusing lens 74. The reflected visible light 65R passes through a hot mirror 68, in this embodiment along optical axis AA, where it is incident on the first camera 60 as described above. Each of the first camera 60 and second camera 62 then outputs corresponding visible and IR images 71 and 73 to the ECU 25 for further processing.

[0033] 3, a representative pixel grid 75 illustrating a simplified version of composite image 22 (FIGS. 1 and 4) includes multiple rows and columns of digital pixels 75P, each constituent digital pixel 75P corresponding, in turn, to a particular region of the imaging spectrum, i.e., red (R), green (G), blue (B), or infrared (IR) light, in this example, NIR light.

[0034] As mentioned above, the basic principle of operation of the present disclosure is to provide the surgeon with an improved view of the IOL 320 (FIGS. 2B and 4), but in some cases, difficult-to-visualize anatomical structures or ocular conditions. This goal is achieved by combining the visible image 71 and NIR image 73 of FIG. 1 when imaging the target eye 16. That is, different wavelength spectrums are independently focused onto separate, purposefully configured photoreceptors, i.e., first camera 60 and second camera 62, as shown in FIG. 1. After the visible image 71 and NIR image 73 are collected from the first camera 60 and second camera 62, the ECU 25 executes the edge detection algorithm 55 to detect, isolate, and track the peripheral edges 45 depicted in FIGS. 2B and 4. The ECU 25 then merges this information with the visible image 71 from the first camera 60 to construct the composite image 22 of the target eye 16.

[0035] Several techniques can be used to generate the composite image 22 of FIG. 1, including the exemplary composite image 22 of FIG. 4 described below. For example, the ECU 25 can integrate NIR and color / RGB pixels, where some of the pixels 75P of FIG. 3 that would normally correspond to green (G) pixels in a Standard Bayer RGB color filter array (CFA) are replaced with IR pixels to form the composite image 22. The ECU 25 can display the composite image 22 via the display screen 24, as shown in FIG. 1, with the peripheral edge 45 of the IOL 320 of FIG. 2B possibly being represented in the composite image after first being detected from the IR pixels using edge detection logic 55.

[0036] As part of the disclosed technique, one or more RGB images may first be converted to a grayscale image before identifying edges for the purpose of identifying red reflection regions. Techniques such as Hough circle detection may be used to identify the best region of interest (ROI) for red reflection. Within the identified ROI, ECU 25 may identify reflection pixels with the highest blue channel signal. As understood in the art, these pixels include red reflection and reflection of the light source. Accordingly, one or more embodiments of the present method may replace these identified pixels with the average ROI intensity to help compensate for hidden red reflection. After replacing these pixels, ECU 25 may calculate the red channel intensity to help quantify the red reflection.

[0037] FIG. 4 shows a composite image 22 of the target eye 16 of FIG. 1 after placement of an IOL 360 therein. That is, the IOL 360 is centered relative to the iris 27 and pupil 28, and in some cases, a portion of the sclera 26 is in the field of view within the composite image 22. Haptics 144 are also observed at the outer periphery of the optical zone 42 of the implanted IOL 360. The IOL 360 is illustrative of a type of device or anatomical structure that, due to its transparency and relatively small size, can be difficult to identify with the naked eye, or even under high magnification. By detecting the peripheral edge 45 in the NIR image 73 of FIG. 1 and then combining the visible and IR pixels 75P of FIG. 3, the surgeon is provided with an improved view of the IOL 360.

[0038] To further assist the surgeon in visualizing the IOL 360, the ECU 25 in one or more embodiments may output an overlay graphic 450, such as a 2D or 3D trace, curve, shape, or other suitable indicator of the location of the peripheral edge 45. The overlay graphic 450 may be superimposed on the composite image 22 as shown. If the patient 18 of FIG. 1 moves the target eye 16 during the course of the procedure, the programmed eye tracking functionality of the visualization system 14 ensures that the overlay graphic 450 follows the movement of the peripheral edge 45, i.e., that the overlay graphic 450 remains superimposed on the peripheral edge 45.

[0039] 5, method 500 may be performed by ECU 25 of FIG. 1 as a series of steps or "logic blocks," each of which is executable by processor 52 of ECU 25. Method 500 according to the non-limiting exemplary embodiment of FIG. 5 begins at block B501 ("Illumination (16)"), as depicted in FIG. 1, where visible light 65L and NIR light 67L from respective first and second light sources 65 and 67 are illuminated onto target eye 16. As this process occurs, patient 18 should maintain focus on optical target 61. Method 500 proceeds to block B502, where respective visible light 65L and NIR light 67L are incident on target eye 16.

[0040] In block B502 (“Capture Images (71, 73)”), first camera 60 and second camera 62 of FIG. 1 receive reflected visible and NIR light 65R and 67R. In response, first camera 60 outputs visible image 71. Similarly, second camera 62 outputs NIR image 73. Once ECU 25 receives visible image 71 and NIR image 73, or begins receiving a stream of such images according to the calibrated sampling frequency, method 500 proceeds to block B504.

[0041] Block B504 of method 500 ("Lens Edge (450) Detection") involves detecting the peripheral edge 45 of the IOL 320 of FIGS. 2B and 4 using edge detection logic 55 of ECU 25. The edge detection process herein occurs primarily, or in some cases, only in the NIR spectrum. Edge detection by block B504 may include determining coordinates in an appropriate frame of reference, e.g., an XYZ Cartesian coordinate system, of a point in free space corresponding to the detected peripheral edge 45. Because the peripheral edge 45 will move with movement of the IOL 360 or target eye 16, the edge detection process of block B504 is ongoing throughout the procedure unless interrupted by the surgeon.

[0042] As will be understood in the art, various edge detection algorithms or image processing / computer vision software routines may be executed by ECU 25 for this purpose. By way of example and not limitation, ECU 25 may utilize a neural network or programmed logic to recognize patterns in NIR image 73 that represent peripheral edge 45. Alternatively, ECU 25 may execute the Marr-Hildreth algorithm, or calculate gradients in first and second derivatives, etc. Method 500 proceeds to block B506 once ECU 25 has detected and is actively tracking the position of peripheral edge 45.

[0043] Block B506 ("Generate Composite Image (22)") involves combining the previously acquired visible image 71 and NIR image 73 into a composite image 22, for example, as represented by pixel grid 75 of FIG. 3 and illustrated in FIG. 4. Once ECU 25 has constructed composite image 22, method 500 proceeds to block B508. As with block B504, performance of block B506 is controlled, for example, via signals from HMI 15 of FIG. 1 (arrow CC), unless otherwise initiated by the surgeon. 25 ), which can be run continuously or at a calibrated rate throughout the process, sending signals as control commands (arrows CC 25Once the ECU 25 constructs the composite image 22, the method 500 proceeds to block B508.

[0044] Block B508 of FIG. 5 (“Apply Overlay Graphic (450)”) involves generating and overlaying an overlay graphic 450 onto the composite image 22, a simplified example of which is shown in FIG. 4. This control action may include displaying a color image of the target eye 16, similar to that depicted in FIG. 4, on the display screen 24 of FIG. 1 or within the left and right optics (not shown) of the ophthalmic microscope 20 of FIG. 1 as a background. The control action of block B508 may include overlaying a 2D or 3D trace onto the peripheral edge 45 detected in block B504. The method 500 then returns to block B502, such that blocks B502, B504, B506, and B508 are executed in a loop during the course of an eye treatment.

[0045] As understood in the art, a surgeon may wish to selectively turn on or off any of the features of blocks B502-B508 as needed. As an example, a surgeon may not necessarily require the composite image 22 or overlay graphic 450, in which case the surgeon can temporarily disable the second camera 62 of FIG. 1 , with the associated overlay graphic 450 generating the above-described features. The HMI 15 shown in FIG. 1 may be used for this purpose. Similarly, a surgeon may not necessarily want to illuminate the target eye 16 with bright visible light, for example, due to the age or light sensitivity of the patient 18. In this case, the surgeon may decide to temporarily disable the first light source 65, i.e., the visible light source. Thus, the described method 500, in its various possible embodiments, may allow for the use of lower light levels because a bright red-reflecting light source is not required. The present teachings may similarly benefit vitreoretinal surgery. For example, by reducing the potential for phototoxicity, patient 18 may be more comfortable viewing optical target 61 of FIG. 1 and may allow pupil 28 (FIGS. 2A, 2B, and 4) to dilate naturally without the use of dilating medications and the complications that may sometimes be associated therewith.

[0046] The features of the various embodiments shown in the figures or mentioned in the description herein should not necessarily be understood as independent embodiments. Each of the features described in one of the examples of one embodiment may be combined with one or more other desired features from other embodiments to obtain other embodiments not described verbally or with reference to the drawings. Accordingly, such other embodiments are also included within the scope of the appended claims. While the detailed description and drawings support and explain the present disclosure, the scope of the present disclosure is defined only by the claims. While several embodiments for implementing the claimed disclosure have been described in detail, various alternative designs and embodiments exist for implementing the disclosure defined in the appended claims.

Claims

1. 1. A visualization system for use during an ophthalmic procedure on a target eye, comprising: a first light source operable to direct visible light toward the target eye, the first light source comprising an array of red, green, and blue (RGB) laser diodes; a second light source operable to direct near-infrared (NIR) light toward the target eye, the second light source comprising at least one NIR laser diode; and a hot mirror configured to direct reflected light from the target eye along two optical paths, including a visible optical path and a NIR optical path, the reflected light including reflected visible light and reflected NIR light; and a first complementary metal-oxide semiconductor (CMOS) image sensor disposed in the visible light path and configured to detect the reflected visible light and output a visible image comprised of RGB pixels; a second CMOS image sensor disposed in the NIR light path and configured to detect the reflected NIR light and output an NIR image composed of NIR pixels; an electronic control unit (ECU) programmed to use edge detection logic to detect a peripheral edge of an intraocular lens (IOL) in the NIR image, merge the visible image with the NIR image to construct a composite image, and apply an overlay graphic on the composite image to indicate the peripheral edge of the IOL.

2. The visualization system of claim 1 , further comprising a display screen in communication with the ECU and operable to display the overlay graphic on the composite image.

3. 10. The visualization system of claim 1, further comprising a human-machine interface in communication with the ECU, the human-machine interface operable to send control commands to the ECU to thereby change control settings of the first light source.

4. The visualization system of claim 1 , wherein the edge detection logic comprises a neural network and / or a Marr-Hildreth algorithm.

5. 1. A visualization system for use during an ophthalmic procedure on a target eye, comprising: a first light source operable to direct visible light toward the target eye; a second light source operable to direct near-infrared (NIR) light toward the target eye; a hot mirror configured to direct reflected light from the target eye along two optical paths, including a visible optical path and a NIR optical path, the reflected light including reflected visible light and reflected NIR light; and a first camera disposed in the visible light path and configured to detect the reflected visible light and output a visible image; a second camera disposed in the NIR light path and configured to detect the reflected NIR light and output an NIR image; an electronic control unit (ECU) programmed to use edge detection logic to detect peripheral edges of the imaged portion of the target eye in the NIR image, merge the visible image with the NIR image to construct a composite image, and indicate the peripheral edges in the composite image.

6. 6. The visualization system of claim 5, further comprising a display screen in communication with the ECU and operable to display an overlay graphic on the composite image, the overlay graphic indicating the location of the peripheral edge.

7. The visualization system of claim 5 , wherein the first camera and the second camera include complementary metal-oxide semiconductor (CMOS) image sensors.

8. 6. The visualization system of claim 5, wherein the ophthalmic procedure comprises a lens exchange surgery in which an intraocular lens (IOL) is inserted into the target eye, and the imaged portion of the target eye comprises the IOL.

9. 6. The visualization system of claim 5, further comprising a human-machine interface in communication with the ECU, the human-machine interface operable to send control commands to the ECU to thereby change a control setting of the first light source.

10. The visualization system of claim 5 , wherein the edge detection logic comprises a neural network.

11. The visualization system of claim 5 , wherein the edge detection logic comprises a Marr-Hildreth algorithm.

12. The visualization system of claim 5 , wherein the visible light path and the NIR light path are arranged orthogonal to each other.

13. 1. A method for use during an ophthalmic procedure on a target eye, comprising: directing visible light from a first light source toward the target eye; directing near-infrared (NIR) light from a second light source toward the target eye; using a hot mirror to direct reflected visible light and reflected NIR light from the target eye along a visible light path and a NIR light path, respectively; detecting the reflected visible light via a first camera disposed in the visible light path and outputting a visible image accordingly; detecting the reflected NIR light via a second camera disposed in the NIR light path and outputting an NIR image accordingly; detecting, via an electronic control unit (ECU), a peripheral edge of the imaged portion of the target eye in the NIR image using edge detection logic; merging the visible image with the NIR image to construct a composite image; indicating the peripheral edges in the composite image.

14. 14. The method of claim 13, wherein indicating the peripheral edge in the composite image includes displaying an overlay graphic on the composite image via a display screen in communication with the ECU, the overlay graphic indicating the location of the peripheral edge.

15. 14. The method of claim 13, wherein detecting the reflected visible light via a first camera and detecting the reflected NIR light via the second camera comprises using one or more complementary metal-oxide semiconductor (CMOS) image sensors.

16. 14. The method of claim 13, wherein the ophthalmic procedure includes a lens exchange surgery in which an intraocular lens (IOL) is inserted into the target eye, and detecting the peripheral edge of the imaging portion of the target eye in the imaging portion of the target eye includes detecting a peripheral edge of the IOL.

17. 14. The method of claim 13, wherein detecting the peripheral edge of the imaged portion of the target eye comprises detecting a peripheral edge of a red reflex of the target eye.

18. 14. The method of claim 13, further comprising sending control commands to the ECU via a human-machine interface to thereby change control settings of the first light source during the ophthalmic procedure.

19. The method of claim 13 , wherein detecting the peripheral edges of the imaged portion includes using a neural network as at least part of the edge detection logic.

20. The method of claim 13, wherein detecting the peripheral edge of the imaged portion includes using a Marr-Hildreth algorithm as at least part of the edge detection logic.

Images