Ophthalmic visualization using spectrally independent imager, edge detection, and visible IR image merging

Abstract

A visualization system for use in an ophthalmic procedure (e.g., a lens replacement procedure) on a target eye includes a first light source and a second light source operable to direct visible light and NIR light toward the target eye, respectively. The hot mirror directs reflected light from the eye along two paths, namely a visible light path for reflecting visible light and an NIR light path for reflecting NIR light. First and second cameras are positioned in the visible light path and the NR light path, respectively, to detect reflected visible light or reflected NIR light and output a visible image or NIR image. An electronic control unit (ECU) performs a method to detect a perimeter edge of an imaged portion of a target eye in an NIR image using edge detection logic, merge a visible image with the NIR image to construct a combined image, and indicate the perimeter edge in the combined image.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 384,694, filed on November 22, 2022, which is hereby incorporated by reference in its entirety. Background Art

[0003] The present disclosure relates to automated systems and methods for viewing implantable medical devices and surrounding ocular anatomical structures during ophthalmic procedures. As understood in the art, eye surgery typically requires the attending surgeon or medical team to illuminate the lens, retina, vitreous body, and surrounding tissues within the patient's eye. In many eye surgeries, including but not limited to cataract surgery, refractive lens exchange (REL), and other lens exchange procedures, visualization of ocular anatomical structures and possible ocular implantable devices is critical.

[0004] Specifically for lens replacement surgery, the surgeon first uses an ultrasonic probe to break up the patient's natural lens. After removing the resulting lens fragments through a small corneal incision, the surgeon inserts the replacement lens behind the patient's iris and pupil. The replacement lens, known in the art as an intraocular lens (IOL), then replaces the patient's natural lens to function. During cataract surgery, the patient's "red reflex," produced by coaxial light reflected back from the retina to the observer, provides a background with suitable contrast for observing the patient's natural lens structure and the structure of the replacement lens. Different microscopes and illumination settings will affect the intensity and contrast of the red reflex, with the stability and intensity of the red reflex being key characteristics of the microscope during eye surgery. Summary of the invention

[0005] Disclosed herein is a system and accompanying method for automatically visualizing a patient's target eye during an ophthalmic procedure. It may be difficult to observe an ocular implantable device, such as, but not limited to, an intraocular lens (IOL) during lens replacement surgery. Similar observation difficulties exist in ocular tissues, such as the internal limiting membrane (ILM) between the retina and the vitreous body. Therefore, the solution disclosed herein relates to improving the ability of surgeons to visualize ocular implantable devices and ocular anatomical structures during ophthalmic procedures, including, but not limited to, cataract surgery, refractive lens replacement (REL), vitrectomy or other vitreoretinal surgery, etc.

[0006] Current clinic-based surgical visualization tools often fail to fully exploit the many potential benefits of infrared (IR) imaging. While this is particularly true during lens replacement surgery, it remains the case when diagnosing eye conditions such as capsular tears or visualizing similar thin structures such as the ILM described above. The human eye cannot see light in the IR spectrum. However, in a properly equipped operating room, IR imaging can be used to enhance traditional visible spectrum imaging. In general, the technical solutions described in detail below utilize different image sensors to simultaneously collect images of two different light paths, and provide the ability to collect and enhance image data of specific layers on the lens of a patient's eye.

[0007] In a possible embodiment, the method first illuminates the target eye with separate visible light and near IR (NIR) light (i.e., light from different spectrally specific light sources). The reflected light of different spectra from the target eye is directed to also different wavelength tuned imagers or cameras. In one or more embodiments, the camera, which may be embodied as a visible CMOS imager and a NIRCMOS imager, is configured to detect the visible spectrum and the NIR spectrum, respectively.

[0008] The NIR image from the reflected NIR light is processed via edge detection logic of an electronic control unit (ECU) to detect edges in the image, such as the peripheral edge of an IOL. The ECU combines the visible image and the NIR image into a combined image and also outputs a data set describing the corresponding positions of the peripheral edges. The ECU can generate a two-dimensional (2D) or three-dimensional (3D) overlay graphic based on this data set, which in one or more embodiments is ultimately superimposed on the combined image.

[0009] A possible embodiment of the visualization system includes a first light source and a second light source, a thermal mirror, a first complementary metal oxide semiconductor (CMOS) image sensor and a second complementary metal oxide semiconductor image sensor, and an ECU. In this embodiment, the first light source is operable to direct visible light to a target eye, the first light source comprising an array of red, green, and blue (RGB) laser diodes. The second light source directs NIR light to the target eye and comprises at least one NIR laser diode. The thermal mirror is configured to direct reflected light from the target eye along two paths, the two paths comprising a visible light path and a NIR light path. The reflected light comprises reflected visible light and reflected NIR light.

[0010] As part of this exemplary embodiment, a first CMOS image sensor is positioned in the visible light path and is configured to detect reflected visible light and output a visible image composed of RGB pixels. A second CMOS image sensor is positioned in the NIR light path and is configured to detect reflected NIR light and output a NIR image composed of NIR pixels. The ECU is programmed to detect the peripheral edge of an intraocular lens (IOL) in the NIR image using edge detection logic, merge the visible image with the NIR image to construct a combined image, and apply an overlay graphic to the combined image to indicate the peripheral edge of the IOL.

[0011] A visualization system according to another embodiment includes a first light source operable to direct visible light to a target eye and a second light source operable to direct NIT light to the target eye. The thermal mirror is configured to direct reflected light from the target eye along two paths, the two paths comprising a visible light path and a NIR light path, wherein the reflected light comprises reflected visible light and reflected NIR light. A first camera is positioned in the visible light path, and detects reflected visible light and outputs a visible image. A second camera positioned in the NIR light path detects reflected NIR light and outputs a NIR image. An electronic control unit (ECU) is programmed to detect a peripheral edge of an imaging portion of the target eye in the NIR image using edge detection logic, merge the visible image with the NIR image to construct a combined image, and indicate the peripheral edge in the combined image.

[0012] Also disclosed herein is a method for use during an ophthalmic procedure performed on a target eye. The method may include directing visible light from a first light source to the target eye and directing NIR light from a second light source to the target eye. The method further includes using a hot mirror to guide reflected visible light and reflected NIR light from the target eye along a visible light path and a NIR light path, respectively. As part of this exemplary embodiment, the method includes: detecting reflected visible light via a first camera positioned in the visible light path, and outputting a visible image in response thereto; and detecting reflected NIR light via a second camera positioned in the NIR light path, and outputting a NIR image in response thereto. In addition, the ECU uses edge detection logic to detect a peripheral edge of an imaged portion of the target eye in the NIR image, merge the visible image with the NIR image to construct a combined image, and thereafter indicate the peripheral edge in the combined image.

[0013] The above features and advantages and other possible features and advantages of the present disclosure will become apparent from the following detailed description taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a schematic illustration of an exemplary operating room configured with a visualization system as set forth in detail herein.

[0015] Figure 2A Yes, you can Figure 1 Illustration of a front view of the target eye visualized within the operating room is shown.

[0016] Figure 2B yes Figure 2B Illustration of a cross-sectional side view of the target eye depicted in FIG.

[0017] Figure 3 is a representative combined visible and infrared pixel grid illustrating a possible implementation of the present teachings.

[0018] Figure 4 yes Figure 2B Illustration of a front view of a target eye with an overlay graphic depicted to indicate the peripheral edge of the IOL.

[0019] Figure 5 Is a description for use Figure 1 The visualization system in Figure 2A and Figure 2B Flow chart of an embodiment of a method for detecting a lens edge in an image of a target eye.

[0020] The foregoing and other features of the present disclosure will become more fully apparent from the following description and appended claims taken in conjunction with the accompanying drawings. DETAILED DESCRIPTION

[0021] Embodiments of the present disclosure are described herein. However, it should be understood that the disclosed embodiments are merely examples, and other embodiments may take various alternative forms. These drawings are not necessarily drawn to scale. Some features may be amplified or minimized to show the details of specific components. Therefore, the specific structural and functional details disclosed herein should not be interpreted as restrictive, but only as a representative basis for teaching those skilled in the art to adopt the present disclosure in different ways. As will be understood by those of ordinary skill in the art, the various features illustrated and described with reference to any of the accompanying drawings may be combined with the features illustrated in one or more other accompanying drawings to obtain embodiments that are not explicitly illustrated or described. The illustrated feature combinations provide representative embodiments for typical applications. However, for specific applications or implementations, various combinations and modifications consistent with the teachings of the present disclosure may be required for these features.

[0022] In the following description, certain terms may be used for reference purposes only and are therefore not intended to be limiting. For example, terms such as "above" and "below" refer to directions referenced in the accompanying drawings. Terms such as "front", "rear", "before", "after", "left", "right", "rear", and "side" describe the orientation and / or position of a portion of a component or element within a consistent but arbitrary reference system, which will become clear by reference to the text and associated drawings describing the component or element in question. In addition, terms such as "first", "second", "third", etc. may be used to describe individual components. Such terminology may include the words specifically mentioned above, their derivatives, and words of similar meaning.

[0023] Referring to the drawings, wherein like reference numerals refer to like parts, Figure 1 Schematically depicted in FIG. A representative operating room 10 is shown. The operating room 10 may be equipped with a multi-axis surgical robot (not shown), an operating platform 12 (such as an adjustable table or chair), and a visualization system 14 configured as described herein. The operating room 10 may be used when performing a surgical or diagnostic procedure on an eye 16 of a patient 18. Therefore, for clarity, the eye 16 that is the surgical site of a specific subject according to the following disclosure is hereinafter referred to as the target eye 16.

[0024] As envisioned in this article, Figure 1 Representative ophthalmic surgeries performed in the operating room 10 of the invention include lens replacement procedures (e.g., cataract surgery or refractive lens exchange (REL)), diagnosis or treatment of a condition of the target eye 16 (such as a capsular tear), or visualization of the internal limiting membrane (ILM) (not shown) or other ocular anatomical structures of the target eye 16. During such procedures, it may be difficult for the surgeon to visualize the implanted device and / or ocular anatomical structures. Although lens replacement surgery is described in the examples presented below, those skilled in the art will appreciate that other ophthalmic surgeries or outpatient procedures may similarly benefit from the present teachings.

[0025] In one or more embodiments, Figure 1 The visualization system 14 shown may be connected or in communication with an ophthalmic microscope 20 through which the surgeon is able to view the target eye 16. Alternatively, the visualization system 14 may be partially or completely integrated with the hardware and software of the ophthalmic microscope 20. Using the visualization system 14, the surgeon is able to view one or more combined images 22 of the target eye 16, which may be viewed within the operating room 10 via a corresponding high-resolution medical display screen 24, and possibly through an eyepiece (not shown) of the ophthalmic microscope 20.

[0026] exist Figure 1An electronic control unit (ECU) 25 is also present within the exemplary operating room 10 of FIG. The ECU 25 used with or as a component of the visualization system 14 within the scope of the present disclosure is programmed in software and equipped (i.e., configured) in hardware to execute computer-readable instructions embodying the method 500, a representative embodiment of which is described below with reference to FIG. Figure 5 Execution of the method 500 in turn allows the surgeon to better visualize certain features of the target eye 16 when diagnosing or treating the target eye 16, as described above.

[0027] Brief reference Figure 2A and Figure 2B , the target eye 16 includes an iris 27 surrounded by a sclera 26. A pupil 28 is centrally located within / surrounded by the iris 27. Figure 2B As shown, the target eye 16 also includes a cornea 30 that spans and protects an iris 27 and a pupil 28. Light entering through the pupil 28 passes through a natural lens 32, which in turn is connected to the surrounding anatomical structures of the target eye 16 via ciliary muscles 34. Figure 2B Also shown are a vitreous cavity 35 filled with vitreous fluid (not shown), a retina 36 lining the rear of the vitreous cavity 35 , and an optic nerve 39 disposed at the rear of the vitreous cavity 35 opposite to the lens 32 .

[0028] When executing the above instruction set embodying method 500 or its variants, such that Figure 1 The ECU 25 is operable to assist in real-time visualization of the target eye 16. To this end, the visualization system 14 facilitates automatic detection and tracking of the peripheral edge 45 of the intraocular lens (IOL) 320 in an exemplary cataract surgery or refractive lens exchange (REL). In various embodiments, Figure 2B The representative IOL 320 may variously be embodied as a monofocal IOL, an astigmatism-correcting IOL, an extended depth of focus IOL, a toric IOL, a multifocal IOL, or an accommodating IOL 320. Such an IOL 320 includes an optical zone 42 configured to focus light entering through the pupil 28 onto the retina 38. The IOL 320 may also include arms or haptics 44 that are shaped and sized to stabilize and secure the IOL 320 within the target eye 16, as will be understood by those skilled in the art.

[0029] Reference again Figure 1 , the ECU 25 is configured to identify Figure 2BECU 25 then combines the different visible and near infrared (NIR) spectra to construct a combined image 22 of the target eye 16 and may generate an overlay graphic 450 indicating the current position of the peripheral edge 45 (see FIG. Figure 4 ). After that, the ECU 25 controls the display via, for example, an electronic display control signal (arrow CC 24 ) commands the display screen 24 to superimpose and display the overlay graphic 450 on top of the combined image 22.

[0030] Although for the sake of clarity and simplicity of illustration, Figure 1 The illustrated ECU 25 is depicted as a monolithic block, but within the scope of the present disclosure, the ECU 25 may include one or more networked devices, each having a central processing unit or other processor (P) 52 and a sufficient amount of memory (M) 54 (including non-transitory (e.g., tangible) media that participate in providing data / instructions that can be read by the processor 52). Instructions embodying the edge detection algorithm 55 may be stored in the memory 54 and executed by the processor 52 to perform the various functions described herein, thereby implementing Figure 5 The method 500 is illustrated.

[0031] The memory 54 may take many forms, including but not limited to non-volatile media and volatile media. Non-volatile media may include optical and / or magnetic disks or other persistent storage, while volatile media may include dynamic random access memory (DRAM), static RAM (SRAM), etc., any or all of which may constitute the main memory of the ECU 25. Input / output (I / O) circuitry 56 may be used to facilitate communication with various peripheral devices used during ophthalmic procedures, including Figure 1 The connection and communication of various hardware of the visualization system 14).

[0032] Other hardware not depicted but commonly used in the art may be included as part of the ECU 25, including but not limited to a local oscillator or high-speed clock, signal buffers, filters, etc. A human machine interface (HMI) 15 may be included within the structure of the visualization system 14 to allow the surgeon to, for example, input signals (arrow CC) to the surgeon. 25 ) interacts with the ECU 25. In various embodiments, the ECU 25 may also control the ECU 25, for example, via a microscope control signal (arrow CC 20 ), or via input signal (arrow CC 25 ) directly controls the ophthalmic microscope 20. Within the scope of the present disclosure, the HMI 15 may be implemented in different ways, including but not limited to a foot switch, a touch screen, buttons, a control knob, a speaker for voice activation, etc. Figure 1The ECU 25 may be configured to communicate via a network (not shown) (e.g., a serial bus, a local area network, a controller area network, a controller area network with flexible data rate), or via Ethernet, Wi-Fi, Bluetooth TM , near field communication, and / or other forms of wired or wireless data connections for communication.

[0033] Still reference Figure 1 , the visualization system 14 contemplated herein includes a first camera 60 (labeled as camera (VIS) for clarity) and a second camera (camera (NIR)) 62. The first camera 60 and the second camera 62 are each configured to detect light within a specific portion of the electromagnetic spectrum. In particular, the first camera 60 is configured or "tuned" to detect incident reflected light 65R within the human visible spectrum, which is typically defined as corresponding to wavelengths of about 380 nanometers (nm) to about 750 nm. The second camera 62 is itself configured to detect reflected light 67R in the NIR range, which, for the purposes of executing the present strategy, is typically defined as "eye-safe" wavelengths of about 780 nm to about 1.4 micrometers (μm).

[0034] In a possible configuration, the first camera 60 and the second camera 62 may be embodied as complementary metal oxide semiconductor (CMOS) image sensors, such as commercially available CMOS imagers from Teledyne Technologies of Thousand Oaks, CA. As recognized herein, if an attempt is made to use a single CMOS imager to simultaneously detect visible light and NIR light, the resulting image will be suboptimal, at least in terms of its clarity or color. Suboptimal images are produced by CMOS imagers having a wide sensitivity spectrum. Therefore, as described herein, focusing NIR light and visible light independently of each other ensures optimal clarity and color in both relevant spectral ranges.

[0035] Figure 1 The visualization system 14 shown in also includes a first light source 65 and a second light source 67, where the labels "V" and "NIR" correspond to visible light and NIR light, respectively. That is, the first light source 65 and the second light source 67 are configured to emit light within a specified portion of the electromagnetic spectrum toward the target eye 16. Specifically, the first (visible) light source 65 emits visible light 65L, i.e., human visible light. The second light source 67 alone emits NIR light 67L. Therefore, within the scope of the present disclosure, the visualization system 14 employs the first light source 65 and the second light source 67 as spectrally specific light sources, and similarly employs the first camera 60 and the second camera 62 as spectrally specific imagers.

[0036] Various solutions may be used to implement the respective first light source 65 and the second light source 67. For example, the first light source 65 for generating the visible light 65L may include a red (R) laser diode, a green (G) laser diode and a blue (B) laser diode, for example, as an RGB laser diode array configured to generate the visible light 65L as white light. Commercially available highly compact RGB laser modules may be used for this purpose, for example the Veglas laser modules from ams OSRAM AG. TM RGB Laser Module. Similarly, the NIR light source 67 may be embodied as one or more commercially available NIR laser diodes.

[0037] During the illustrated surgical procedure, visible light 65L and near IR light 67L are reflected from target eye 16 at an angle θ. Reflected visible light 65R and reflected NIR light 67R are directed along an optical axis AA, which is along Figure 2A The optical target 61 may be static, or the optical target 61 may have one or more parameters (e.g., size, font, appearance, etc.) that the ECU 25 may control via a target control signal (arrow CC). 61 ) adjusts the one or more parameters. The reflected NIR light 67R is then reflected from the hot mirror 68 while also allowing the reflected visible light 65R to pass through the hot mirror, which is typically embodied as a dielectric mirror and a dichroic filter as understood in the art. The hot mirror 68 can be arranged at about 45° to the second camera 62 as shown, so that the paths of the reflected visible light 65R and the reflected NIR light 67R are arranged orthogonally (90°) relative to each other.

[0038] Thus, the reflected NIR light 67R is directed to the second camera 62, possibly passing through a focusing lens 74. In this embodiment, the reflected visible light 65R passes through a hot mirror 68 along an optical axis AA, and then the reflected visible light 65R is incident on the first camera 60 described above. Thereafter, the first camera 60 and the second camera 62 output corresponding visible images 71 and IR images 73, respectively, to the ECU 25 for further processing.

[0039] refer to Figure 3 , indicating a simplified version of the combined image 22 ( Figure 1 and Figure 4 ) includes a plurality of rows and columns of digital pixels 75P. Each constituent digital pixel 75P in turn corresponds to a specific region of the imaged spectrum, i.e., red (R) light, green (G) light, blue (B) light, or infrared (IR) (in this example, NIR).

[0040] As described above, the basic operating principle of the present disclosure is to provide the surgeon with an IOL 320 ( Figure 2B and Figure 4 ) and possibly other difficult to visualize anatomical structures or ocular conditions. This goal is to improve the observation of the eye 16 when imaging the target eye. Figure 1 This is achieved by combining the visible image 71 and the NIR image 73 in FIG. That is, spectra of different wavelengths are as follows: Figure 1 The images are shown as being independently focused onto separate, purposefully configured photoreceptors (i.e., first camera 60 and second camera 62). After collecting the visible image 71 and NIR image 73 from the first camera 60 and second camera 62, the ECU 25 executes the edge detection algorithm 55 to detect, isolate, and track Figure 2B and Figure 4 The ECU 25 then combines this information with the visible image 71 from the first camera 60 to construct the combined image 22 of the target eye 16.

[0041] There are many ways to generate Figure 1 The combined image 22 includes the following description Figure 4 Representative combined image 22. For example, ECU 25 can integrate NIR pixels and color / RGB pixels. Here, Figure 3 Some of the pixels 75P in the image (typically corresponding to green (G) pixels in a standard Bayer RGB color filter array (CFA)) are replaced with IR pixels to form the combined image 22. Figure 1 As shown, the ECU 25 can display the combined image 22 via the display screen(s) 24, where the edges are first detected by the IR pixels using edge detection logic 55. Figure 2B After the peripheral edge 45 of the IOL 320 is viewed, the peripheral edge may be represented in the combined image.

[0042] As part of the disclosed method, one or more RGB images may first be converted to grayscale images before identifying edges to identify red light reflection areas. Methods such as Hough circle detection may be used to identify the best region of interest (ROI) for red light reflection. Within the identified ROI, the ECU 25 may identify the reflective pixels with the highest blue channel signal. As understood in the art, these pixels contain red light reflection and reflection of the light source. Therefore, one or more embodiments of the present method may replace these identified pixels with the average ROI intensity to help compensate for hidden red light reflection. After replacing these pixels, the ECU 25 may calculate the red channel intensity to help quantify the red light reflection.

[0043] like Figure 4 As shown, it is shown Figure 1Combined image 22 of target eye 16 after IOL 360 is placed therein. That is, IOL 360 is centered relative to iris 27 and pupil 28, with portions of sclera 26 possibly visible within combined image 22. Similarly, haptics 144 are visible at the outer periphery of optical zone 42 of implanted IOL 360. Due to its transparency and relatively small size, IOL 360 is an example of a type of device or anatomical structure that is difficult to discern with the naked eye or even under high-power magnification. By Figure 1 The peripheral edge 45 is detected in the NIR image 73 and then Figure 3 With the combination of 75P visible pixels and IR pixels, surgeons can gain an improved 360-degree view of the IOL.

[0044] To further assist the surgeon in visualizing IOL 360, in one or more embodiments, ECU 25 may output an overlay graphic 450, such as a 2D or 3D trace, curve, shape, or other suitable indicator of the location of perimeter edge 45. As shown, overlay graphic 450 may be superimposed on combined image 22. Figure 1 If the patient 18 happens to move the target eye 16 during the procedure, the programmed eye tracking capabilities of the visualization system 14 will ensure that the overlay graphic 450 follows the movement of the peripheral edge 45, ie, the overlay graphic 450 remains superimposed on the peripheral edge 45.

[0045] See also Figure 5 , method 500 may be performed by Figure 1 The ECU 25 is executed as a series of steps or "logic blocks", each of which can be executed by the processor 52 of the ECU 25. Figure 5 Method 500 of the non-limiting exemplary embodiment of the present invention begins at block B501 ("Illuminate (16)"), i.e., Figure 1 As depicted, the target eye 16 is illuminated with visible light 65L and NIR light 67L from respective first and second light sources 65, 67. As this process proceeds, the patient 18 should continue to gaze at the optical target 61. When the respective visible light 65L and NIR light 67L are incident on the target eye 16, the method 500 proceeds to block B502.

[0046] At block B502 ("Capture Image (71, 73)"), Figure 1The first camera 60 and the second camera 62 receive the reflected visible light 65R and the reflected NIR light 67R. In response, the first camera 60 outputs a visible image 71. In a similar manner, the second camera 62 outputs a NIR image 73. Once the ECU 25 has received the visible image 71 and the NIR image 73, or has begun receiving such image streams according to a calibrated sampling frequency, the method 500 proceeds to block B504.

[0047] Block B504 ("Detect Lens Edge (450)") of method 500 includes using edge detection logic 55 of ECU 25 to detect Figure 2B and Figure 4 The peripheral edge 45 of the IOL 320 of the target eye 16 is detected. The edge detection process in this article mainly occurs or may only occur in the NIR spectrum. The edge detection according to block B504 may include determining the coordinates of a point in free space corresponding to the detected peripheral edge 45 in an appropriate reference system (e.g., an XYZ Cartesian coordinate system). Since the peripheral edge 45 moves with the movement of the IOL 360 or the target eye 16, the edge detection process of block B504 continues throughout the procedure unless interrupted by the surgeon.

[0048] As is understood in the art, the ECU 25 may run various edge detection algorithms or image processing / computer vision software routines for this purpose. As an example and not limitation, the ECU 25 may utilize a neural network or programmed logic to identify patterns in the NIR image 73 that represent the perimeter edge 45. Alternatively, the ECU 25 may execute a Marr-Hildreth algorithm, or may calculate gradients of first and second derivatives, etc. Once the ECU 25 has detected and is actively tracking the location of the perimeter edge 45, the method 500 proceeds to block B506.

[0049] Block B506 ("Generate Combined Image (22)") entails combining the previously collected visible image 71 and NIR image 73 into a combined image 22, e.g., as determined by Figure 3 The pixel grid 75 represents and Figure 4 Once the ECU 25 has constructed the combined image 22, the method 500 proceeds to block B508. As with block B504, execution of block B506 may be performed continuously or at a calibrated rate throughout the procedure unless the surgeon, for example, via a command from Figure 1 The signal of HMI 15 (arrow CC 25 ) is preempted separately, the HMI can be operated to transmit a signal as a control command (arrow CC 25 ). Once the ECU 25 has constructed the combined image 22, the method 500 proceeds to box B508.

[0050] Figure 5 Block B508 (“Apply Overlay Graphic (450)”) includes generating an overlay graphic 450 and superimposing it on the combined image 22, a simplified example of which is shown in FIG. Figure 4 This control action may include placing a color image of the target eye 16 (similar to Figure 4 The depiction in Figure 1 on the display screen 24, or as a background display Figure 1 The control action of block B508 may include superimposing the 2D or 3D trace on the perimeter edge 45 detected in block B504. The method 500 thereafter returns to block B502, so that blocks B502, B504, B506, and B508 are performed in a loop during the eye surgery procedure.

[0051] As is understood in the art, the surgeon may wish to selectively turn on or off any of the features of blocks B502-B508 as needed. As an example, the surgeon may not always need the combined image 22 or overlay graphic 450, in which case the surgeon may temporarily disable the combined image 22 or overlay graphic 450. Figure 1 The second camera 62 in and the above-mentioned related overlay graphic 450 generate features. Figure 1 The HMI 15 shown in can be used for this purpose. Likewise, the surgeon may not always wish to shine bright visible light into the target eye 16, for example due to the age or photosensitivity of the patient 18. In this case, the surgeon may decide to temporarily deactivate the first light source 65 (i.e., the visible light source). Thus, in various possible embodiments of the described method 500, the method may allow for the use of lower light levels because a bright red light reflecting light source is not required. The present teachings may also provide benefits for vitreoretinal surgery. For example, by reducing the risk of phototoxicity, the patient 18 will be able to see more easily. Figure 1 The optical target 61, and the pupil 28 ( Figure 2A , Figure 2B and Figure 4 ) can dilate naturally without the use of mydriatic medications and the possible complications that are sometimes associated with them.

[0052] The embodiments shown in the drawings or the features of the various embodiments mentioned in this specification are not necessarily to be understood as independent embodiments from each other. Each feature described in one of the examples of an embodiment can be combined with one or more other desired features from other embodiments to produce other embodiments that are not described in words or with reference to the drawings. Accordingly, such other embodiments fall within the scope framework of the attached claims. The specific embodiments and the drawings are supportive and descriptive for the present disclosure, but the scope of the present disclosure is limited only by the claims. Although some embodiments for implementing the claimed disclosure have been described in detail, there are various alternative designs and embodiments to practice the disclosure defined in the attached claims.

Claims

1. A visualization system for use during an ophthalmic procedure performed on a target eye, the visualization system comprising: a first light source operable to direct visible light toward the target eye, the first light source comprising an array of red, green, and blue (RGB) laser diodes; a second light source operable to direct near infrared (NIR) light toward the target eye, the second light source comprising at least one NIR laser diode; a thermal mirror configured to direct reflected light from the target eye along two paths, the two paths comprising a visible light path and a NIR light path, wherein the reflected light comprises reflected visible light and reflected NIR light; a first complementary metal oxide semiconductor (CMOS) image sensor positioned in the visible light path and configured to detect the reflected visible light and output a visible image composed of RGB pixels; a second CMOS image sensor positioned in the NIR light path and configured to detect the reflected NIR light and output a NIR image composed of NIR pixels; and an electronic control unit (ECU) programmed to detect a peripheral edge of an intraocular lens (IOL) in the NIR image using edge detection logic, merge the visible image with the NIR image to construct a combined image, and apply an overlay graphic to the combined image to indicate the peripheral edge of the IOL.

2. The visualization system of claim 1, further comprising: A display screen is in communication with the ECU and is operable to display the overlay graphic on the combined image.

3. The visualization system of claim 1, further comprising: A human machine interface is in communication with the ECU, wherein the human machine interface is operable to transmit a control command to the ECU to change a control setting of the first light source.

4. The visualization system of claim 1, wherein: The edge detection logic includes a neural network and / or a Marr-Hilders algorithm.

5. A visualization system for use during an ophthalmic procedure performed on a target eye, the visualization system comprising: a first light source operable to direct visible light to the target eye; a second light source operable to direct near infrared (NIR) light toward the target eye; a thermal mirror configured to direct reflected light from the target eye along two paths, the two paths comprising a visible light path and a NIR light path, wherein the reflected light comprises reflected visible light and reflected NIR light; a first camera positioned in the visible light path and configured to detect the reflected visible light and output a visible image; a second camera positioned in the NIR light path and configured to detect the reflected NIR light and output a NIR image; and an electronic control unit (ECU) programmed to detect a perimeter edge of an imaged portion of the target eye in the NIR image using edge detection logic, merge the visible image with the NIR image to construct a combined image, and indicate the perimeter edge in the combined image.

6. The visualization system of claim 5, further comprising: A display screen is in communication with the ECU and is operable to display the overlay graphic on the combined image, wherein the overlay graphic indicates a location of the perimeter edge.

7. The visualization system of claim 5, wherein: The first camera and the second camera include complementary metal oxide semiconductor (CMOS) image sensors.

8. The visualization system of claim 5, wherein: The ophthalmic procedure comprises a lens replacement procedure in which an intraocular lens (IOL) is inserted into the target eye, and wherein an imaging portion of the target eye comprises the IOL.

9. The visualization system of claim 5, further comprising a human-machine interface, the human-machine interface communicating with the ECU, wherein: The human machine interface is operable to transmit control commands to the ECU to change control settings of the first light source.

10. The visualization system of claim 5, wherein: The edge detection logic includes a neural network.

11. The visualization system of claim 5, wherein: The edge detection logic includes a Marr-Hilders algorithm.

12. The visualization system of claim 5, wherein: The visible light path and the NIR light path are arranged orthogonally to each other.

13. A method for use during an ophthalmic procedure performed on a target eye, the method comprising: directing visible light from a first light source toward the target eye; directing near infrared (NIR) light from a second light source toward the target eye; directing the reflected visible light and the reflected NIR light from the target eye along a visible light path and a NIR light path, respectively, using a thermal mirror; detecting the reflected visible light via a first camera positioned in the visible light path, and outputting a visible image in response thereto; detecting the reflected NIR light via a second camera positioned in the NIR light path, and outputting a NIR image in response thereto; detecting, via an electronic control unit (ECU), a perimeter edge of an imaged portion of the target eye in the NIR image using edge detection logic; merging the visible image with the NIR image to construct a combined image; as well as The perimeter edge is indicated in the combined image.

14. The method of claim 13, wherein indicating the perimeter edge in the combined image comprises displaying an overlay graphic on the combined image via a display screen in communication with the ECU, wherein: The overlay graphic indicates the location of the perimeter edge.

15. The method of claim 13, wherein detecting the reflected visible light via a first camera and detecting the reflected NIR light via the second camera comprises using one or more complementary metal oxide semiconductor (CMOS) image sensors.

16. The method of claim 13, wherein the ophthalmic procedure comprises a lens replacement surgery in which an intraocular lens (IOL) is inserted into the target eye, and wherein, Detecting a peripheral edge of an imaging portion of the target eye in an imaging portion of the target eye includes detecting a peripheral edge of the IOL.

17. The method of claim 13, wherein: Detecting a peripheral edge of an imaged portion of the target eye includes detecting a peripheral edge of a red reflex of the target eye.

18. The method of claim 13, further comprising transmitting control commands to the ECU via a human machine interface to change control settings of the first light source during the ophthalmic procedure.

19. The method of claim 13, wherein: Detecting a perimeter edge of the imaged portion includes using a neural network as at least a portion of the edge detection logic.

20. The method of claim 13, wherein: Detecting a perimeter edge of the imaged portion includes using a Marr-Hilderhausen algorithm as at least a portion of the edge detection logic.

Images