Warm white light illumination of digital images during microsurgery and digital image processing

Abstract

A method of enhancing digital images during microsurgery, e.g., ophthalmic surgery, includes collecting digital images of a target anatomical structure using a digital camera when the target anatomical structure is illuminated by warm white light. The method includes identifying a predetermined stage of the microsurgery via a processor in communication with the digital camera. Within the image, the processor digitally separates a first pixel region, e.g., a pupil pixel region, from a second pixel region, e.g., an iris pixel region, and adjusts characteristics of its constituent pixels. The method, optionally recorded as instructions on a computer-readable medium, may be used to enhance the red reflex at a predetermined stage of the ophthalmic surgery. The system includes an illumination source for emitting warm white light having a color temperature of less than about 4000° K, a camera, and a processor.

Description

[Technical field]

[0001] The present disclosure generally relates to an automated method and associated hardware and software system-based digital image processing solutions for collecting and processing digital images of a target anatomical structure of a human patient when the target anatomical structure is illuminated with warm white light. [Background technology]

[0002] Modern microsurgery uses a wide range of precise, task-specific surgical instruments, each of which is manipulated by the surgeon in the surgical scene. In the operating room, one or more high-resolution display screens are often used to display magnified stereoscopic images of the surgical scene and the target anatomical structures. The digital presentation of the magnified images of the target anatomical structures allows the attending surgeon to properly visualize the surgical scene when performing a given surgical task.

[0003] Appropriate task lighting is necessary for visualization of the surgical scene. Surgical task instruments are task specific, and lighting devices may include any or all of the following: microscope mounted illumination sources, overhead surgical lighting arrays, surgeon worn headlights, or intraocular illuminators or light wands. Each lighting device emits light with a specific wavelength range and corresponding color temperature. Therefore, task lighting is often followed by digital image processing to present a more realistic representation of the target anatomical structure via the display screens and optics used in the operating room. Summary of the Invention [Means for solving the problem]

[0004] Disclosed herein is a method and associated system and software solution for selectively enhancing digital images collected in real time during microsurgery using warm white light illumination. Due to the benefits of minimizing the exposure of the human eye to blue wavelength light, the various embodiments described herein are primarily directed to ophthalmic surgery, including but not limited to cataract surgery, lens replacement, and other procedures that benefit from the detection and analysis of the eye's natural red reflex. However, those skilled in the art will appreciate that other microsurgeries seeking to limit blue light exposure of a given target anatomical structure can benefit from the disclosed solution.

[0005] In a particular embodiment, a method for enhancing a digital image during microsurgery includes collecting a digital image of a target anatomical structure of a human patient during microsurgery. This occurs by operation of a digital camera when the target anatomical structure is illuminated by warm white light. The method in this embodiment also includes identifying a predetermined stage of the microsurgery via a processor in wired or wireless communication with the digital camera. Within the digital image, operation of the processor separates a first pixel region from a second pixel region, e.g., in a typical ophthalmic surgery, a pupil pixel region from a surrounding iris pixel region, and then adjusts characteristics of constituent pixels that comprise the first or second pixel region. The image with the adjusted characteristics, i.e., the adjusted image, is then presented or displayed via one or more display screens.

[0006] Also disclosed herein is a system for enhancing digital images during ophthalmic surgery as described above. The system may include a digital camera and a processor as described above, along with an illumination source, such as a microscope mounted lamp, in communication with or integrated with the digital camera and the lamp. The illumination source is operable to direct warm white light onto / into a target eye. The digital camera is operable to collect a digital image of the target eye as the eye is illuminated by the warm white light.

[0007] In an exemplary configuration of the system, the processor detects a predetermined stage of the ophthalmic surgery as requiring enhancement of the red reflex of the target eye. For example, non-limiting representative stages of surgery during cataract surgery include incision, lens insertion, and lens removal. The processor in this embodiment tracks the movement of the target eye, for example, using a digital camera and motion tracking logic. In a digital image of the target eye during the predetermined stage of the ophthalmic surgery, the processor digitally separates the pupil pixel region from the iris pixel region surrounding the pupil pixel region, and adjusts the characteristics of the constituent pixels including the pupil pixel region or the iris pixel region, thereby generating an adjusted image. The adjusted image as provided herein has an enhanced red reflex. The processor is also operable to output video display control signals to at least one display screen to cause the screen to display the adjusted image during the predetermined stage of the ophthalmic surgery.

[0008] Another aspect of the present disclosure includes a computer readable medium having instructions recorded thereon for enhancing the red reflex during exemplary ophthalmic surgery, where execution of the instructions by a processor causes the processor to perform logical blocks or sequences of the method as described in detail below in this disclosure.

[0009] The above and other possible features and advantages of the present disclosure will become apparent from the following detailed description of the best modes for carrying out the disclosure when considered in conjunction with the accompanying drawings.

[0010] The drawings described herein are for illustrative purposes only and are schematic in nature and are intended to be illustrative rather than limiting on the scope of the present disclosure. [Brief description of the drawings]

[0011] [Figure 1] 1 illustrates an exemplary operating room using a processor configured to execute instructions embodying methods according to the present disclosure. [Diagram 2]2 is a schematic diagram of the processor shown in FIG. 1 in a non-limiting embodiment, including associated signal processing during an exemplary ophthalmic surgery, where the processor is part of an electronic control unit (ECU). [Diagram 3] Figure 2 is a schematic diagram of a system that enhances the red reflex and limits blue light exposure during a representative ophthalmic surgery. [Figure 4] 1 shows a pixelated image of a target eye in which the pupil and iris pixel regions are separated. [Diagram 5] 1 is a flow chart illustrating a method for digitally enhancing images collected under warm white light illumination conditions according to the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] The above summary is not intended to represent all possible embodiments or every aspect of the present disclosure. Rather, the foregoing summary is intended to illustrate some of the novel aspects and features disclosed herein. The above features and advantages, as well as other features and advantages of the present disclosure, will become readily apparent from the following detailed description of representative embodiments and modes of carrying out the present disclosure, when considered in conjunction with the accompanying drawings and the appended claims.

[0013] Here, an embodiment of the present disclosure will be described. However, it should be understood that the disclosed embodiment is merely an example, and other embodiments may take various alternative forms. The figures are not necessarily to scale. Some features may be exaggerated or minimized to show details of specific components. Therefore, specific structural and functional details disclosed herein should not be construed as limiting, but merely as a representative basis for teaching those skilled in the art to employ the present disclosure in various ways.

[0014] As will be appreciated by those skilled in the art, various features shown and described with reference to any one of the figures may be combined with features shown in one or more other figures to produce embodiments not explicitly shown or described. The combinations of features shown provide representative embodiments for typical applications. However, various combinations and modifications of features consistent with the teachings of the present disclosure may be desirable for particular applications or implementations.

[0015] In the following description, certain terms may be used for reference purposes only and are not intended to be limiting. For example, terms such as "top" and "bottom" refer to directions in the drawings to which they refer. Terms such as "front", "rear", "forward", "rear", "left", "right", "rear" and "side" describe the orientation and / or location of a component or part of an element within a consistent but arbitrary frame of reference that is made clear by reference to the text and associated drawings that describe the component or element under discussion. Furthermore, terms such as "first", "second", "third", etc. may be used to describe separate components. Such terms include those specifically mentioned above, derivatives thereof and words of similar import.

[0016] As understood in the art, the term "full spectrum light" describes the full wavelength range of human visible light, conventionally defined as 380 nanometers (nm) to 700 nm. In addition to wavelength, visible light is often described in terms of color temperature, using descriptions such as "warm white light," "neutral white light," and "cool white light." Color temperature is expressed in Kelvin (°K), and in particular, warm white light typically refers to light having a color temperature below about 4000°K. Such light falls primarily in the orange and red ranges of full spectrum light. In contrast to warm white light, cool white light has a higher color temperature, from about 5500°K to 7000°K or more, and is often dominated by blue light. Neutral white light lies somewhere in the middle of the conventionally defined color temperature ranges of warm white light and cool white light.

[0017] Medical microscopes are used in microsurgery to help surgeons accurately visualize relevant target anatomical structures. For example, ophthalmic microscopes and associated high-resolution image processing software allow ophthalmologists to accurately visualize the internal and external anatomical structures of the target eye. These microscopes are equipped to illuminate and image the eye. This is often done using full spectrum or neutral white light to produce natural looking, realistic images.

[0018] However, full spectrum light and daylight light contain high levels of blue light, i.e., light energy with wavelengths falling within the range of 380 nm to 450 nm. Prolonged exposure to blue light poses the risk of phototoxicity to exposed tissues, primarily the cornea, retina and lens, and to the exposed and irradiated surfaces. Such exposure can lead to increased patient recovery times, post-operative discomfort and cellular damage. Thus, the solution of the present invention is directed to utilizing the properties of warm white light in other ways while minimizing blue light exposure, such as by selectively enhancing the red reflex response of the target eye during typical ophthalmic surgery, as described below.

[0019] Referring now to the drawings, in which like reference numbers refer to like components, in FIG. 1, an operating room 10 is shown diagrammatically as it may appear during a typical microsurgery procedure. As will be appreciated by those skilled in the art, the operating room 10 may include a multi-axis surgical robot 12 and a surgical platform 14. The surgical robot 12 may be connected to a microscope 16, shown in FIG. 1 as a digital ophthalmic microscope, through which the surgeon may view the target anatomical structure of the patient at high magnification. An illumination source 35 and a digital camera 36 may be coupled to or integral with the microscope 16. For example, by using associated hardware and software, the surgeon using the microscope 16 may view a high magnification enhanced digital image 19 of the target anatomical structure. Visualization is facilitated via one or more high resolution display screens 20 and 200, any or all of which may include a touch screen 200T, e.g., a capacitive display surface. As shown, the enhanced digital image 19 is of a representative target eye 30 (see FIG. 2), with the image 19 of FIG. 1 including the pupil 300, the surrounding iris 350 and a portion of the sclera 400.

[0020] Also present within the operating room 10 is a cabinet 22 that contains an electronic control unit (ECU) 50C, the processor 52 of which is shown in Figure 2. The ECU 50C as shown may, in a possible implementation, be housed within the cabinet 22. Other embodiments in which the processor 52 is integrated or incorporated with other hardware within the operating room 10 separate from the cabinet 22 are described below, and thus the illustrated implementation of Figure 1 is non-limiting and exemplary, and the associated processing functions of the ECU 50C and processor 52 are described below as interchangeable without regard to the particular location of either device.

[0021] The ECU 50C is configured to receive digital image data (arrows 38), possibly stereoscopic images, as represented by the arrows "Image 1" and "Image 2" in FIG. 1. While collecting the digital image data (arrows 38), the ECU 50C executes instructions embodying a method 50, an example of which is described below with reference to FIG. 5. The ECU 50C is used as part of a system 70, the components of which are shown in FIGS. 2 and 3, which in some embodiments is operable to selectively enhance the red reflex response ("red reflex") of the target eye 30 (see FIGS. 2 and 3). The enhancement of the red reflex is performed in different embodiments automatically or upon command of the surgeon in accordance with the present disclosure. Furthermore, the red reflex enhancement can be limited within the scope of the present disclosure to only certain stages of the ophthalmic surgery where detection and evaluation of the red reflex by the surgeon is beneficial, such as making an incision or removing or inserting an intraocular lens in an exemplary ophthalmic surgery.

[0022] Regarding the red reflex, during cataract surgery and some other ophthalmic procedures, surgeons may wish to detect and evaluate the reflex performance of the eye in response to incident light. Thus, the term "red reflex" generally refers to a detectable reflex phenomenon that occurs when light enters the pupil 300 and reflects off the retina 18 behind the vitreous cavity 23, both of which are shown in FIG. 2 with a characteristic red hue. The red reflex test is frequently used by ophthalmic surgeons and other clinicians to detect possible abnormalities in the anatomical structures behind the eye. Also detectable by the red reflex test are opacities located along the optical axis 11, also shown in FIG. 2 with the pupil 300 and the lens 29. Such opacities are often present due to cataracts, for example, which result in a progressive clouding of the lens 29. Other possible causes of red reflex deficiency include corneal scarring and vitreous hemorrhage. Thus, the lack of an adequate red reflex response is of concern to surgeons when diagnosing or treating various eye diseases.

[0023] In connection with this established exemplary use, the ECU 50C shown in Figure 1 is programmed with instructions or computer executable code embodying an algorithm, which instructions or code are executed in turn to perform the method 50 of Figure 5. When performing the method 50, the ECU 50C seamlessly presents the enhanced digital image 19 via the display screens 20 and / or 200. That is, the digital image processing functions are performed by the ECU 50C unobtrusively and transparently from the surgeon's perspective, such that the enhanced digital image 19 ultimately achieves a desirable enhanced red reflex while minimizing exposure to potentially harmful blue light.

[0024] 2, the target eye 30 is shown undergoing a representative eye surgery performed with the aid of an ophthalmic safety system 70 that emphasizes the red reflex. As part of such an eye surgery with the aid of the system 70, the target eye 30 is illuminated with warm white light (arrow WL) that is directed onto and ultimately into the target eye 30 by an illumination source 35. The illumination source 35 may be embodied as a lamp, e.g., a light emitting diode (LED) based, halogen based, or other suitably configured illumination source 35 coupled to or integral with the ophthalmic microscope 16 shown in FIG. 1. In a possible embodiment, the warm white light (arrow WL) is "warm" in the sense of having a color temperature of less than about 4000° K, while in other embodiments color temperatures slightly outside of this range are possible, e.g., up to about 4500° K.

[0025] The system 70 in the illustrated embodiment also includes a digital camera 36 operable to collect a digital image of the target eye 30 under illumination conditions using warm white light (arrow WL). In an exemplary embodiment, the digital camera 36 may be a high dynamic range (HDR) digital camera of the microscope 16 shown in FIG. 1 and described above. Thus, the components of the system 70 may be integral with the microscope 16, i.e., assembled internal components thereof or attached external components thereof, and the process steps of the method 50 of FIG. 4 are programmed functions of the microscope 16. Instructions embodying the method 50 may be recorded in a non-transitory computer-readable medium, such as the memory 54 of the ECU 50C, to enable the ECU 50C or the processor 52 to cause the disclosed operations of the system 70, and may be executed by the processor 52 of the ECU 50C as illustrated, or a processor 52 located separately from the ECU 50C in other embodiments, to realize other embodiments. As mentioned above, in alternative embodiments, the processor 52 may be integrated into other hardware, such as the microscope 16 and / or the digital camera 36, ​​and the inclusion of the processor 52 in the structure of the ECU 50C is non-limiting.

[0026] During a given stage of a typical ophthalmic surgery where the surgeon wishes to examine and evaluate the red reflex of the target eye 30, the processor 52 will cause the illumination source 35 to emit warm white light (arrow WL), which may entail simply turning on the illumination source 35 at the beginning of the surgery. At the same time, the processor 52 will, for example, cause a corresponding camera control signal (arrow CC 36 ) to instruct digital camera 36 to collect digital image data (arrow 38). The collected digital image data (arrow 38) is communicated via transmission lines or wirelessly to processor 52 for performing various digital image processing steps embodying method 50.

[0027] When selectively enhancing the red reflex of the target eye 30 as part of the method 50, the processor 52 ultimately generates a video display control signal (arrow CC 20) to the display screen 20 and / or 200 to cause the display screen 20 and / or 200 to display a magnified dynamic image of the target eye 30 as shown below. At other times when the red reflex is not being assessed, the digital camera 36 may be used as needed to image the target eye 30 with illumination possibly using light from other parts of the electromagnetic spectrum as needed at the surgeon's discretion.

[0028] The ECU 50C is shown in FIG. 2 as a schematic unitary box for clarity and simplicity of illustration only. An embodiment implementing the ECU 50C may include one or more networked computing devices, each with a processor 52 and a sufficient amount of memory 54, including a non-transitory (e.g., tangible) medium on which a set of computer-readable instructions embodying the method 50 ("RR-Algo") that is readable and executable by the processor 52 is recorded or stored. A graphical user interface (GUI) device 60 may be used to facilitate intuitive interaction with the system 70 of the surgeon and the accompanying surgical team, examples of possible uses of which are described in more detail below. The memory 54 may take many forms, including, but not limited to, non-volatile and volatile media. Instructions embodying the method 50 may be stored in the memory 54 and selectively executed by the processor 52 to perform various functions, as described below. The ECU 50C, either as a stand-alone device or integrated into the digital camera 26 and / or microscope 16, may also include resident machine vision / motion tracking logic 58 ("Vision-Track") that tracks the target eye 30 and possibly performs other tasks such as identifying surgical instruments and / or the surgeon that occur during the course of ocular surgery, as will be shown below.

[0029] As will be appreciated by those skilled in the art, non-volatile media may include optical and / or magnetic disks or other persistent memory, and volatile media may include dynamic random access memory (DRAM), static RAM (SRAM), etc., any or all of which may constitute the primary memory of ECU 50C. Input / output (I / O) circuitry 56 may be used to facilitate connection and communication with various peripheral devices used during surgery, including digital camera 36, ​​illumination source 35, and high resolution display screens 20 and / or 200. Other hardware not shown, but commonly used in the art, may be included as part of ECU 50C, including, but not limited to, local oscillators or high speed clocks, signal buffers, filters, amplifiers, etc.

[0030] 3, operation of system 70 generally as described above includes illumination of target eye 30 with warm white light (arrow WL), which is directed onto / into target eye 30 by illumination source (L) 35, for example, directly or using polarized light. The incident warm white light (arrow WL) enters target eye 30 through a dilated pupil 300 and reflects off the surface of vitreous chamber 21 or retina 18, which is posterior in FIG. 2, as understood in the art. The reflected light (arrow WL R ) can be redirected via mirror 39 towards an absorptive filter 40. The absorptive filter 40 then redirects the reflected light (arrow WL R ) towards a pre-positioned set of photosensors 42 in wired or wireless communication with ECU 50C or processor 52 described above. The absorbing filters 40 and photosensors 42 may be integral to digital camera 36 in some embodiments, or the absorbing filters 40 may be positioned externally as shown for clarity of illustration in FIG.

[0031] As part of this technique, the processor 52 is configured to detect a predetermined stage of the ophthalmic surgery that requires detection of the red reflex, either at the sole discretion of the surgeon or in other embodiments as assisted by the detection capabilities or machine learning assistance of the ECU 50C. For example, identifying the predetermined stage of the ophthalmic surgery may optionally be accomplished by receiving an input signal (arrow CC) from the GUI device 60. 60 1-3 may be operatively connected to the GUI device 60 to include receiving an input signal (arrow CC) shown as a bidirectional signal indicating a two-way handshake communication. 60 ) indicates a predetermined stage of the eye surgery. The surgeon may manually input the predetermined stage of the eye surgery by interacting with the display screen 20 or 200 (itself optionally configured as a capacitive touch surface). Alternatively, the ECU 50C or processor 52 may determine the identity of the surgeon performing the eye surgery, such as by using machine learning, e.g., a neural network to recognize certain behavioral patterns during the course of the surgery. In such an embodiment, the input signal (arrow CC 60 ) indicates the identity of the surgeon.

[0032] If detection of the red reflex is desired, the processor 52 can be used to track the movement of the target eye 30 using the digital camera 36 and resident motion tracking logic 58, with the operation of the digital camera 36 being controlled by a camera control signal (arrow CC 36 3, the target eye 30 during surgery has a dilated pupil 300 surrounded by an iris region 350, which is further surrounded by the sclera, i.e., the white of the eye, of the target eye 30. The pupil 300 and direction of gaze of the target eye 30 may be detected and tracked in real time by the processor 52 using logic 58 of FIG. 2 as understood in the art, such tracking being indicated in FIG. 3 by a tracking circle 300-T.

[0033] 4, within a collected digital image 19 of a target eye 30, constructed from digital image data (arrow 38) transmitted to the processor 52 during a given stage of an ophthalmic procedure, the processor 52 digitally separates the pupil pixel region 30P from the iris pixel region 30I surrounding the pupil pixel region 30P. As used herein, the pupil pixel region 30P corresponds to the image pixels that correspond to the imaged surface region of the pupil 300 of FIG. 3. Similarly, the iris pixel region 30I corresponds to the image pixels that correspond to the imaged surface region of the iris region 350 of FIG. 3, and possibly of the surrounding sclera 400, e.g., as sclera pixel region 30S. The processor 52 then differentially adjusts the characteristics of the constituent pixels including the pupil pixel region 30P or the iris pixel region 30I, and possibly both, to thereby generate adjusted image data having a digitally enhanced red reflex.

[0034] This may be done based on a predetermined stage of ophthalmic surgery, or adjusting the characteristics of the constituent pixels may be done based on the identity of the surgeon. For example, the processor 52 may learn the lighting preferences of the surgeon over time, e.g., using a neural network or other suitable machine learning algorithm, and adjust the characteristics of the constituent pixels based at least in part on the lighting preferences, either alone or via an input signal (arrow CC) via the processor 52 using the GUI device 60. 60 ) can be generated. Using an exemplary usage scenario, if, for example, "Surgeon A" has consistently demonstrated a particular lighting preference in the previous N surgeries, ECU 50C can register Surgeon A's preferences for a different surgeon ("Surgeon B") in memory 52 and then implement the lighting preferences during the N+1th surgery without requiring any affirmative response or input from the surgeon. Processor 52 can then generate a video display control signal (arrow CC) as shown in FIG. 20 ) to display screen 20 and / or 200 to cause display screen 20 and / or 200 to display the adjusted image during a given stage of the eye surgery.

[0035] 5, a non-limiting embodiment of the method 50 can be used to selectively enhance the red reflex response of the target eye 30 (FIGS. 2 and 3) during ophthalmic surgery while simultaneously minimizing exposure to blue light. As noted above, the present teachings are particularly suited for use during some stages of cataract surgery, such as the incision and cataract removal stages of a typical cataract surgery. As noted above, aspects of the present teachings can be applied to other types of microsurgery or other medical procedures in which it is beneficial to reduce exposure of human tissue to incident blue light.

[0036] Method 50, particularly when used to optimize surgical and patient health outcomes for ophthalmic procedures, combines the use of eye-safe warm white light illumination with the programmed digital image processing capabilities of ECU 50C or processor 52 to simultaneously create an ideal, realistic digital rendering and projection of the surgical scene via selectively adjusted images.

[0037] An exemplary embodiment of the method 50 begins at block B51 with illuminating and imaging the target eye 30, as best shown in Figure 3. Block B51 includes collecting a digital image of the target eye 30 using the digital camera 36 when the target eye 30 is illuminated with warm white light (arrow WL). If the digital camera 36 is embodied as or includes an HDR digital camera, for example as an integral component of the microscope 16 of Figure 1, collecting the digital image data (arrow 38 of Figure 2) may include collecting a three-dimensional HDR image of the target eye 30.

[0038] As part of block B51, warm white light (arrow WL in FIGS. 2 and 3) can be directed coaxially or obliquely onto / into the target eye 30 by illumination source 35 shown in FIG. 2, possibly using polarization techniques as understood in the art. Illumination source 35 can be an integral component of microscope 16 in some configurations. In some implementations of method 50, warm white light (arrow WL) has a color temperature less than about 4000° K, while in other implementations a color temperature range above 4000° K is possible. However, because it is a goal of method 50 to reduce exposure to blue light, most or substantially all of the constituent light used as warm white light (arrow WL) should be comprised of light within the red spectrum, e.g., 620 nm to 750 nm, and / or orange spectrum, e.g., 590 nm to 620 nm, as opposed to full spectrum light.

[0039] One approach to implementing block B51 is to control the spectrum of the light emitted by the illumination source 35 so that the emitted light contains more red wavelengths. Warmer light results in higher red reflection. Alternatively, an excitation light filter 37 may be implemented after the illumination source 35 to block blue and green wavelengths, thereby increasing the red wavelength ratio. Such an approach would allow a cold illumination source 35 to be used, followed by blocking blue light (e.g., 400-500 nm) and some green light (e.g., 500-600 nm) via an absorption light filter 37. The resulting spectrum would have more red wavelength content, thus resulting in the intended warm white light (arrow WL). The method 50 then proceeds to block B53.

[0040] Block B53 uses the digital camera 36, ​​and in particular its photosensor 42, to measure the redirected reflected light (arrow WL in FIG. 3). R Possible embodiments of photosensor 42 include photon detectors in the form of, for example, photocells, photoelectric sensors, phototubes, or other suitable light detectors. For example, a typical digital camera construction uses a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) integrated circuit to detect the redirected reflected light (arrow WLR The detected photons (') may be processed into a corresponding electronic signal. The processed readings from photosensor 42 are ultimately sent to ECU 50C as image data (arrow 38) and method 50 then proceeds to block B54.

[0041] Block B54 involves identifying, via the processor 52 in communication with the digital camera 36, ​​a predetermined stage of the eye surgery that requires or would benefit from detection of the red reflex. To this end, identifying the predetermined stage of the eye surgery is performed in response to an input signal (arrow CC) from the GUI device 60. 60 1-3 may be operatively connected to a GUI device 60 shown in FIGS. 2 and 3 to receive an input signal (arrow CC 60 ) sometimes indicate certain stages of ophthalmic surgery.

[0042] Alternatively, the processor 52 may optionally identify the surgeon performing the ophthalmic surgery, in which case the input signal (arrow CC 60 ) indicates the identity of the surgeon. Exemplary techniques for identifying the surgeon include registering the surgeon's actions, such as touching an icon on GUI device 60 to select the surgeon's name from a displayed list of surgeon's names, or automatically identifying the surgeon using machine vision / face recognition software. Other possible embodiments include learning the surgeon's lighting preferences over time via processor 52, for example using a neural network or other machine learning algorithm, and then transmitting an input signal (arrow CC) via processor 52 based on the surgeon's identity and the surgeon's indicated lighting preferences. 60 ).

[0043] In some implementations of method 50, identifying the predetermined stage of the ophthalmic surgery may include detecting a surgical instrument via processor 52, 50C, for example, using digital camera 36 and associated machine vision logic. Such logic may be part of tracking logic 58 or a separate algorithm configured to detect and identify the surgical instrument based on its shape and size. Processor 52 may then automatically detect the predetermined stage of the ophthalmic surgery based on the identity of the surgical instrument. Method 50 proceeds to block B55 once the predetermined stage of the ophthalmic surgery is detected by any of the techniques described above or other suitable means. Method 50 alternatively proceeds to block B56 if processor 52 does not detect the predetermined stage of the ophthalmic surgery.

[0044] In block B55, the processor 52 digitally separates the pupil pixel region (PPR) 30P from the iris pixel region 30I surrounding the PPR 30P, as shown in FIG. 4. As part of block B55, the processor 52, alone or using associated functionality of the ECU 50C, can track the movement of the target eye 30 using the digital camera 36 and the motion tracking logic 58 described above. As is well understood in the art of artificial intelligence, eye tracking algorithms are commonly used to detect and track the user's gaze direction and transform the resulting detection data, such as an estimated optical axis, into a relevant frame of reference. In the present method 50, the relevant frame of reference includes the plane of constituent image pixels that make up the various digital images described by the collected image data (arrows 38). By tracking the movement of the pupil 300 of FIG. 3 and mapping its corresponding pixel location, the processor 52 can track the movement of the target eye 30 in real time, such that the pupil pixel region 30P remains identifiable in the displayed image throughout the course of the ophthalmic surgery. Once processor 52 has digitally isolated pupil pixel region 30P, method 50 proceeds to block B57.

[0045] Block B56 may include executing a default image processing algorithm on the full digital image of the target eye 30, including the pupil pixel region 30P and the iris pixel region 301, and possibly including imaged areas located outside the iris pixel region 301. If execution of block B54 determines that the red reflex is not required, in other words, the full digital image may be processed using a default image processing algorithm, with automatic white balance (AWB) being one possible default solution.

[0046] As understood in the art, the AWB algorithm automatically compensates for ambient lighting conditions when producing a more natural looking digital image. A typical AWB algorithm estimates the illuminant of the imaged scene in the form of a pixel image, and then corrects the color balance in the pixel image. For example, the color balance correction may include discarding the colors of some pixels at the extremes of the red, green and blue histogram spectrum using an automatic or user selected histogram adjustment, and then stretching the remaining range. Other techniques include averaging the hues of the lightest and darkest pixels in the image, or applying a weighting function. Once the default algorithm has been applied, the method 50 proceeds to block B60.

[0047] In block B57, the ECU 50C then determines whether a gain adjustment is required in the pupil pixel region 30P. A possible way of implementing block B57 includes allowing the surgeon to select the gain adjustment as an option, such as via the GUI device 60 of FIG. 3, by touching a corresponding icon on a touch-sensitive screen of the GUI device 60. Alternatively, the processor 52 may process the brightness, color, or other characteristics of the full image of the target eye 30 acquired in block B51 to determine whether improved red reflex contrast can be best achieved by processing the pupil pixel region 30P, the iris pixel region 30I, or both. As part of block B57, the processor 52 may compare the spectral content of the constituent pixels of the respective pupil pixel region 30P and iris pixel region 30I to each other and / or to a threshold value. If a gain adjustment is required for the constituent pixels of the pupil pixel region 30P, the method 50 proceeds to block B58. If not, the method 50 proceeds to block B59.

[0048] In block B58, processor 52 adjusts the characteristics of the constituent pixels that comprise pupil pixel region 30P to selectively emphasize the red reflex. As part of block B58, for example, processor 52 may increase the digital gain characteristically, where the adjustment is only for the constituent pixels of pupil pixel region 30P. As understood in the art of digital image processing, analog gain adjustments may be employed to adjust detection sensitivity, for example, in block B53, by amplifying or attenuating the corresponding voltage generated by each photon typically detected by photosensor 42 of FIG. 2. In contrast to analog gain adjustments, the digital gain adjustments employed in block B58 are used to adjust pixel values ​​after such voltages are read and digitized. Once the digital gain has been boosted in pupil pixel region 30P, method 50 proceeds to block B61.

[0049] Block B59 includes adjusting characteristics of the constituent pixels that comprise iris pixel region 30I, such as by applying a white balance algorithm to only the constituent pixels of iris pixel region 30I. Image processing algorithms performed simultaneously or alternatively in block B59 may include tone mapping, which compresses the tonal values ​​of the high dynamic range image to a lower range. Method 50 then proceeds to block B61.

[0050] In block B60, processor 52 displays the unadjusted image data (arrow 38) via display screen 20 and / or 200 of FIG. 1. Block B60 also generates a video display control signal (arrow CC in FIG. 2). 20 ) to the display screen 20 and / or 200 to cause illumination of the display screen 20 and / or 200 with the displayed magnified image of the target eye 30. The method 50 then returns to block B51.

[0051] Block B61 involves displaying the adjusted image via a display screen. The content of the image displayed varies depending on whether block B61 is reached from block B58, in which processor 52 boosts the digital gain of the pupil pixel region as described above, or from block B59, in which processor 52 applies a white balance algorithm or other suitable image processing technique to portions of the digital image that are outside the pupil pixel region. Method 50 then returns to block B51.

[0052] The present teachings as presented above combine warm white light illumination with selective digital image processing to selectively enhance the red reflex while reducing the risk of blue phototoxicity during cataract removal, lens replacement, and other ocular surgeries. This method 50 departs from the conventional practice of microscope-based illumination using broad spectrum light, i.e., light in the neutral white wavelength range.

[0053] Although digital color adjustments may be used on the displayed images to produce realistic looking surgical scenes under such broad spectrum or warm white light lighting conditions, the light entering the patient's eye is not affected and therefore contains high levels of blue wavelength light. Thus, conventional image color adjustments do not avoid the risks of phototoxicity discussed above. Furthermore, the present teachings are useful for selectively enhancing the red reflex during certain stages of surgery automatically and therefore unobtrusively to the surgeon. These and other advantages will be readily apparent to those skilled in the art in view of the foregoing disclosure.

[0054] The detailed description and drawings support and explain the disclosure, but the scope of the disclosure is defined only by the claims. Although some of the best modes and other embodiments for carrying out the claimed disclosure have been described in detail, there are various alternative designs and embodiments for carrying out the disclosure as defined in the appended claims. Furthermore, the features of the embodiments shown in the drawings or the various embodiments mentioned herein are not necessarily to be understood as independent embodiments from each other. Rather, each of the features described in one of the example embodiments may be combined with one or more of the other desired features from the other embodiments, resulting in other embodiments not described in writing or with reference to the drawings. Therefore, such other embodiments fall within the scope of the appended claims.

Claims

1. 1. A method for enhancing digital images during microsurgery, comprising: collecting digital image data of the target anatomical structure using a digital camera during the microsurgery procedure when the target anatomical structure is illuminated with warm white light; identifying, via a processor in communication with the digital camera, a predetermined stage of the microsurgery; separating a first region of pixels from a second region of pixels within the digital image data; selectively adjusting, via the processor, in response to an input signal, characteristics of constituent pixels comprising the first pixel region or the second pixel region to thereby provide an enhanced first pixel region or an enhanced second pixel region; transmitting video display control signals to at least one digital display screen to present an adjusted digital image of the target anatomical structure having the enhanced first pixel area or the enhanced second pixel area; The method includes:

2. The microsurgery is ophthalmic surgery and the target anatomical structure is a target eye of a human patient, and the method comprises: identifying a predetermined stage of the ophthalmic surgery requiring detection of a red reflex of the target eye, the first pixel area being a pupil pixel area of ​​a pupil of the target eye and the second pixel area being an iris pixel area of ​​an iris of the target eye; The method of claim 1 , comprising:

3. The method of claim 2 , further comprising directing the warm white light to the target eye using an illumination source of an ophthalmic microscope.

4. 2. The method of claim 1, wherein the digital camera comprises a high dynamic range (HDR) digital camera, and acquiring digital images comprises acquiring three-dimensional HDR images of the target anatomical structure via the HDR digital camera.

5. 2. The method of claim 1, wherein the processor is operatively connected to a graphical user interface (GUI) device, and identifying the predetermined stage of the microsurgery comprises receiving an input signal from the GUI device indicative of the predetermined stage of the microsurgery.

6. 2. The method of claim 1, wherein identifying the predetermined stage of the microsurgery includes detecting, via the processor, a surgical instrument used in the predetermined stage of the microsurgery using machine vision logic, and automatically detecting the predetermined stage of the microsurgery based on an identification of the surgical instrument.

7. learning, via the processor, lighting preferences of a surgeon performing the microsurgery over time using a neural network; adjusting, via the processor, the characteristics of the constituent pixels based on the lighting preferences; The method of claim 1 further comprising:

8. 2. The method of claim 1, wherein adjusting the characteristics of the constituent pixels that include the first pixel region or the second pixel region includes applying a white balance algorithm to only the constituent pixels that include the first pixel region or the second pixel region.

9. The method of claim 1 , further comprising tracking movement of the target anatomical structure via the processor during the predetermined stage of the microsurgery.

10. 2. The method of claim 1, wherein adjusting the characteristics of the constituent pixels including the first pixel region or the second pixel region includes automatically increasing, via the processor, a digital gain of only the constituent pixels of the first pixel region or the second pixel region.

11. 2. The method of claim 1, wherein separating the first pixel region from the second pixel region includes automatically tracking movement of the target anatomical structure using the digital camera and motion tracking logic of the processor.

12. 1. A system for enhancing digital image data during ophthalmic surgery, comprising: an illumination source operable to direct warm white light to a target eye, the warm white light having a color temperature of less than about 4000° K; a digital camera operable to collect a digital image of the target eye when the target eye is illuminated by the warm white light; a processor in communication with the digital camera, detecting a predetermined stage of the ophthalmic surgery requiring an enhanced red reflex in the target eye; automatically tracking movement of the target eye using the digital camera and motion tracking logic; digitally separating a pupil pixel region from an iris pixel region surrounding the pupil pixel region in the digital image of the target eye during the predetermined stage of the eye surgery; adjusting a characteristic of a constituent pixel including the pupil pixel region or the iris pixel region to thereby generate an adjusted image having the enhanced red reflex; outputting video display control signals to one or more display screens, thereby causing said display screens to display the adjusted image having the enhanced red reflex during the predetermined stage of the eye surgery; a processor configured to: A system including:

13. The system of claim 12 further comprising the display screen.

14. The system of claim 12 , wherein the illumination source is a lamp coupled to or integrated with an ophthalmic microscope.

15. The system of claim 14 , wherein the digital camera comprises a high dynamic range (HDR) digital camera of the ophthalmic microscope.

16. 13. The system of claim 12, further comprising a graphical user interface (GUI) device, wherein the processor is configured to identify the predetermined stage of the eye surgery by receiving an input signal from the GUI device, the input signal indicating the predetermined stage of the eye surgery.

17. 13. The system of claim 12, wherein the processor is configured to identify an identity of a surgeon performing the ophthalmic surgery, and the input signal is indicative of the identity of the surgeon, whereby the processor is configured to adjust the characteristics of the constituent pixels based on the identity of the surgeon.

18. The processor, in response to an input signal, calculates the characteristics of the constituent pixels as: applying a white balance algorithm to only the constituent pixels of the iris pixel region; or increasing the digital gain of only the constituent pixels of the pupil pixel region; The system of claim 12 , configured to regulate by:

19. 1. A computer readable medium having instructions recorded thereon for enhancing red reflex response during ophthalmic surgery, the execution of the instructions by a processor causing the processor to: receiving a digital image of the target eye from a digital camera in communication with the processor when the target eye is illuminated with warm white light from an ophthalmic microscope, the warm white light having a color temperature of less than about 4000° K; separating a pupil pixel region from an iris pixel region surrounding the pupil pixel region during a predetermined stage of the ophthalmic surgery requiring detection of an enhanced red reflex in the digital image of the target eye; adjusting characteristics of constituent pixels that include the pupil pixel region or the iris pixel region to thereby generate the adjusted image having the enhanced red reflex, comprising applying a white balance algorithm to only the constituent pixels of the iris pixel region or automatically increasing a digital gain of only the constituent pixels of the pupil pixel region; transmitting video display control signals to at least one display screen, thereby causing said at least one display screen to display said adjusted image during said predetermined stage of said eye surgery; A computer-readable medium for causing

20. 20. The computer-readable medium of claim 19, wherein the processor is operatively connected to a graphical user interface (GUI) device, and execution of the instructions by the processor causes the processor to identify the predetermined stage of the ophthalmic surgery using an input signal from the GUI device, the input signal indicating the predetermined stage of the ophthalmic surgery and / or an identity of a surgeon performing the ophthalmic surgery.

Images