Adaptive optical system and method for vitreoretinal surgery

Abstract

The present disclosure provides an adaptive optical system comprising at least one active pixel sensor array that detects light and sends a signal to a processor. The adaptive optical system further comprises a wavefront correction system comprising at least one wavefront control structure and the processor, and executing instructions on the processor to produce a digital image in which at least one wavefront aberration in the light detected by the active pixel sensor array is partially or completely corrected. The present disclosure also provides methods of using the adaptive optical system.

Description

Technical Field

[0001] This disclosure relates to vitreoretinal surgery and surgical equipment, and more specifically, to adaptive optics systems and associated methods for improving digital images during vitreoretinal surgery. Background Technology

[0002] Ophthalmic surgery is surgery performed on the eye or any part of the eye. Every year, ophthalmic surgery saves and improves the vision of tens of thousands of patients. However, given the sensitivity of vision to even small changes in the eye and the tiny, fragile nature of many eye structures, ophthalmic surgery is challenging to perform, and even the reduction of small or unusual surgical errors or a slight improvement in the accuracy of surgical technique can make a huge difference in a patient's postoperative vision.

[0003] A type of ophthalmic surgery (vitrectomy) encompasses a variety of delicate procedures involving the internal parts of the eye, such as the vitreous fluid, retina, epiretinal membrane, and internal limiting membrane. Different vitreoretinal surgical procedures (sometimes with lasers) are used to improve visual perception in the treatment of many eye diseases, including epiretinal membrane, diabetic retinopathy, vitreous hemorrhage, macular hole, retinal detachment, vitreomacular traction syndrome, macular hole, and complications of cataract surgery, among others.

[0004] During ophthalmic surgeries, such as vitreoretinal surgery, ophthalmologists typically use non-electro-optical surgical microscopes with eyepieces to view magnified images of the eye undergoing surgery. More recently, vitreoretinal surgeons may use eyepiece-less digital imaging systems to aid visualization during vitreoretinal surgery. These systems can include 3D high dynamic range (“HDR”) camera systems with a pair of 2D complementary metal-oxide-semiconductor (CMOS) single-chip or three-chip sensors, allowing surgeons to view the retina on a display screen using polarized glasses, digital eyepieces, or a head-mounted display. The display screen eliminates the need for eyepieces to observe the surgery and allows others in the operating room to see the procedure just like the surgeon. Compared to traditional optical analog surgical microscopes, this system also allows for improved image quality at high magnification and increased depth of field, thus improving visualization of the eye. Summary of the Invention

[0005] This disclosure provides an adaptive optics system for improving digital images during vitreoretinal surgery. The adaptive optics system includes at least one active pixel sensor array that detects light and sends signals to a processor. The adaptive optics system also includes a wavefront correction system comprising at least one wavefront control structure and the processor, on which instructions are executed to generate a digital image in which at least one wavefront distortion in the light detected by the active pixel sensor array is partially or completely corrected.

[0006] Adaptive optics systems and methods of using them may include the following additional features: i) the system may include an array of multiple active pixel sensors; ii) the wavefront control structure may include a spatial light modulator (SLM), a liquid crystal on silicon spatial light modulator (LCoS-SLM), a transmissive LCoS-SLM, a reflective LCoS-SLM, a deformable mirror, or any combination thereof; iii) the wavefront control structure may include a phase-only SLM; iv) the digital image may be displayed on a digital display, a screen, a head-up display, a head-mounted display, or any combination thereof; v) the system may be (Components of Novartis AG, Switzerland); vi) The active pixel sensor array may be a CMOS monochrome sensor, a 4K monochrome CMOS sensor, a 1080P monochrome CMOS sensor, or any combination thereof; vii) The system may include an on-chip processor on the active pixel sensor array that implements region of interest (ROI) gain control; viiii) The system may include an amplitude-only SLM to implement ROI gain control on the active pixel sensor array; ix) The system may include an image reference system; x) The image reference system may include an image reference as a surgical tool placed in the eye; xi) The surgical tool placed in the eye may be a vitrectomy cutter, forceps, scissors, PIC forceps, scraper, bending ring, spatula, miniature vitreoretinal (MVR) blade, miniature cannula, or any combination thereof.

[0007] This disclosure further provides an adaptive optics system including a time-series color system comprising red, green, and blue light sources emitting sequences of red, green, and blue light at timed intervals. The system also includes at least one active pixel sensor array for detecting each of the red, green, and blue light and sending a signal to a processor. The system may include the following additional features: i) the time-series color system may include red light-emitting diodes (LEDs), red-green LEDs, and red-blue LEDs that pulse sequentially at a total rate of 180 Hz or higher; ii) the time-series color system may include superluminescent red light-emitting diodes (SLEDs), green light-emitting SLEDs, and blue light-emitting SLEDs that pulse sequentially at a total rate of 180 Hz or higher; iii) the time-series color system may be provided by an internal illuminator; iv) the active pixel sensor array may be an active pixel sensor array without a Bayer filter; v) the active pixel sensor array may sequentially capture red, green, and blue images.

[0008] This disclosure further provides a medical system including a processor, at least one active pixel sensor array coupled to the processor, a wavefront correction system coupled to the processor, an image reference system, and a storage medium coupled to the processor. The storage medium includes instructions that, when executed by the processor, cause the medical system to use the image reference system to determine the wavefront distortion of the reflected wavefront of light reflected from inside the patient's eye. The storage medium further includes instructions that, when executed by the processor, cause the medical system to use the wavefront correction system to correct the wavefront distortion of the reflected wavefront of light reflected from inside the eye.

[0009] This disclosure further provides a method for improving digital images by correcting wavefront distortion to determine the wavefront distortion of the reflected wavefront of light reflected from inside the eye using an image reference system; and a method for correcting wavefront distortion of the reflected wavefront of light reflected from inside the eye using a wavefront correction system. This disclosure also provides a method for improving digital images by illuminating the interior of the eye with a sequence of red, green, and blue light emitted at timed intervals using a time-series color system to eliminate lateral color diffusion and improve color reproduction; using at least one active pixel sensor array without a Bayer filter to detect the red, green, and blue wavefronts reflected from inside the eye; sequentially capturing red, green, and blue images of the interior of the eye using the active pixel sensor array without a Bayer filter; and integrating the red, green, and blue images into the viewer's visual cortex, or reformatting the red, green, and blue images for use in an organic light-emitting diode (OLED) display to provide a color image of the interior of the eye with lateral color diffusion eliminated.

[0010] Unless clearly mutually exclusive, various aspects of adaptive optics systems and their methods of application can be combined with each other. Furthermore, unless clearly mutually exclusive, additional features of the aforementioned adaptive optics systems and their associated methods can also be combined with each other. Attached Figure Description

[0011] To gain a more thorough understanding of this disclosure and its features and advantages, reference is now made to the following description taken in conjunction with accompanying drawings, which are not to scale, and in which the same reference numerals indicate the same features, and in these drawings:

[0012] Figure 1 This is a schematic diagram of an adaptive optics system, which includes a wavefront correction system, a time-series color system, and two monochrome active pixel sensor arrays.

[0013] Figure 2 This is a schematic diagram of a part of an adaptive optics system, which includes a wavefront correction system and a time-series color system;

[0014] Figure 3 This is a schematic diagram of an adaptive optics system that serves as a component of a visualization system;

[0015] Figure 4 Is as A schematic diagram of the adaptive optics system of a component of a 3D visualization system (Novartis AG, Switzerland);

[0016] Figure 5 This is a schematic diagram of an adaptive optics system that serves as a component of the camera head on a surgical microscope.

[0017] Figure 6 This is a schematic diagram of an adaptive optics system, which includes an image reference system.

[0018] Figure 7 This is a flowchart illustrating a method for correcting wavefront distortion to improve digital images used in vitreoretinal surgery;

[0019] Figure 8 This is a flowchart illustrating a method for eliminating lateral color diffusion in an adaptive optics system to improve digital images used in vitreoretinal surgery;

[0020] Figure 9 This is a schematic diagram of a computer system that includes an adaptive optics system.

[0021] Figures 10A to 10C This is a schematic diagram of a medical system, which includes an adaptive optics system; and

[0022] Figure 11 This is a schematic diagram of a medical system that includes an adaptive optics system, a surgeon, and a patient. Detailed Implementation

[0023] This disclosure provides systems and related methods, including adaptive optics, for improving digital images used in vitreoretinal surgery.

[0024] Vitreoretinal surgeons face unique challenges when visualizing the inside of the eye. For example, any view obtained through a patient's pupil is affected by optical aberrations. Optical aberrations can be caused by eye diseases or previous surgeries resulting in corneal asphericity or intraocular lens implants, leading to aberrations in the image seen by the surgeon. Spherical aberrations can be caused by pupillary dilation, strabismus, or the need to visualize the peripheral retina, cataracts, intraocular lenses, and corneal asphericity. Chromatic aberrations (which can be lateral or axial) can be caused by the eye's optical system or retinal visualization system's inability to focus different colors onto the same focal point or plane. Aberrations can interfere with the surgeon's ability to visualize the inside of the eye and make surgery more difficult. In analog systems, the methods for correcting aberration effects are very limited, and many methods are simply not correctable. However, digital visualization systems do allow for a variety of corrective measures, as described in this article, which can improve the image presented to the surgeon and others assisting in vitreoretinal surgery.

[0025] In particular, the systems and methods disclosed herein can reduce the effects of aberrations in digital images of the eye as seen by surgeons and others. By doing so, the systems and methods can reduce aberrations and improve digital image resolution, digital image color reproduction, digital image dynamic range, or any combination thereof during visualization of any aspect of vitreoretinal surgery. Compared to current systems and methods, the systems and methods of this disclosure can improve digital images for vitreoretinal surgery by including an adaptive optics system that can reduce spherical aberration or chromatic aberration, reduce aberrations arising from astigmatism, regular astigmatism, coma, or trilobal aberration caused by myopia and hyperopia, eliminate lateral color diffusion, allow region of interest (“ROI”) gain control, or any combination thereof.

[0026] Current systems and methods for digital visualization during vitreoretinal surgery do not include adaptive optics systems. Instead, they rely on dual-sensor high dynamic range camera systems, where aberrations can manifest as wavefront distortion, leading to image distortion and degraded image quality. Wavefront distortion in vitreoretinal surgery can also be caused by retinal visualization systems (e.g., macular or wide-angle contact lenses) or non-contact observation systems, such as... (OCULUS Surgical Company, USA) and (Carl Zeiss Medical Technology AG, Germany) or corneal asphericity, which may be caused by refractive keratotomy (PRK), laser-assisted in situ keratomileusis (LASIK), penetrating keratoplasty (PK), radial keratotomy (RK), limbal laxity incision (LRI), Descemet's membrane endothelial keratoplasty (DMEK), Descemet's membrane ablation endothelial keratoplasty (DSEK), anterior lamellar keratoplasty (ALK), inlay, trauma, keratoconus, or pterygium. Alternatively, wavefront distortion in vitreoretinal surgery may be caused by tilting due to a slanted viewing angle to see the peripheral retina, pupillary dilation, lens with or without cataracts, intraocular lens (especially multifocal or deep focal intraocular lenses), or corneal astigmatism in aphakic eyes. Current adaptive optics systems used in ophthalmic research are typically high-magnification, non-stereoscopic, monochromatic, non-real-time, have very small fields of view, require minimal subject movement, and are unsuitable for surgery. The adaptive optics system described herein, which improves digital imaging for vitreoretinal surgery, can utilize broadband white light, operate in real-time, display stereoscopic images, has no significant latency, and provides compatibility with current surgical microscopes and state-of-the-art digital-assisted vitreoretinal surgery (“DAVS”) systems. (Novartis AG, Switzerland) Same magnification.

[0027] The adaptive optics system described herein can improve digital images for vitreoretinal surgery by correcting wavefront aberrations using a wavefront correction system. The wavefront correction system can include a wavefront control structure that uses the corrected wavefront to control and correct wavefront aberrations that produce reflected light. The wavefront control structure can be a deformable mirror, wherein the surface of the mirror can deform to control and correct wavefront aberrations. The wavefront control structure can also be a spatial light modulator (“SLM”). An SLM can be a pixelation device that modulates the phase or polarization of a light wave. An SLM can be used for wavefront shaping by controlling the phase of the incident light at each pixel. Compared to deformable mirrors, SLMs offer significantly higher resolution, thereby enabling complex wavefront shaping, dot spread function engineering, and pixel engineering, thus reducing the effects of aberrations. The wavefront control structure can be a phase-only SLM. The wavefront control structure can also be a liquid crystal on silicon (“LCoS-SLM”), which can be a transmissive LCoS-SLM or a reflective LCoS-SLM. Wavefront shaping in SLMs can correct wavefront distortion and reduce the effects of aberrations (including spherical aberrations) to improve digital images.

[0028] In the adaptive optics system described herein, a wavefront correction system can be used in conjunction with a time-series color system. The time-series color system provides red, green, and blue light sources that emit sequences of red, green, and blue light at timed intervals. The red, green, and blue light sources can pulse sequentially at a total rate higher than 180 Hz to avoid color flickering or color splitting. The time-series color system provides red, green, and blue light and can illuminate the eye during surgery. The red, green, and blue light sources can be provided by an internal illuminator, which is a fiber optic probe inserted into the plana of the ciliary body. The wavefront correction system can correct wavefront distortions in the red, green, and blue light provided by the internal illuminator and reflected by the eye. Implementing both the wavefront correction system and the time-series color system within the same adaptive optics system allows for different wavefront shaping or transformations for each of the red, green, and blue light sources, which can reduce chromatic aberration in digital images by focusing red, green, and blue light to the same convergence point or image plane. This can reduce lateral chromatic aberration, axial chromatic aberration, or combinations thereof.

[0029] Alternatively, a time-series color system may utilize at least one light source, which is a color other than red, green, or blue. A time-series color system may also utilize at least three light sources, which may be colors of different wavelengths.

[0030] The adaptive optics system described herein for improving digital images used in vitreoretinal surgery can use image references to improve the digital image of the eye. The use of image references is similar to that of guide stars used in adaptive optics systems for astronomy. Typically, an image reference can be a reference object of known shape placed inside the eye. In the adaptive optics system described herein, surgical instruments can be used as image references. Surgical instruments can be vitrectomy cutters, forceps, scissors, PIC forceps, scrapers, bending rings, spatulas, MVR blades, microcannulas, or any combination thereof. If the shape of the surgical instrument is unknown before use, an aberration-free image of the instrument can be created outside the eye before it is placed inside. This allows the same surgical instrument to be used as an image reference when placed inside the eye during surgery. A wavefront correction system can be used to correct wavefront distortions in the view path (which can be reflected light from an internal illuminator) to restore the image of the image reference to its aberration-free appearance, which can be an aberration-free image of the surgical instrument. The light can be derived from a time-series color system provided by an internal illuminator. The processor can analyze wavefront distortion in the digital image of the reference image and send instructions to the wavefront correction system to control and correct the wavefront distortion, thereby improving the digital image of the reference image. The same instructions can be used to control and correct wavefront distortion in light reflected from the eye to improve the eye's digital image. In this example, a Shaker-Hartmann wavefront sensor may not be used.

[0031] The adaptive optics system described herein for improving digital images used in vitreoretinal surgery may include an active pixel sensor array, which is a light sensor array that detects light and transmits information for creating a digital image. The active pixel sensor array may be an array of light-capturing units, typically each unit representing a pixel. Pixels in the active pixel sensor array may detect only luminance information, not color information. A color filter, known as a Bayer filter, may be used to allow the active pixel sensor array to function as a color sensor. The Bayer filter applies a red, green, or blue filter to each pixel and may be placed in front of the active pixel sensor array. The Bayer filter allows for the calculation of correct luminance and color information for each pixel and can produce a color digital image. The adaptive optics system described herein may include a single active pixel sensor array. The active pixel sensor array may include a left channel and a right channel for providing a 3D stereoscopic image. Alternatively, the adaptive optics system may include two active pixel sensor arrays, or may include multiple active pixel sensor arrays. A pair of active pixel sensor arrays may also be used to provide a 3D stereoscopic image.

[0032] The active pixel sensor array in an adaptive optics system may include a Bayer filter, or it may be an active pixel sensor array without a Bayer filter. A Bayer filter can introduce lateral color diffusion into digital images used for vitreoretinal surgery, potentially leading to reduced color reproduction. In the adaptive optics system described herein, including a time-series color system allows the active pixel sensor array to sequentially capture red, green, and blue images. These red, green, and blue images can be integrated into the viewer's visual cortex to provide a color image for the eye. Alternatively, the red, green, and blue images can be electronically processed by an image processor and reformatted for use in OLED displays, digital eyepieces, or head-mounted displays. This eliminates the need for a Bayer filter on the active pixel sensor array used to acquire the color images. Therefore, the adaptive optics system described herein can use an active pixel sensor array without a Bayer filter but with a time-series color system for color visualization, which reduces lateral color diffusion on the active pixel sensor array and increases color reproduction in digital images used for vitreoretinal surgery.

[0033] The adaptive optics system disclosed herein for improving digital imaging in vitreoretinal surgery may further include an on-chip processor on an active pixel sensor array, which can implement region of interest (ROI) gain control to improve dynamic range. ROI gain control can also be implemented by including an additional amplitude-only SLM to improve dynamic range. The amplitude-only SLM can be positioned in the focal plane in front of the active pixel sensor array.

[0034] The systems and methods disclosed herein can provide any combination of advantages over those typically provided by general visualization sensors used in vitreoretinal surgery, including: (1) improving image resolution for digital images used in vitreoretinal surgery by using a wavefront correction system to correct wavefront distortion; (2) improving color reproduction for digital images used in vitreoretinal surgery by combining a time-series color system with an active pixel sensor array without a Bayer filter; and (3) improving dynamic range for digital images used in vitreoretinal surgery by using an active pixel sensor array with ROI gain control.

[0035] Now for reference Figure 1The adaptive optics system 100 includes a wavefront correction system 110, a time-series color system 120, and active pixel sensor arrays 130a and 130b. The adaptive optics system 100 can use the wavefront correction system 110 to correct wavefront distortion 140 to a corrected wavefront 150 to improve image resolution. The wavefront correction system 110 may include a wavefront control structure 111. The wavefront control structure 111 can control and correct the phase of the wavefront distortion 140. The wavefront control structure 111 can be an SLM, an LCoS-SLM, a reflective LCoS-SLM, a deformable mirror, or any combination thereof. The wavefront control structure 111 can also be a phase-only SLM that can correct the phase of the wavefront. The time-series color system 120 can be a light source for the adaptive optics system 100 and can be a red, green, and blue light source that emits a sequence of red, green, and blue light spaced at time intervals. The time-series color system 120 may include red light-emitting diodes (LEDs), green light-emitting LEDs, and blue light-emitting LEDs, which can pulse sequentially at a total rate higher than 180 Hz to avoid flicker. The LEDs may also be or include superluminescent diodes (SLEDs) that can support nanofibers illuminated tools, such as cannulas, cutters, tweezers, and scissors. The time-series color system 120 may be provided by an internal fiber optic illuminator probe, and red, green, and blue light may be provided through the pars plana of the eye for visualization. Wavefront distortion 140 of the red, green, and blue light provided by the internal illuminator and reflected by the eye can be corrected by a wavefront correction system 110. In this case, wavefront distortion 140 may be caused by the lens, intraocular lens, cornea, and retina visualization system of an eye with or without cataracts, after light from the internal illuminator has been reflected from the retina, sclera, choroid, vitreous body, scar tissue, epiretinal membrane, internal limiting membrane, hemorrhage, or surgical instruments. The wavefront correction system 110 can use different wavefront shaping for each of the three colors of light emitted by the time-series color system 120—red, green, and blue—which can reduce axial and lateral chromatic aberration in the adaptive optics system 100. The wavefront correction system 110 may also include a processor 180, on which instructions can be executed to generate a digital image in which at least one wavefront distortion in the light detected by the active pixel sensor arrays 130a and 130b is partially or completely corrected.

[0036] Active pixel sensor arrays 130a and 130b can detect white light, or each of red, green, and blue light emitted by the time-series color system 120, and can send signals to processor 180. In one example, processor 180 can receive one or more analog signals from one or more pixel sensor arrays 130a and 130b, etc. The one or more analog signals from one or more pixel sensor arrays 130a and 130b can be associated with detected light, which may be reflected from inside the eye. The analog signals may include information associated with one or more light frequencies. In another example, processor 180 can receive one or more digital signals from the time-series color system 120. One or more digital signals from one or more of the pixel sensor arrays 130a and 130b can be associated with detected light, which may be reflected from inside the eye. The digital signals may include information associated with one or more light frequencies.

[0037] Active pixel sensor arrays 130a and 130b can be light-harvesting unit arrays, typically each unit representing a pixel. Active pixel sensor arrays 130a and 130b can contain pixels with photodetectors and active amplifiers. The adaptive optics system 100 can include a single active pixel sensor array, can include two active pixel sensor arrays, or can include multiple active pixel sensor arrays. Light can pass through lens 160 and then be incident on active pixel sensor arrays 130a and 130b. Active pixel sensor arrays 130a and 130b can be monochrome active pixel sensor arrays, complementary metal-oxide-semiconductor (CMOS) sensors, or charge-coupled device (CCD) sensors. CMOS sensors can be single-chip 1080P CMOS sensors or 4K monochrome CMOS sensors. Single-chip CMOS color sensors typically use Bayer filters, which are color filter arrays that can cause lateral color diffusion. Active pixel sensor arrays 130a and 130b can include Bayer filters or can be active pixel sensor arrays without Bayer filters. The time-series color system 120 can eliminate the need for a Bayer filter by using active pixel sensor arrays 130a and 130b to provide a digital color image by allowing the sequential capture of red, green, and blue images. Using an active pixel sensor array without a Bayer filter eliminates lateral color diffusion and improves image color reproduction. Color aberration correction can also be achieved using an active pixel sensor array without a Bayer filter.

[0038] The amplitude-only SLM 170 can implement ROI gain control to improve dynamic range. The amplitude-only SLM 170 can be positioned in the focal plane preceding the active pixel sensor arrays 130a and 130b. Alternatively, the on-chip processor 135 on the active pixel sensor arrays 130a and 130b can implement region of interest (ROI) gain control to improve dynamic range. Processor 135 can be an additional processor to processor 180. Processor 135 and processor 180 can be physically separate processors in an adaptive optics system, or they can be part of the same processor performing several functions.

[0039] like Figure 2 As shown, the time-series color system 120 in the adaptive optics system 100 may include a red light source 200, a green light source 201, and a blue light source 202. Light sources 200, 201, and 202 may emit sequences of red, green, and blue light spaced at time intervals. Light sources 200, 201, and 202 may be LEDs and may pulse sequentially at a total rate higher than 180 Hz to avoid flickering or color splitting. Light sources 200, 201, and 202 may also be SLEDs. Light sources 200, 201, and 202 may be provided by an internal illuminator, and the light may be provided through the ciliary plane of the eye for visualization via an internal illuminator fiber optic probe. The red light source 200 may emit a red light wave 210, which may have a red wavefront distortion 220 after being reflected by the retina or surgical instruments. The green light source 201 may emit a green light wave 211, which may have a green wavefront distortion 221 after being reflected by the retina or surgical instruments. Blue light source 202 can emit blue light waves 212, which, after being reflected by the retina or surgical instruments, can have blue wavefront aberration 222. Wavefront correction system 110 can correct wavefront aberrations 220, 221, and 222 into corrected red wavefront 230, corrected green wavefront 231, and corrected blue wavefront 232 by performing corrective wavefront shaping. Wavefront shaping performed by wavefront correction system 110 can be different for each of the wavefront aberrations 220, 221, and 222, and axial and lateral chromatic aberration can be reduced by ensuring that the colors are focused to the same convergence point or image plane.

[0040] Figure 1 The adaptive optics system 100 shown can be as follows: Figure 3 The components of the vitreoretinal surgery visualization system 300 shown are for visualizing the eye 301 during surgery. The visualization system 300 may include a surgical microscope 310, an adaptive optics system 100 (which may include a processor 180), and a digital display 330, such as a screen, a heads-up display, a head-mounted display, or any combination thereof. The digital display 330 may also include multiple displays. The visualization system 300 may be a DAVS system, 3D Visualization System (Novartis AG, Switzerland) Figure 4 ), or a camera head mounted on a surgical microscope ( Figure 5 ).

[0041] The visualization system 300 may include a time-series color system 120 (which may be a light source for the adaptive optics system 100), and may be a red, green, and blue light source emitting a sequence of red, green, and blue light spaced apart over a period of time. The time-series color system 120 may be provided by an internal illuminator 305, and the red, green, and blue light may be provided through the ciliary plana of the eye 301 via an internal illuminator fiber optic probe 306. The time-series color system 120 may illuminate the eye 301 using sequentially pulsed red, green, and blue light sources. For example, the time-series color system 120 may illuminate the eye 301 using sequentially pulsed red, green, and blue light sources at a total rate higher than 180 Hz, which can be used to avoid flicker. Wavefront distortion 140 may be caused by the lens, cornea, and contact or non-contact retinal visualization system of an eye with or without cataracts after light from the internal illuminator has been reflected by the retina, sclera, choroid, vitreous body, scar tissue, hemorrhage, or surgical instruments. Wavefront aberration 140 present in the red, green, and blue wavefronts reflected from the eye 301 can be corrected by wavefront correction system 110 to reduce the effects of aberrations and improve the digital image resolution of the digital image of the eye 301 on digital display 330. Wavefront correction system 110 may include processor 180. Wavefront correction system 110 can execute instructions via processor 180 to generate a digital image. For example, wavefront correction system 110 can generate a digital image in which wavefront aberration 140 in the light detected by active pixel sensor arrays 130a and 130b is partially or completely corrected to a corrected wavefront 150.

[0042] Active pixel sensor arrays 130a and 130b can be light-harvesting unit arrays (typically each unit represents a pixel), can include a Bayer filter, or can be active pixel sensor arrays without a Bayer filter. Utilizing red, green, and blue light sources in the time-series color system 120 allows active pixel sensor arrays 130a and 130b to generate color images without a Bayer filter, which reduces lateral color diffusion. Lateral color diffusion in a color image generated without a Bayer filter is reduced compared to lateral color diffusion in a color image generated using a Bayer filter. This reduction in lateral color aberration improves color reproduction of the digital image of the eye 301 on the digital display 330. Visualization system 300 can include a single active pixel sensor array. Visualization system 300 can include multiple active pixel sensor arrays. Active pixel sensor arrays 130a and 130b can send signals to processor 180. ROI gain control can be implemented by active pixel sensor arrays 130a and 130b, and can be implemented by an on-chip processor or an additional amplitude-only SLM. Only the amplitude SLM can be positioned in the focal plane preceding the active pixel sensor arrays 130a and 130b (not shown). ROI gain control implemented by the active pixel sensor arrays 130a and 130b can improve the dynamic range of the digital image of the eye 301 on the digital display 330.

[0043] The processor 180 may include, for example, a field-programmable gate array (FPGA), a microprocessor, a microcontroller, a digital signal processor (DSP), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), or any other digital or analog circuit system configured to interpret and / or execute program instructions and / or process data.

[0044] Processor 180 may include any physical means capable of storing and / or executing instructions. Processor 180 may execute processor instructions to implement at least a portion of one or more systems, one or more flowcharts, one or more processes, and / or one or more methods described herein. For example, processor 180 may execute instructions to generate an image of eye 301. Processor 180 may be configured to receive instructions from a storage medium. In one example, processor 180 may include a storage medium. In another example, the storage medium may be external to processor 180. The storage medium may store instructions. Instructions stored by the storage medium may be executed by processor 180 and may be configured, encoded, decoded, and / or encoded using instructions representing at least a portion of one or more systems, one or more flowcharts, one or more methods, and / or one or more processes described herein.

[0045] An FPGA can be configured, encoded, and / or coded to implement at least a portion of one or more systems, flowcharts, processes, and / or methods described herein. For example, an FPGA can be configured, encoded, and / or coded to generate an image of eye 301. An ASIC can be configured to implement at least a portion of one or more systems, flowcharts, processes, and / or methods described herein. For example, an ASIC can be configured, encoded, and / or coded to generate an image of eye 301. A DSP can be configured, encoded, and / or coded to implement at least a portion of one or more systems, flowcharts, processes, and / or methods described herein. For example, a DSP can be configured, encoded, and / or coded to generate an image of eye 301.

[0046] Although processor 180 and active pixel sensor arrays 130a and 130b are depicted separately, a single device may include processor 180 and active pixel sensor arrays 130a and 130b. In one example, a single computer system may include processor 180 and active pixel sensor arrays 130a and 130b. In another example, the device may include an integrated circuit that may include processor 180 and active pixel sensor arrays 130a and 130b.

[0047] Processor 180 can interpret and / or execute program instructions and / or process data stored in a storage medium. The storage medium can be configured partially or entirely as application memory, system memory, or both. The storage medium can include any system, apparatus, or device configured to hold and / or accommodate one or more memory devices. Each memory device can include any system, module, or device (e.g., a computer-readable medium) configured to retain program instructions and / or data for a period of time. The described one or more servers, electronic devices, or other machines can include one or more similar processors or memories that can store and execute program instructions for implementing the functions of the associated machine.

[0048] The surgical microscope 310 can display an image of the eye 301, such as a digital image generated by the processor 180 or another processor. In addition to the image of the eye 301, the surgical microscope 310 can also display other information. This other information can be generated by the processor 180 or another processor and can include graphical or textual information, such as warnings, charts, color codes, surgical parameters, endoscopic video, optical coherence tomography (OCT) images, or augmented reality information.

[0049] Digital display 330 can similarly display digital images of eye 301 generated by processor 180 or another processor, as well as other information generated by processor 180 or another processor. This information may include graphical or textual information such as surgical parameters, surgical mode, flow rate, intraocular pressure, endoscopic video, OCT images, warnings, digital images, color-coded information, or augmented reality information. The information displayed on digital display 330 may not match the information displayed or seen on surgical microscope 310. Processor 180 can reformat video produced using time-series color system 120 as a light source for display on digital display 330, which can be viewed with circularly polarized glasses, digital eyepieces, or a head-mounted display.

[0050] The visualization system 300 may further include other components to facilitate its use, such as memory for storing images displayed on the digital display 330, electrical connections, and hardware for positioning and focusing any lens (such as lens 160) and positioning the active pixel sensor arrays 130a and 130b.

[0051] The adaptive optics system 100 can be used in the visualization system 400 The DSM-1 system (Novartis AG, Switzerland) is a component of which the adaptive optics system can replace, for example, Figure 4 The sensor portion is shown. For example, the adaptive optics system 100 may utilize an optomechanical focusing system 410, a zoom system 420, a variable working distance system 430, a display system 440, and a storage medium 450. In one example, the storage medium 450 may store data that can be used to implement at least a portion of one or more systems, one or more flowcharts, one or more processes, and / or one or more methods described herein. In another example, the storage medium 450 may store instructions executable by the processor 180 when implementing at least a portion of one or more systems, one or more flowcharts, one or more processes, and / or one or more methods described herein. As shown, the storage medium 450 may be communicatively coupled to the processor 180. As shown, the display 440 may be communicatively coupled to the processor 180.

[0052] The adaptive optics system 100 can be used as a component of the camera head 510 in the visualization system 500, which may further include, for example... Figure 5 The surgical microscope 520 shown is without an eyepiece. The visualization system 500 may also include a processor 180, and may further include a display system 540 and a storage medium 550. The camera head 510 may be... 1.0 (Novartis AG, Switzerland) Camera lens. In one example, storage medium 550 may store data that can be used to implement at least a portion of one or more systems, one or more flowcharts, one or more processes, and / or one or more methods described herein. In another example, storage medium 550 may store instructions executable by processor 180 in implementing at least a portion of one or more systems, one or more flowcharts, one or more processes, and / or one or more methods described herein. Figure 5 As shown, storage medium 550 can be communicatively coupled to processor 180. Display 540 can also be communicatively coupled to processor 180.

[0053] An adaptive optics system 100 for improving digital images used in vitreoretinal surgery may include, for example: Figure 6 The image reference system 600 is shown. (For example...) Figure 6 As shown, the image reference system 600 may include an image reference 605, which may be a reference object with a known shape. Image reference 605 may be an object placed inside the eye 301 or a surgical instrument. Image reference 605 may be a vitrectomy tool, forceps, scissors, PIC forceps, scraper, bending ring, spatula, MVR blade, microcannula, or other surgical instrument. If the shape of image reference 605 is unknown, it may also be placed outside the eye 301 in various positions before the start of surgery to obtain an aberration-free image of image reference 605. This allows image reference 605 to be used as an image reference when placed inside the eye 301 during surgery. Red, green, and blue light from the time-series color system 120, which may be provided by the internal illuminator 305, may be reflected from image reference 605, may have wavefront distortion 610, and may employ an optical path 630. Red, green, and blue light from the time-series color system 120 can also be reflected from the eye 301, can have wavefront distortion 620, and can employ an optical path 640. The processor 670 can analyze the wavefront distortion 610 in the digital image of the image reference 605 provided by the active pixel sensor arrays 130a and 130b, and send instructions to the wavefront correction system 110 to control and correct the wavefront distortion 610 to a corrected wavefront 680 to restore the aberration image of the image reference 605 to an aberration-free image. These same instructions can be used to correct the wavefront distortion 620 to a corrected wavefront 690. The corrected wavefront 690 can be sent to the active pixel sensor arrays 130a and 130b, allowing the digital image of the eye 301 to have improved resolution. Although processor 670 is depicted separately from processor 180 and active pixel sensor arrays 130a and 130b, they may be the same processor or part of a single physical device, such as a single computer or a set of integrated circuits.

[0054] Figure 7 A flowchart of a method for correcting wavefront distortion to improve a digital image according to this disclosure is presented. In step 700, light is used to illuminate the interior of the eye during surgery. The light may include a sequence of red, green, and blue light, such as a sequence of light emitted by a time-series color system 120 and provided by an internal illuminator 305. The light is reflected from the interior of the eye to produce a reflected wavefront of the light. In step 710, the wavefront distortion of the reflected light wavefront is determined. This step may involve the simultaneous use of an image reference system, such as image reference system 700. In step 701, light is used to illuminate an image reference during surgery. The light used to illuminate the image reference may include red, green, and blue light, such as light emitted by a time-series color system 120 and provided by an internal illuminator 305. In step 711, the wavefront distortion of the reflected wavefront of the light reflected from the image reference is determined. In step 720, a wavefront correction system (which may include a wavefront control structure, such as an SLM, a reflective LCoS-SLM, a transmissive LCoS-SLM, a deformable mirror, or any combination thereof) is used to correct wavefront distortion of the reflected wavefront of light reflected from inside the eye, thereby improving the image resolution of the digital image inside the eye.

[0055] Figure 8 A flowchart is presented for a method of eliminating lateral color diffusion and improving color reproduction to improve digital images used in vitreoretinal surgery. In step 800, red, green, and blue light (e.g., light emitted by a time-series color system 120) are emitted in a time-separated sequence to illuminate the interior of the eye during surgery. In step 810, the red, green, and blue light are reflected from the interior of the eye and can be detected by an active pixel sensor array. The active pixel sensor array can be an active pixel sensor array without a Bayer filter. In step 820, red, green, and blue images are sequentially captured by the active pixel sensor array without a Bayer filter. The red, green, and blue images can be integrated into the viewer's visual cortex or reformatted by an image processor for display on an OLED display or head-mounted display to produce a color image of the interior of the eye with lateral color diffusion eliminated. This method can improve color reproduction of digital images of the interior of the eye.

[0056] Figure 9 The image depicts a computer system 900. The computer system 900 may include a processor 910, a volatile storage medium 920, a non-volatile storage medium 930, and an input / output (I / O) device 940. The volatile storage medium 920, the non-volatile storage medium 930, and the I / O device 940 may be communicatively coupled to the processor 910.

[0057] The term "memory medium" can mean "memory," "storage device," "memory apparatus," "computer-readable medium," and / or "tangible computer-readable storage medium." For example, storage media can include, but is not limited to, the following: direct access storage devices (including hard disk drives), sequential access storage devices (such as magnetic tape drives), optical discs (CDs), random access memory (RAM), read-only memory (ROM), CD-ROMs, DVDs, electrically erasable programmable read-only memory (EEPROM), flash memory, non-volatile media, and / or any combination thereof. Figure 9 As shown, the non-volatile storage medium 930 may include processor instructions 932. Processor instructions 932 can be executed by processor 910. In one example, one or more portions of processor instructions 932 can be executed via non-volatile storage medium 930. In another example, one or more portions of processor instructions 932 can be executed via volatile storage medium 920. One or more portions of processor instructions 932 can be transferred to volatile storage medium 920.

[0058] Processor 910 can execute processor instructions 932 when implementing at least a portion of one or more systems, one or more flowcharts, one or more processes, and / or one or more methods described herein. For example, processor instructions 932 can be configured, encoded, and / or encoded using instructions according to at least a portion of one or more systems, one or more flowcharts, one or more methods, and / or one or more processes described herein. Although processor 910 is illustrated as a single processor, processor 910 may be or include multiple processors. One or more of the storage medium and memory medium can be software products, program products, and / or articles of manufacture. For example, software products, program products, and / or articles of manufacture can be configured, encoded, and / or encoded using instructions executable by a processor, according to a portion of one or more systems, one or more flowcharts, one or more methods, and / or one or more processes described herein.

[0059] Processor 910 may include any suitable system, apparatus, or device operable to interpret and execute program instructions stored in a storage medium and / or received via a network, process data, or both. Processor 910 may further include one or more microprocessors, microcontrollers, digital signal processors (DSPs), application-specific integrated circuits (ASICs), or other circuit systems configured to interpret and execute program instructions, process data, or both.

[0060] I / O device 940 may include any one or more tools that allow, permit, and / or enable a user to interact with computer system 900 and its associated components by facilitating user input and output to the user. Facilitating user input may allow a user to manipulate and / or control computer system 900, and facilitating user output may allow computer system 900 to instruct the computer system 900 on the effects of the user's manipulation and / or control. For example, I / O device 940 may allow a user to input data, instructions, or both into computer system 900 and otherwise manipulate and / or control computer system 900 and its associated components. I / O devices may include user interface devices such as a keyboard, mouse, touchscreen, joystick, handheld lens, tool tracker, coordinate input device, or any other I / O device suitable for use with the system.

[0061] I / O device 940 may include one or more buses, one or more serial devices, and / or one or more network interfaces, which may facilitate and / or enable processor 910 to implement at least a portion of one or more systems, processes, and / or methods described herein. In one example, I / O device 940 may include a storage interface that may facilitate and / or enable processor 910 to communicate with external memory. The storage interface may include one or more of Universal Serial Bus (USB) interfaces, SATA (Serial ATA) interfaces, PATA (Parallel ATA) interfaces, and Small Computer System Interface (SCSI). In a second example, I / O device 940 may include a network interface that may facilitate and / or enable processor 910 to communicate with a network. I / O device 940 may include one or more of wireless network interfaces and wired network interfaces. In a third example, I / O device 940 may include one or more of Peripheral Component Interconnect (PCI) interfaces, PCI Express (PCIe) interfaces, Serial Peripheral Interconnect (SPI) interfaces, and Internal Integrated Circuit (I2C) interfaces. In the fourth example, the I / O device 940 may include circuitry that allows the processor 910 to communicate data with one or more sensors. In the fifth example, the I / O device 940 may facilitate and / or allow the processor 910 to communicate data with one or more of the following: a display 950 or an adaptive optics system 100. Figure 9 As shown, I / O device 940 can be coupled to network 970. For example, I / O device 940 may include a network interface.

[0062] Network 970 may include a wired network, a wireless network, an optical network, or any combination thereof. Network 970 may include and / or be coupled to various types of communication networks. For example, network 970 may include and / or be coupled to a local area network (LAN), a wide area network (WAN), the Internet, the public switched telephone network (PSTN), a cellular telephone network, a satellite telephone network, or any combination thereof. A WAN may include a private WAN, a corporate WAN, a public WAN, or any combination thereof.

[0063] although Figure 9 The computer system 900 is illustrated as being external to the adaptive optics system 100, but the adaptive optics system 100 may include the computer system 900. For example, the processor 910 may be or include the processor 180.

[0064] Figures 10A to 10C The illustration shows an example of a medical system 1000. (Example:) Figure 10A As shown, the medical system 1000 may include an adaptive optics system 100. For example... Figure 10B As shown, the medical system 1000 may include an adaptive optics system 100 and a computer system 900. The adaptive optics system 100 may be communicatively coupled to the computer system 900. Figure 10C As shown, the medical system 1000 may include an adaptive optics system 100, which may include a computer system 900.

[0065] The adaptive optics system 100 can be used as a component of the medical system 1100, such as... Figure 11As shown. Medical system 1100 may include adaptive optics system 100. Medical system 1100 may include computer system 900. Surgeon 1110 may view a digital image of patient 1120's eye 1301 on a microscope integrated display (MID) 1130, display 1150, or any combination thereof. MID 1130, display 1150, or any combination thereof may display an image of eye 1301 in which at least one wavefront of light reflected from inside eye 1301 is fully or partially corrected. Compared to a digital image of the eye captured without adaptive optics system 100, the digital image of eye 1301 may have improved image resolution, improved image color reproduction, improved image dynamic range, or any combination thereof. Medical system 1100 may include a processor; at least one active pixel sensor array coupled to the processor; a wavefront correction system coupled to the processor; an image reference system; and a storage medium, such as the storage medium in adaptive optics system 100 or image reference system 600. The storage medium may be coupled to a processor and may include instructions that, when executed by the processor, cause the medical system to use an image reference system to determine the wavefront distortion of the reflected wavefront of light reflected from inside the eye 1301 of the patient 1120. The storage medium may also include instructions that, when executed by the processor, cause the medical system to use a wavefront correction system to correct the wavefront distortion of the reflected wavefront of light reflected from inside the eye 1301.

[0066] Unless explicitly mutually exclusive, the adaptive optics system 100, visualization system 300, visualization system 400, visualization system 500, image reference system 600, computer system 900, and medical system 1000, medical system 1100, and their components can be combined with other elements of the visualization tools and systems described herein. For example, the image reference and processor in image reference system 600 can be used with other visualization systems described herein.

[0067] The subject matter disclosed above should be considered illustrative rather than restrictive, and the appended claims are intended to cover all such modifications, enhancements, and other embodiments falling within the true spirit and scope of this disclosure. For example, while adaptive optics systems are most commonly needed during vitreoretinal surgery to improve digital images, they can also be employed using the systems and methods described herein if they are useful in another procedure (such as a purely diagnostic procedure not otherwise considered surgery).

Claims

1. An adaptive optics system, comprising: A time-series color system, comprising a red light source, a green light source, and a blue light source, wherein the red light source, green light source, and blue light source are operable to emit a sequence of red light, green light, and blue light at time intervals. At least one active pixel sensor array, the active pixel sensor array being operable to detect light and send a signal to a processor, wherein the at least one active pixel sensor array is operable to detect each of red light, green light and blue light, and send a signal to the processor; Image reference systems, including image references; and A wavefront correction system, comprising at least one wavefront control structure and the processor, and capable of operating to: Instructions are executed on the processor to: Correction of wavefront distortion of the reflected wavefront of light reflected from inside the patient's eye, determined using the image reference system, and A digital image is generated in which at least one wavefront distortion in the light detected by the active pixel sensor array is partially or completely corrected.

2. The adaptive optics system of claim 1, further comprising a plurality of active pixel sensor arrays.

3. The adaptive optics system as described in claim 1, wherein, The wavefront control structure includes a spatial light modulator (SLM), a deformable mirror, or any combination thereof.

4. The adaptive optics system as described in claim 3, wherein, The spatial light modulator (SLM) includes a liquid crystal on silicon (LCoS) spatial light modulator (SLM).

5. The adaptive optics system as described in claim 4, wherein, The liquid crystal on silicon spatial light modulator (LCoS-SLM) includes transmissive LCoS-SLM, reflective LCoS-SLM, or any combination thereof.

6. The adaptive optics system as claimed in claim 1, wherein, The wavefront control structure includes a phase-only SLM.

7. The adaptive optics system as claimed in claim 1, wherein, The digital image is displayed on a digital display, screen, head-up display, head-mounted display, or any combination thereof.

8. The adaptive optics system as claimed in claim 1, wherein, The active pixel sensor array is a complementary metal-oxide-semiconductor CMOS monochrome sensor.

9. The adaptive optics system as claimed in claim 8, wherein, The complementary metal-oxide-semiconductor CMOS monochrome sensor includes a 4K monochrome CMOS sensor, a 1080P monochrome CMOS sensor, or any combination thereof.

10. The adaptive optics system of claim 1, further comprising an on-chip processor on the active pixel sensor array, the on-chip processor being operable to implement region of interest (ROI) gain control.

11. The adaptive optics system of claim 1, further comprising an amplitude-only SLM for implementing ROI gain control on the active pixel sensor array.

12. The adaptive optics system of claim 1, wherein, The image reference includes surgical tools placed in the eye.

13. The adaptive optics system of claim 12, wherein, The surgical instruments are vitrectomy tools, forceps, scissors, scrapers, curved rings, spatulas, miniature vitreoretinal (MVR) blades, miniature cannulas, or any combination thereof.

14. The adaptive optics system of claim 1, wherein, The time-series color system includes red light-emitting diodes (LEDs), green LEDs, and blue LEDs that pulse sequentially at a total rate of 180 Hz or higher.

15. The adaptive optics system as claimed in claim 1, wherein, The time-series color system includes red superluminescent diodes (SLEDs), green SLEDs, and blue SLEDs that pulse sequentially at a total rate of 180 Hz or higher.

16. The adaptive optics system of claim 1, wherein, The time-series color system is provided by an internal illuminator.

17. The adaptive optics system of claim 1, wherein, The active pixel sensor array is an active pixel sensor array without a Bayer filter.

18. The adaptive optics system of claim 1, wherein, The active pixel sensor array sequentially captures red, green, and blue images.

19. The adaptive optics system of claim 12, wherein, The surgical tool used was a PIC forceps.

20. A medical system comprising: processor; A time-series color system, comprising a red light source, a green light source, and a blue light source, wherein the red light source, green light source, and blue light source are operable to emit a sequence of red light, green light, and blue light at time intervals. At least one active pixel sensor array coupled to a processor, wherein the at least one active pixel sensor array is operable to detect each of red light, green light and blue light, and send a signal to the processor; A wavefront correction system coupled to the processor; Image reference systems, including image references; and A storage medium coupled to the processor and including instructions that, when executed by the processor, cause the medical system to: The image reference system is used to determine the wavefront distortion of the reflected wavefront of light from inside the patient's eye; and The wavefront correction system is used to correct wavefront distortion of the reflected wavefront of light reflected from inside the eye.

21. A method for correcting wavefront distortion to improve a digital image, the method comprising: A time-series color system is used to emit a sequence of red, green, and blue light at time intervals to illuminate the inside of the eye; Use at least one active pixel sensor array to detect each of the red, green, and blue light reflected from inside the eye; Use an image reference system that includes an image reference to determine the wavefront distortion of the reflected wavefront of light reflected from inside the eye; as well as A wavefront correction system is used to correct wavefront distortion of the reflected wavefront of light reflected from inside the eye.

22. A method for eliminating lateral color diffusion and improving color reproduction to improve a digital image, the method comprising: A time-series color system is used to emit a sequence of red, green, and blue light at timed intervals to illuminate the inside of the eye; At least one active pixel sensor array without a Bayer filter is used to detect the red, green, and blue light wavefronts reflected from inside the eye. The active pixel sensor array without Bayer filters is used to sequentially capture red, green, and blue images of the inside of the eye; as well as The red, green, and blue images can be integrated into the viewer's visual cortex, or the red, green, and blue images can be reformatted for use in an organic light-emitting diode (OLED) display to provide a color image inside the eye that eliminates lateral color diffusion. The method further includes: The wavefront distortion of the reflected wavefront of light reflected from inside the eye is determined using an image reference system that includes an image reference. as well as A wavefront correction system is used to correct wavefront distortion of the reflected wavefront of light reflected from inside the eye.

Images