Intraoperative visualization, measurement and assistance for ophthalmic treatment

Abstract

A system for performing ophthalmic treatment includes a surgical microscope configured to capture a surface image of an eye of a patient; and an imaging device mounted on the surgical microscope and configured to capture a tomographic image of an eye of a patient. A controller is coupled with the surgical microscope and the imaging device, the controller configured to receive the surface image and the tomographic image and to provide feedback based on the tomographic image and the surface image to facilitate an ophthalmic treatment. Feedback may involve avoiding capsular bag rupture during phacoemulsification, safely performing retinal membrane dissection, IOL implantation, treating glaucoma, or performing other ophthalmic treatments.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Provisional Application No. 63 / 578,360, filed August 23, 2023, which is incorporated herein by reference in its entirety. Background Technology

[0003] This disclosure generally pertains to performing eye surgeries.

[0004] Light received by the eye is focused by the cornea and lens onto the retina at the back of the eye, which contains photoreceptor cells. The area between the cornea and lens is called the anterior segment. The inner part of the eye between the lens and retina is called the posterior segment and is filled with a transparent gel called the vitreous humor. Many eye diseases can be treated by ophthalmic treatment of the inner or posterior segment.

[0005] This will be an advancement in the field, facilitating ophthalmic treatment. Summary of the Invention

[0006] In some embodiments, a system for performing ophthalmic treatment includes: a surgical microscope configured to capture surface images of a patient's eye; and an imaging device mounted on the surgical microscope and configured to capture tomographic images of the patient's eye. A controller is coupled to the surgical microscope and the imaging device, and is configured to receive the surface images and tomographic images and provide feedback based on the tomographic and surface images to facilitate ophthalmic treatment. Attached Figure Description

[0007] To gain a detailed understanding of the features described above, reference can be made to the embodiments for a more specific description of the briefly summarized disclosure, some of which are illustrated in the accompanying drawings. However, it should be noted that the drawings only illustrate exemplary embodiments and should not be construed as limiting the scope of the disclosure, and other equally effective embodiments are permissible.

[0008] Figure 1A The surgical microscope used by surgeons during ophthalmic treatments is shown.

[0009] Figure 1B A surgical microscope equipped with an accessory imaging device according to an embodiment of the present invention is shown.

[0010] Figure 1C An accessory imaging device for use with a gonioscope, according to certain embodiments, is shown.

[0011] Figure 1D An accessory imaging device adjustable relative to a surgical microscope, according to certain embodiments, is shown.

[0012] Figure 2 An eye receiving glaucoma treatment according to certain embodiments is shown.

[0013] Figure 3 This is a flowchart of a method for providing intraoperative assistance during glaucoma treatment according to certain embodiments.

[0014] Figure 4A and Figure 4B Examples of visual assistance displayed during glaucoma treatment according to certain embodiments are shown.

[0015] Figure 5 A retinal receiving visually assisted membrane peeling treatment is shown according to certain embodiments.

[0016] Figures 6A to 6C This is a process flowchart of a method for providing visual assistance during membrane peeling treatment, according to certain embodiments.

[0017] Figure 2 Figure A illustrates ophthalmic treatment using surgical instruments and imaging devices according to certain embodiments.

[0018] Figure 7A and Figure 7B Cataract treatments were demonstrated, including phacoemulsification and intraocular lens implantation.

[0019] Figures 8A to 8F This is a process flowchart of a method for providing assistance during cataract treatment, according to certain embodiments. Detailed Implementation

[0020] refer to Figure 1A The surgical operating environment 100 can be used by surgeon 106 to perform ophthalmic treatments on the eye 102 of patient 104. The surgical operating environment 100 may include a surgical microscope 108 suspended on a support 110, which helps to position the surgical microscope 108 above the eye 102 at a height desired by surgeon 106. For example, the surgical microscope 108 may be implemented as the NGENUITY 3D visualization system provided by Alcon Inc., Fort Worth, Texas.

[0021] refer to Figure 1BImaging device 112 is mounted on surgical microscope 108 and provides imaging according to one or both of the following: (a) an imaging mode different from that of surgical microscope 108, and (b) a different viewing angle or zoom magnification than that of surgical microscope 108, such as a visible light camera having a wider viewing angle than surgical microscope 108. One or more additional accessory devices 114 may also be mounted on surgical microscope 108 to facilitate ophthalmic treatment. Accessory device 114 may be another imaging device with the same or different imaging mode as imaging device 112. Accessory device 114 may also be a sensor other than an imaging device, such as an intraocular pressure (IOP) sensor (e.g., a contact or non-contact intraocular pressure measurement (NCT) IOP sensor) or other types of sensors. Accessory device 114 may also be a light source for illuminating eye 102. Examples of imaging methods used to implement the surgical microscope 108, imaging device 112, and accessory device 114 may include a visible light camera, an infrared camera, a fundus autofluorescence (FAF) camera, a multispectral imaging (MSI) camera, a hyperspectral imaging (HSI) camera, a wide-angle observation (WAV) camera, an optical coherence tomography (OCT) imaging device, or a scanning laser ophthalmoscope (SLO). Accessory device 114 may be a laser or acoustic rangefinder or an imaging device.

[0022] In the illustrated embodiment, mounting ring 116 is secured to surgical microscope 108, such as around the objective lens and / or optical axis of surgical microscope 108. Imaging device 112 and accessory device 114 are secured to mounting ring 116, either on opposite sides of ring 116 (e.g., 180 degrees apart around the center of ring 116) or at other locations. The mounting ring provides mounting points where any of the imaging device 112 and accessory device 114 listed above can be detachably mounted, including during surgery, to replace one imaging device and / or accessory device 114 with another different imaging device 112 and / or accessory device 114.

[0023] Images received from or derived from the surgical microscope 108, imaging device, and possible accessory devices 114 may be displayed in (a) a display device (e.g., a stereoscopic display device) within the surgical microscope 108 or in (b) an external display device 118 (e.g., a monitor, projector, or other display device).

[0024] refer to Figure 1CIn some embodiments, such as during ophthalmic treatment of glaucoma, the surgical microscope 108 can be used in conjunction with a gonioscope 120. The gonioscope 120 can be implemented using a goniometric prism or a gonioscope. A portion of the image transmitted through the gonioscope 120 can be mirrored. Therefore, a portion of the image captured by the surgical microscope 108 and received through the gonioscope 120 can be flipped to correspond to the actual orientation of the eye 102, for example, using the method described in U.S. Patent 10,201,270, which is incorporated herein by reference in its entirety.

[0025] refer to Figure 1D In some embodiments, one or both of the imaging device 112 and the accessory device 114 may be mounted on the surgical microscope via an adjustable support 122. The adjustable support 122 may facilitate adjusting the position of one or both of the imaging device 112 and the accessory device 114 substantially (e.g., within 2 degrees) parallel to the optical axis of the surgical microscope 108. The adjustable support 122 may also be adjustable in one or more other dimensions perpendicular to the optical axis of the surgical microscope 108. Control for adjusting the position of the imaging device 112 and / or the accessory device 114 may be manual or actuated. When actuated, the interface for adjusting the adjustable support 122 may include physical buttons mounted on the surgical microscope 108, voice commands, gesture controls, a touchscreen, or other interfaces.

[0026] Figure 2 and Figure 3 This demonstrates the use of surgical operating environment 100 for the treatment of glaucoma. See details for further information. Figure 2 Glaucoma treatment is typically performed in the anterior segment 200 of the eye 102, located behind the transparent, spherical cornea 202 through which light enters the eye 102. The iris 204 is a ring of muscles that defines the pupil of the eye, through which light enters. The lens 206, located behind the pupil, works with the cornea 202 to focus light onto the photoreceptor cells of the retina 208. The retina 208 is formed inside the eyeball 210, opposite the anterior segment 200. The eyeball 210, located between the lens 206 and the retina 208, is filled with a transparent gel-like substance called the vitreous humor 212.

[0027] The ciliary body 214 includes ligaments and muscles that connect the iris 204 and lens 206 of the eye to the choroid 216. The muscles of the ciliary body 214 are responsible for changing the shape of the lens 206. The choroid 216 is a vascularized layer lining the eyeball 210.

[0028] The ciliary body 214 produces aqueous humor, which is the fluid occupying the anterior segment 200. Aqueous humor flushes the lens 206 and iris 204 and flows to the periphery of the anterior segment 200. The periphery of the anterior segment includes structures that allow aqueous humor outflow when functioning normally. These structures include the trabecular meshwork 218 and the scleral venous sinus 220. The trabecular meshwork 218 appears to act as a filter, restricting aqueous humor outflow and generating back pressure directly related to IOP. The scleral venous sinus 220 extends beyond the trabecular meshwork 218. The scleral venous sinus 220 is fluidly connected to a collecting channel (not shown) that allows aqueous humor to flow out of the anterior segment 200.

[0029] Glaucoma can be treated by inserting the rod 222, for example, into the anterior segment 200 through an incision at the limbus 224, the boundary between the cornea 202 and the sclera (white) of the eye. The rod 222 is then used to make incisions in one or more structures around the anterior segment 200 and may be used to place a stent to facilitate drainage of aqueous humor. For example, an incision or stent may be made in the trabecular meshwork 218 to facilitate drainage into the scleral venous sinus 220. In other methods, the stent extends from the anterior segment into the suprachoroidal space between the choroid 216 and the eyeball 210.

[0030] For details, please refer to the following: Figure 3 The method 300 shown can be implemented using the surgical operating environment 100 to facilitate glaucoma treatment. Method 300 can be performed by receiving images from the surgical microscope 108 and the imaging device 112, as well as any outputs of the accessory device 114 (when the accessory device 114 is present and in use), via a computing device.

[0031] Method 300 includes capturing one or more surface images of eye 102 in step 302 and capturing one or more tomographic images of eye 102 in step 304. As used herein, a “surface image” means an image that captures light reflected from the surface of eye 102 and / or light transmitted and reflected through one or more transparent structures of the eye, including cornea 202 and lens 206. Surface images can be visible light images, multispectral or hyperspectral images, infrared images, or other types of images. Surface images can be one of two or more images providing a stereoscopic view of eye 102. As used herein, a “tomographic image” means an image containing a cross-section of the tissue of eye 102, including tissue at depths not visible in surface images. In a tomographic image, the depth within the tissue of the eye represented by the pixels in the image is known, while a surface image can flatten light reflected from different depths within the tissue of eye 102 into a single image. Tomographic images can be composed of multiple cross-sectional images to form a three-dimensional image. Tomographic images can be three-dimensional images that can be viewed along multiple different tomographic planes. In some embodiments, the tomographic image is an OCT image. The tomographic image captured in step 304 can constitute a three-dimensional image of at least a portion of the eye 102 (e.g., the anterior segment 200). The surface image and the tomographic image can be registered to each other, that is, pixels representing anatomical structures in the surface image can be mapped to pixels (or voxels) in the three-dimensional image corresponding to the same anatomical structure.

[0032] When using a gonioscope 120, method 300 may include step 306 reversing the image received through the gonioscope 120 to undo the mirroring applied by the gonioscope 120 (e.g., an image received from a surgical microscope 108). Step 306 may be omitted when the gonioscope 120 is not used.

[0033] Method 300 includes identifying anatomical structures within a three-dimensional image and one or more surface images in step 308. Identifying anatomical structures may include processing one or both of the three-dimensional image and one or more surface images using a machine learning model. For example, for each anatomical structure item of the eye to be identified, training data entries may be created, including the three-dimensional image and one or more surface images, and labels indicating the portion of the three-dimensional image and one or more surface images corresponding to the anatomical structure item. Then, in step 308, the machine learning model may be trained using the training data entries to identify the anatomical structure item. Multiple machine learning models may exist, each trained to identify one or more different anatomical structure items.

[0034] Method 300 may include identifying one or more sites for setting up an incision or placing a stent based on anatomical structures in step 310. For example, these sites may be selected to set up an incision or place a stent into the scleral venous sinus 220. Thus, the site selected in step 310 may be located on the trabecular meshwork 218 above the scleral venous sinus 220. A drainage channel guides fluid away from the scleral venous sinus. Therefore, the insertion site may also be selected near the drainage channel, for example, within 0.5 mm of the drainage channel. Identifying one or more sites may include identifying sites with a numbering and distribution (e.g., minimum spacing between sites) specified in the treatment protocol. Sites may be identified in the trabecular meshwork 218, the filtering bleb, or other locations 102 of the eye. Step 310 may further include identifying a vector for each insertion site. The vector may specify the direction in which the incision is to be formed or the stent insertion site is to be positioned, such as to extend into the scleral venous sinus or to have a desired relationship with other anatomical structures of the eye 102.

[0035] The method may include, in step 312, superimposing one or more representations of one or more sites onto an image (such as the surface image in step 302, the tomographic plane in step 304, a rendering of a three-dimensional image, or some other image). Step 312 may further include superimposing a representation of each vector identified in step 310 onto the image. Then, in step 314, the image superimposed with the representations of sites and / or vectors may be displayed on a display device 118, the display of the surgical microscope 108 (e.g., a stereoscopic display), or elsewhere.

[0036] For example, refer to Figure 4A The displayed image can be labeled with location markers 400 (representing the location identified in step 310) and vector markers 402 (representing the vector identified in step 310). Other anatomical structures can be labeled, such as marker 404a for marking pigmented trabecular meshwork, which is typically the selected location for setting incisions or inserting stents. One or more markers 404b can mark the location of drainage channels.

[0037] Refer again Figure 3 Method 300 may further include detecting incision formation or stent placement in step 316. Step 316 may include detecting movement of the detection rod 222, detecting changes in the trabecular mesh in a three-dimensional image and / or surface image captured after incision formation or stent placement, detecting markers or otherwise detecting the stent in the three-dimensional image, or employing other methods.

[0038] Method 300 may include detecting fluid flow through the incision or stent in step 318. Step 318 may additionally or alternatively include detecting the IOP of the eye 102. The fluid flow may be detected using the accessory imaging device 112. For example, the velocity of the fluid flow through the incision and / or stent can be obtained by detecting the red / blue shift in the reflected light using any method known in the art. The velocity of the fluid flow may be measured in a specific area, such as in the incision or stent region. Dye may be injected into the anterior segment to aid in the visualization of the fluid flow. Fluid flow conditions may be inferred by detecting changes or rates of change in the IOP sensed after the incision is formed, such as when the accessory device 114 is an IOP sensor.

[0039] Method 300 may include detecting the degree of dilation of the scleral venous sinus 220 in step 320. For example, the scleral venous sinus 220 may be identified in a first three-dimensional image captured prior to incision formation and / or stent placement. The scleral venous sinus 220 may then be identified in one or more second three-dimensional images captured after incision formation and / or stent placement. The size of the representation of the scleral venous sinus 220 in the first three-dimensional image and one or more second three-dimensional images may then be calculated, such as the number of voxels representing the portion identified as a scleral venous sinus. Therefore, the degree of dilation may be calculated as the ratio of the number of voxels representing the scleral venous sinus 220 in the first three-dimensional image to the number of voxels representing the scleral venous sinus 220 in the second three-dimensional image.

[0040] Method 300 may include overlaying a drainage indication onto an image of the eye 102 (e.g., a surface image of the eye 102 captured before or after incision formation or stent placement) in step 322. For example, as... Figure 4B As shown, drainage indicators may include symbols, such as the arrow 406 shown. The attributes of the symbol can indicate the degree of drainage, such as size (e.g., larger indicates better drainage) and color (green = adequate drainage improvement, yellow = insufficient drainage improvement, red = no significant drainage improvement). The attributes of the drainage indicator can be a function of part or all of the fluid flow rate detected in step 318 and the degree of dilation of the scleral venous sinus 220 detected in step 320. Step 322 may additionally include overlaying a representation 408 of the incision and / or stent on the image at the location corresponding to the location detected in step 316, with site markings indicating the intended location of the incision and / or stent placement, which may or may not be displayed.

[0041] refer to Figure 5The surgical environment 100 can be used to facilitate the peeling of membranes, such as the internal limiting membrane (ILM) or the epiretinal membrane (ERM), from the retina 208. The membrane can be peeled using an instrument 500 inserted into the eye, which has forceps 502 that extend from the instrument 500 and are actuated to grasp the membrane. The peeling treatment may include the membrane within a peeling boundary 504 (e.g., above the macula of the eye 102). The peeling treatment may include completing all the membrane within the peeling boundary 504 in one step, or it may be performed in sections. For example, the peeling area 506 may be peeled in a first grasping and peeling step, with the remainder peeled in one or more additional grasping and peeling steps.

[0042] Figure 6A , Figure 6B and Figure 6C This method utilizes the surgical operating environment 100 to facilitate dissection treatment. During dissection treatment, the accessory device 114 can be implemented as a light source. Multiple parameters of the light source (such as color and intensity) are controllable. The light source can be a multispectral or hyperspectral light source, such that the multiple parameters include the light intensity within each of three, four, five, or more wavelength bands.

[0043] For details, please refer to the following: Figure 6A Method 600a may include capturing one or more surface images of the retina 208 in step 602 and capturing one or more tomographic images of the retina 208 in step 604. One or more surface images may be captured by individually illuminating the retina 208 with the light source of the surgical microscope 108 or by providing light according to initial values ​​of multiple parameters via the accessory device 114.

[0044] Method 600a may include evaluating, in step 606, characteristics of the membrane representation in one or more surface images and one or more tomographic images. The characteristics of the membrane representation may include image quality metrics corresponding to whether the surgeon can clearly see the membrane during the dissection procedure. Image quality metrics may include values ​​for sharpness, contrast, saturation, or other image quality metrics. The characteristics of the membrane representation may be the output of a machine learning model. For example, each of a plurality of training data entries may include one or more images captured during a previous dissection procedure as input, and one or more manually assigned quality metrics of the membrane representation in the one or more images. The machine learning model can then be trained using the plurality of training data entries to output one or more image quality metrics of the membrane representation in the input image for a given input image. Alternatively, a machine vision algorithm may be configured to output one or more image quality metrics in a similar manner.

[0045] Method 600a may include selecting values ​​for multiple parameters in step 608 based on the evaluation results of step 606. The values ​​for the multiple parameters may be selected based on one or more image quality metrics obtained in step 606. The values ​​for the multiple parameters may include applying a predefined algorithm that converts one or more metrics into corresponding values ​​for the multiple parameters, the predefined algorithm being configured to select values ​​for the multiple parameters that will improve one or more metrics, i.e., enable better visualization of the retina 208 and the membrane to be removed in subsequent surface images. Alternatively or additionally, step 608 may include a search algorithm in which the retina 208 is illuminated with light generated according to a set of values ​​for the multiple parameters, a surface image of the retina 208 is captured, and one or more quality metrics of the surface image are calculated. Multiple sets of values ​​in the search space can be tested in this way, and then the set of values ​​that achieves the best one or more image quality metrics can be selected.

[0046] Figure 6B A method 600b for identifying regions covered by membranes on the retina 208 is demonstrated. Method 600b is particularly suitable for identifying pathological membranes, such as ILMs, which may have irregular shapes and locations. The region of an ILM to be removed from the retina 208 can be easily identified based on the anatomical structure of the retina, i.e., a circle with a known radius centered on the fovea of ​​the retina 208, which is identifiable due to its high pigmentation and lack of vascularization. However, in some embodiments, method 600b can also be used to identify ILMs.

[0047] Method 600b may include capturing one or more surface images and one or more tomographic images as described in steps 610 and 612. Method 600b includes evaluating the reflectance of different regions of the retina in the one or more surface images and possibly the one or more tomographic images in step 614. Step 614 may include evaluating changes in reflectance within individual wavelength bands.

[0048] Method 600b may include identifying a representation of the membrane in one or more surface images and possibly one or more tomographic images in step 616. Step 616 may include identifying the membrane based on changes in reflectance evaluated in step 614 (e.g., changes in reflectance indicating membrane boundaries).

[0049] Method 600b may include superimposing a film indication onto an image (such as one or more surface images) in step 618. For example, as Figure 5As shown, the membrane boundary 504 can be represented as a line, a shaded area, or other visual indicator. Method 600b can be repeated during the dissection procedure so that areas 506 without membrane dissection are also represented in the image, for example, by the absence of a membrane indicator or by being outside the line indicating the current membrane boundary 504. In some embodiments, numbers, text, or other indicators indicate the amount of membrane that has been dissected and / or is still to be dissected. The image overlaid with the membrane indicators can be displayed on the display device 118, the internal display of the surgical microscope 108, or other display devices for the surgeon to view.

[0050] like Figure 6C As shown, the surgical operating environment 100 can be used to perform the illustrated method 600c to provide feedback during the dissection treatment. Method 600c may include: capturing one or more surface images in step 620, capturing one or more tomographic images in step 622, and identifying anatomical structures in step 624. Steps 620, 622, and 624 can be performed according to any of the methods described above.

[0051] Method 600c may further include identifying, in step 626, the position and possible orientation of surgical instruments (such as instrument 500 and forceps 502) in one or more surface images and one or more tomographic images. For example, the position and orientation of the surgical instruments in three dimensions can be determined by a three-dimensional image formed from one or more tomographic images.

[0052] Method 600c may further include evaluating the film reflectivity in step 628. Specifically, the surface image and the changes in the film's reflectivity within the corresponding portion of the surface image can be evaluated. For example, the change in reflectivity in the region surrounding the location of the tweezers 502. Reflectivity changes when the film deforms. Therefore, the change in reflectivity can be used to characterize the film's deformation in step 630. Step 630 may include applying a predefined function, algorithm, or machine learning model to convert the change in reflectivity into a description of the deformation characteristics. The deformation can be converted into a force applied by the tweezers 502, or the reflectivity can be directly converted into an estimate of the force applied by the tweezers 502.

[0053] Method 600c may include providing feedback to the surgeon in step 632 regarding the force applied by the surgeon to the retina 208. For example, the feedback may be a color-coded indication output to the display device 118, an internal display of the surgical microscope, an audio signal, tactile feedback, or other feedback. For instance, visual feedback may be a red symbol or overlay for excessive force, a green symbol or overlay for force within a suitable range for grasping the membrane, and a yellow symbol or overlay for force too weak to grasp the membrane.

[0054] Other forms of feedback can also be provided. For example, the orientation of the surgical instrument relative to the retina 208 can be compared to a range of acceptable relative orientations for the grasping membrane, and feedback can be provided accordingly. Feedback can be a visual, auditory, or textual message indicating that a change in orientation is needed. Feedback can be in the form of an overlay on a surface image or a rendering based on a three-dimensional image indicating the correct orientation of the surgical instrument.

[0055] In some embodiments, the feedback is distance feedback. For example, it may not be desirable for the forceps 502 to contact anatomical structures that do not need to be peeled, such as areas of the retina 208 not covered by the membrane to be peeled, or other anatomical structures of the eye 102. Therefore, feedback can be provided if the forceps 502 is positioned within a threshold distance of the anatomical structure that does not need to be peeled. The feedback may be a visual, auditory, or text message instructing the surgeon to stop moving the forceps 502 along its current trajectory. Distance feedback may be a displayed number or other indication, such as a distance expressed in micrometers or other units, indicating the distance between the forceps and the retina 208, and may include the distance from the area of ​​the retina 208 to be peeled, to assist the surgeon in contacting the membrane with the forceps.

[0056] refer to Figure 7A and Figure 7B During cataract surgery, the lens 206 is removed via phacoemulsification. The lens 206 is located within a capsular bag 700, which is connected to the ciliary body 214 by a filament called the suspensory ligament 702. An artificial lens (IOL) 704 is then placed within the capsular bag 700 to replace the lens 206. The success of cataract surgery depends on the condition of the capsular bag 700 and the suspensory ligament 702. If the capsular bag 700 ruptures or the suspensory ligament 702 is torn, a different type of IOL and a different placement position are required. The success of cataract surgery also depends on the correct selection of the IOL 704 and its proper placement relative to the retina 208 and cornea 202 to reduce postoperative refractive errors in the eye 102.

[0057] Cataract surgery is performed by inserting an instrument 706 through an incision, typically located at the limbus 224. The instrument 706 is used to create an opening 708 (capsulotomy) in the capsular bag 700, through which the lens 206 is removed and then inserted into the IOL 704.

[0058] Figures 8A to 8F This demonstrates a method that can utilize the surgical operating environment 100 to facilitate cataract surgery. Figure 8AA method 800a that can be performed preoperatively is illustrated. Method 800a may include: capturing one or more surface images of the eye 102 in step 802, and capturing tomographic images of the eye 102 (particularly the lens 206, ciliary body 214, capsular bag 700, and suspensory ligament 702) in step 804. Because method 800a is performed preoperatively, one or more surface images and one or more tomographic images can be obtained using other imaging devices not attached to or otherwise associated with the surgical microscope 108.

[0059] Method 800a may include identifying anatomical structures represented in one or more tomographic images and one or more surface images in step 806. Specifically, representations of the lens 206, ciliary body 214, capsule 700, and suspensory ligaments 702 may be identified according to any of the methods described above. Features of the capsule 700 and suspensory ligaments 702 may then be described in step 808. The feature description of the capsule 700 may include the average thickness of the capsule 700, the minimum thickness of the capsule 700, the location of regions of the capsule 700 below a thickness threshold, or other feature descriptions. The feature description of the suspensory ligaments 702 may include the number or average density of suspensory ligaments (e.g., per unit area of ​​the capsule surface), the average diameter of the suspensory ligaments 702 (the average diameter of the thinnest point of all suspensory ligaments 702), the minimum diameter of the suspensory ligaments 702, or other feature descriptions.

[0060] refer to Figure 8B Method 800b can be performed in the surgical operating environment 100 using a preoperative characterization of the capsular bag 700 and suspensory ligament 702 according to method 800a. Method 800b includes some or all of the following steps: capturing one or more surface images of the eye 102 in step 810, capturing one or more tomographic images of the eye 102 in step 812, and identifying the anatomical structures of the eye in step 814. Steps 810, 812, and 814 can be performed according to any of the methods described above.

[0061] Method 800b includes describing the characteristics of one or both of the capsule 700 and the suspensory ligament 702 based on one or more surface images and one or more tomographic images in step 816. Step 816 may be performed in the manner described in step 808 above. Method 800b may include comparing the characteristic descriptions of the capsule 700 and the suspensory ligament 702 in step 818. Preoperative characteristic description of the capsule 700 and the suspensory ligament 702 is performed according to method 800a.

[0062] If, in step 820, one or more differences are found between the feature description of step 818 and the preoperative feature description, and these differences exceed corresponding thresholds, the method may include outputting feedback to the surgeon in step 822. The feedback may be visual or textual information conveying the differences between the preoperative feature description and the feature description of step 818. For example, the feedback may overlay markers on a rendering of a surface image or a three-dimensional image, indicating that an area of ​​the capsular bag 700 is thinner or looser than indicated in the preoperative data, or that the suspensory ligament 702 is thinner, looser, or absent relative to the preoperative feature description. Therefore, the surgeon can determine whether the proposed cataract treatment remains feasible or whether changes should be made. For example, whether the IOP should be reduced to relieve pressure on the capsular bag 700.

[0063] Figure 8C A method 800c is shown for providing feedback to surgeons to help avoid rupture of the capsular bag 700 during cataract surgery. Method 800c may include some or all of the following steps: capturing one or more surface images in step 810, capturing one or more tomographic images in step 812, and identifying the aforementioned anatomical structures in step 814.

[0064] Method 800c may include defining an instrument envelope in step 824. The instrument envelope may include a circular path for performing capsulorhexis, i.e., cutting an opening in the capsular bag 700 through which the lens 206 can be removed. The circular path may be defined relative to the inner surface of the detected iris 204, such as a path offset inward from the iris 204 by a predefined margin. The instrument envelope may be defined relative to the interior of the capsular bag 700 for use in phacoemulsification. For example, the instrument envelope may include a volume within the capsular bag 700, such as a volume offset from the inner surface of the capsular bag by a margin (e.g., between 0.01 and 0.1 mm).

[0065] Method 800c may include identifying, in step 826, a representation of an instrument (e.g., a phaco-vit tool) in a three-dimensional image composed of one or more tomographic images. If, in step 828, the instrument (e.g., its distal end) is found to be within a threshold distance of the instrument envelope, method 800c may include outputting feedback to the surgeon in step 830. The feedback may simply indicate a potential collision along the instrument envelope. The feedback may also indicate a direction for moving the instrument to avoid a collision with the instrument envelope. The feedback may include a visual alarm output on a display device 118 or a display device within the surgical microscope 108. The alarm may be an audible alarm output via a speaker. The alarm may be tactile feedback output via a tactile device mounted on or within the instrument head. Figure 8C As shown, method 800c can be repeated throughout the entire phacoemulsification procedure.

[0066] Figure 8D A method 800d is demonstrated that can utilize the surgical operating environment 100 to facilitate early detection of capsular rupture. Since the lens 206 is removed, the capsular bag 700 holds the vitreous body 212 in the posterior segment. If the capsular bag 700 ruptures, the vitreous body 212 may begin to leak from the posterior segment. However, because the vitreous body 212 and the capsular bag 700 are transparent, it may be difficult for the surgeon to detect capsular rupture. Using the surgical operating environment 100, the different refractive indices of the vitreous body 212, the capsular bag 700, and the injected fluid used to fill the posterior segment during phacoemulsification can be detected.

[0067] Method 800d may include some or all of the following steps: capturing one or more surface images in step 810, capturing one or more tomographic images in step 812, and identifying the aforementioned anatomical structures in step 814.

[0068] Method 800d may include identifying the vitreous boundary in step 832. Step 832 may include detecting a pixel (or voxel) spot within the posterior segment and detecting the boundary of that spot. For example, assuming that the center of the posterior segment (or a point closer to the retina 208) is definitely occupied by the vitreous, step 832 may include working outward from that point to detect boundaries where the refractive index changes or other anatomical boundaries (such as those reaching the retina and / or choroid).

[0069] Method 800d may include assessing whether capsular rupture has occurred in step 834. For example, if the vitreous representation identified in step 832 is found to extend into the capsular bag, across the iris, or to reach other locations in the anterior segment 200, capsular rupture can be detected. If so, feedback can be output in step 836. The feedback may be in the form of a visual message (such as text or other symbols) output to display device 118 or a display device inside surgical microscope 108. The feedback may also be in the form of an audio message output by a speaker, tactile feedback output through the headpiece, or other types of feedback.

[0070] Figure 8E A method 800e is demonstrated that utilizes a surgical operating environment 100 to facilitate the placement of an IOL 704. With the lens 206 removed, the IOL 704 can be placed within the capsular bag 700. Utilizing the surgical operating environment 100 can help ensure proper IOL placement, thereby preventing capsular bag 700 rupture and reducing postoperative refractive errors.

[0071] Method 800e may include some or all of the following steps: capturing one or more surface images in step 810, capturing one or more tomographic images in step 812, and identifying the aforementioned anatomical structures in step 814.

[0072] Method 800d may include determining the desired IOL location relative to the anatomical structure identified in step 838. For example, a cataract surgery treatment plan may specify the desired location (e.g., along the optical axis of eye 102) and possible orientations (e.g., angular location around the optical axis of eye 102) of IOL 704. The desired location may be defined relative to an anatomical structure (e.g., capsular bag 700, iris 204, ciliary body 214, or other anatomical structures). Therefore, determining the desired IOL location may include identifying the position of the IOL relative to the anatomical structure identified in step 814, which corresponds to the relative position of the IOL to the anatomical structure in the treatment plan.

[0073] Method 800d may include determining the actual location of the IOL in step 840. Step 840 may include identifying the voxel corresponding to the IOL in a three-dimensional image composed of one or more tomographic images. Method 800d may include determining in step 842 whether the difference between the actual IOL location and the desired IOL location exceeds a threshold amount, such as exceeding a first threshold distance along the optical axis, exceeding a first threshold angle around the optical axis, and / or exceeding a second threshold angle (i.e., tilt) in a plane parallel to the optical axis.

[0074] If the difference between the actual IOL location and the desired IOL location is found to exceed one or more thresholds, method 800e may include outputting feedback to the surgeon in step 844. The feedback may be in the form of text or audio output conveying the translation or rotation of the IOL required to achieve the desired IOL location. The feedback may also be in the form of an overlay over a surface image or a rendered image of a three-dimensional image showing the desired IOL location. The overlay may further highlight the representation of IOL 704 to more clearly show the difference between the actual and desired IOL locations.

[0075] Figure 8F A method 800f for selecting IOL 704 (i.e., the refractive power or other characteristics of IOL 704) is demonstrated. Method 800f includes some or all of the following steps: capturing one or more surface images of eye 102 in step 810, capturing one or more tomographic images of eye 102 in step 812, and identifying the anatomical structures of eye 102 in step 814. Steps 810, 812, and 814 can be performed according to any of the methods described above.

[0076] Method 800f may include estimating the refractive error of eye 102 in step 846. Estimating the refractive error may include performing ray tracing or other algorithms to estimate the focal point of the eye by taking into account the refraction of the cornea 202, lens 206, capsular bag 700, aqueous humor (the fluid filling the anterior segment 200), and vitreous body 212.

[0077] Method 800f may further include identifying the location of the IOL in step 848 based on the anatomical structures identified in step 814. For example, the placement of the IOL within the pouch 700 can be identified based on the dimensions of the pouch 700 and the known dimensions of the IOL. An IOL can be selected from a range of available IOLs and placed within the volume provided by the pouch 700.

[0078] Method 800f includes selecting a refractive power for an IOL located at a position selected in step 848 based on the refractive error calculated in step 846 at step 850. The refractive power may be the refractive power of the refractive component of a multifocal IOL. Method 800f may include estimating the refractive error of the IOL with the selected refractive power when placed at the position identified in step 848 at step 852. Step 852 may include performing ray tracing or other modeling techniques to estimate the position of the focal point of the combined eye 102 and the IOL. If it is found at step 854 that the refractive error estimated in step 852 meets a threshold, then method 800f is performed. Otherwise, method 800f may continue at step 850. Method 800f may further include outputting feedback at step 856, such as a report of the refractive error estimated in step 852.

[0079] Additional considerations

[0080] The foregoing description is provided to enable any person skilled in the art to practice the various embodiments described herein. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments. For example, changes can be made to the function and arrangement of the elements discussed without departing from the scope of this disclosure. Various procedures or components may be suitably omitted, substituted, or added in various instances. Furthermore, features described with respect to some instances may be combined in some other instances. For example, any number of aspects set forth herein can be used to implement an apparatus or method of practice. Additionally, the scope of this disclosure is intended to cover such apparatus or methods practiced using other structures, functions, or structures and functions other than or different from the aspects of this disclosure set forth herein. It should be understood that any aspect of this disclosure may be embodied by one or more elements of the claims.

[0081] As used herein, the phrase “at least one” in the list of items refers to any combination of these items, including a single member. For example, “at least one of a, b, or c” is intended to cover a, b, c, ab, ac, bc, and abc, as well as any combination of multiples of the same element (e.g., aa, aaa, aab, aac, abb, acc, bb, bbb, bbc, cc, and ccc, or any other order of a, b, and c).

[0082] As used herein, the term "determine" encompasses a wide variety of actions. For example, "determine" can include calculation, operation, processing, derivation, investigation, searching (e.g., searching in a table, database, or other data structure), ascertainment, etc. Furthermore, "determine" can include receiving (e.g., receiving information), accessing (e.g., accessing data in memory), etc. Additionally, "determine" can include parsing, selecting, picking, building, etc.

[0083] The methods disclosed herein include one or more steps or actions for implementing the methods. The method steps and / or actions may be interchanged without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and / or use of specific steps and / or actions may be modified without departing from the scope of the claims. Furthermore, the various operations of the above methods can be performed by any suitable means capable of performing the corresponding functions. These means may include various hardware and / or software components and / or modules, including but not limited to circuits, application-specific integrated circuits (ASICs), or processors. Typically, where operations are illustrated in the figures, those operations may have corresponding means and functional components with similar numbering.

[0084] The various illustrative logic blocks, modules, and circuits described in connection with this disclosure may be implemented or executed using a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof, designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but alternatively, the processor may be any commercially available processor, controller, microcontroller, or state machine coupled to components of the surgical operating environment 100. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors incorporating a DSP core, or any other such configuration.

[0085] The processing system can be implemented using a bus architecture. Depending on the specific application and overall design constraints of the processing system, the bus can include any number of interconnect buses and bridges. The bus can link together various circuits, including processors, machine-readable media, and input / output devices. User interfaces (e.g., keypads, displays, mice, joysticks, etc.) can also be connected to the bus. The bus can also link various other circuits, such as timing sources, peripherals, voltage regulators, power management circuits, etc., which are well known in the art and therefore will not be described further. The processor can be implemented using one or more general-purpose and / or special-purpose processors. Examples include microprocessors, microcontrollers, DSP processors, and other circuit systems capable of executing software. Those skilled in the art will recognize how best to implement the described functions for the processing system, depending on the specific application and the overall design constraints imposed on the system as a whole.

[0086] If implemented in software, functionality can be stored or transmitted as one or more instructions or code on or through a computer-readable medium. Software should be interpreted broadly as instructions, data, or any combination thereof, whether referred to as software, firmware, middleware, microcode, hardware description language, or others. Computer-readable media includes both computer storage media and communication media (such as any medium that facilitates the transfer of computer programs from one place to another). The processor may be responsible for managing the bus and general processing, including the execution of software modules stored on the computer-readable storage medium. The computer-readable storage medium may be coupled to the processor, allowing the processor to read information from and write information to the storage medium. Alternatively, the storage medium may be integrated into the processor. For example, the computer-readable medium may include a transmission line, a carrier wave modulated by data, and / or a computer-readable storage medium on which instructions separate from the wireless node are stored, all accessible to the processor via a bus interface. Alternatively or additionally, the computer-readable medium or any portion thereof may be integrated into the processor, for example, in cases where it may have a cache and / or a general-purpose register file. Examples of machine-readable storage media may include RAM (random access memory), flash memory, ROM (read-only memory), PROM (programmable read-only memory), EPROM (erasable programmable read-only memory), EEPROM (electrically erasable programmable read-only memory), registers, disks, optical disks, hard disks, or any other suitable storage media, or any combination thereof. Machine-readable media may be embodied in computer program products.

[0087] Software modules can comprise a single instruction or a number of instructions, and can be distributed across several different code segments, across different programs, and across multiple storage media. Computer-readable media can include multiple software modules. Software modules include instructions that, when executed by a device such as a processor, cause the processing system to perform various functions. Software modules can include transmission modules and reception modules. Each software module can reside in a single storage device or be distributed across multiple storage devices. For example, when a triggering event occurs, a software module can be loaded from a hard disk drive into RAM. During the execution of a software module, the processor can load some instructions into a cache to improve access speed. Then, one or more cache lines can be loaded into a general-purpose register file for processor execution. When referring to the functionality of a software module, it should be understood that this functionality is implemented by the processor when executing the instructions from that software module.

[0088] The following claims are not intended to be limited to the embodiments shown herein, but are given the full scope consistent with the language of the claims. In the claims, unless specifically stated otherwise, references to the singular element are not intended to mean "one and only one," but rather "one or more." Unless otherwise specifically stated otherwise, the term "some" means one or more. According to 35U.SC The provisions of 112(f) shall not construe any element of a claim unless such element is expressly described using the phrase “means for…” or, in the case of a method claim, using the phrase “steps for…”. All structural and functional equivalents of elements in all aspects described throughout this disclosure that are known or will be known hereafter by one of ordinary skill in the art are expressly incorporated herein by reference and are intended to be covered by the claims. Furthermore, nothing disclosed herein is intended for public disclosure, whether or not such disclosure is expressly stated in the claims.

Claims

1. A system for performing an ophthalmic treatment, the system comprising: a surgical microscope configured to capture surface images of a patient's eye; an imaging device mounted on the surgical microscope and configured to capture tomographic images of the patient's eye; and a controller coupled with the surgical microscope and the imaging device, the controller configured to receive the surface images and the tomographic images and provide feedback based on the surface images and the tomographic images to facilitate the ophthalmic treatment. The imaging device is an optical coherence tomography (OCT) imaging device.

2. The system of claim 1, wherein, The imaging device is mounted on the surgical microscope by an adjustable support.

3. The system of claim 1, wherein, The imaging device is mounted on a ring fixed to the surgical microscope.

4. The system of claim 1, wherein, 5. The system of claim 1, further comprising an accessory device fixed to the surgical microscope, the accessory device comprising at least one of a sensor or a light source. The accessory device is an intraocular pressure sensor.

6. The system of claim 5, wherein, The ophthalmic treatment comprises a glaucoma treatment and the feedback comprises a feedback image derived from at least one of the surface images and the tomographic images, the feedback image comprising a marker showing a site of at least one of making an incision or placing a stent.

7. The system of claim 1, wherein, The ophthalmic treatment comprises a glaucoma treatment and the feedback comprises a feedback image derived from at least one of the surface images and the tomographic images, the feedback image comprising a marker representing an outflow volume of an anterior segment of the patient's eye.

8. The system of claim 1, wherein, The ophthalmic treatment comprises a membrane peeling treatment of a retina and the feedback comprises a feedback image derived from at least one of the surface images and the tomographic images, the feedback image comprising an overlay marker superimposed on a representation of a membrane of the retina in the feedback image.

9. The system of claim 1, wherein, The ophthalmic treatment comprises a membrane peeling treatment of a retina and the feedback comprises feedback regarding an angle of a surgical instrument.

10. The system of claim 1, wherein, The ophthalmic treatment comprises a membrane peeling treatment of a retina and the feedback comprises feedback regarding a force of a surgical instrument.

11. The system of claim 1, wherein, The controller is configured to estimate the force based on a reflectivity of a membrane of the retina.

12. The system of claim 11, wherein, The ophthalmic treatment comprises phacoemulsification and the feedback comprises feedback regarding a condition of a capsular bag and a zonular ligament of the patient's eye.

13. The system of claim 1, wherein, The ophthalmic treatment comprises intraocular lens (IOL) placement and the feedback comprises feedback regarding a position of the IOL.

14. The system of claim 1, wherein, The controller is configured to detect a representation of an instrument in one or both of the surface images and the tomographic images, the controller further configured to output feedback regarding a proximity of the instrument to an anatomical structure of the patient's eye.

15. The system of claim 1, wherein, 16. A method for performing an ophthalmic treatment, the method comprising: capturing by a surgical microscope configured to capture surface images of a patient's eye; capturing tomographic images of the patient's eye by an imaging device mounted on the surgical microscope; receiving the surface images and the tomographic images by a controller coupled with the surgical microscope and the imaging device; and ​ ​ providing feedback by the controller to facilitate the ophthalmic treatment based on the surface image and the tomographic image.

17. The method of claim 16, wherein, The imaging device is an optical coherence tomography (OCT) imaging device.

18. The method of claim 16, wherein, The ophthalmic treatment is a glaucoma treatment, the method further comprising: generating, by the controller, a first feedback image derived from at least one of the surface image and the tomographic image, the first feedback image including a marker showing a location of at least one of making an incision or placing a stent; outputting, by the controller, the first feedback image to a display device; estimating, by the controller, an outflow volume of an anterior segment of the patient's eye based on the tomographic image; generating, by the controller, a second feedback image derived from at least one of the surface image and the tomographic image, the second feedback image including a marker representing the outflow volume of the anterior segment of the patient's eye; and outputting, by the controller, the second feedback image to the display device.

19. The method of claim 16, wherein, The ophthalmic treatment is a membrane peeling treatment of the retina, the method further comprising: evaluating, by the controller, a reflectivity of a membrane of the retina; calculating, by the controller, an estimated force exerted on the membrane of the retina based on the reflectivity; and outputting, by the controller, the feedback, the feedback being based on the estimated force.

20. The method of claim 16, further comprising: detecting, by the controller, a representation of an instrument in one or both of the surface image and the tomographic image; and outputting, by the controller, the feedback, the feedback corresponding to a proximity of the instrument to an anatomical structure of the patient's eye.

Images