Integrated intraocular navigation system for ophthalmic surgery

Abstract

A system includes an imaging device configured to three-dimensionally image at least a portion of a patient's eye, such as one or both of an OCT camera and a 3D camera. The sensor is configured to sense a position of a trocar cannula positioned in an eye of a patient. The controller is configured to receive one or more three-dimensional images from the imaging device and to receive coordinates of the trocar cannula from the sensor. The controller generates a three-dimensional map of the eye from the one or more three-dimensional images and the coordinates, the three-dimensional map including a representation of the trocar cannula. The controller further generates guidance for performing an ophthalmic procedure, such as an instrument envelope and / or an instrument path, from the three-dimensional map. The controller outputs the guidance to the display device or controls the robot arm using the guidance.

Description

Cross-reference to related applications

[0001] This application claims priority to U.S. Provisional Application No. 63 / 584,491, filed September 21, 2023, which is incorporated herein by reference in its entirety. Background Technology

[0002] This disclosure generally relates to devices used in ophthalmic treatments, particularly devices used to perform vitrectomy, cataract treatment, glaucoma treatment by lowering intraocular pressure (IOP), or other ophthalmic treatments.

[0003] The anatomy of the eye is extremely small, and the movements of instruments during ophthalmic surgery must be correspondingly very precise. For manual surgery, an ophthalmic microscope can be used to provide visualization during the procedure, offering a three-dimensional view of the inside of the patient's eye, including the instruments inserted into the eye.

[0004] Facilitating visualization of the interior of the eye undergoing ophthalmic treatment would be an advancement in the field. Summary of the Invention

[0005] In some embodiments, a system includes an imaging device configured to perform three-dimensional imaging of at least a portion of a patient's eye. Sensors are configured to sense the position of a cannula positioned within the patient's eye. A controller is configured to receive one or more three-dimensional images from the imaging device and to receive coordinates of the cannula from the sensors. The controller generates a three-dimensional atlas of the eye based on the one or more three-dimensional images and coordinates, the atlas including a representation of the cannula. The controller further generates guidance for performing ophthalmic surgery based on the three-dimensional atlas. The controller is configured to perform at least one of the following: (a) outputting the guidance to a display device, and (b) controlling actuators coupled to surgical instruments within the cannula according to the guidance. Attached Figure Description

[0006] To gain a detailed understanding of the features described above, reference can be made to the embodiments for a more specific description of the briefly summarized disclosure, some of which are illustrated in the accompanying drawings. However, it should be noted that the drawings only illustrate exemplary embodiments and should not be construed as limiting the scope of the disclosure, and other equally effective embodiments are permissible.

[0007] Figure 1 An example system for providing intraocular navigation according to certain embodiments is shown.

[0008] Figure 2 It is a cross-sectional view of an eye in which an instrument and a trackable cannula are positioned, according to certain embodiments.

[0009] Figure 3A and Figure 3B A traceable cannula according to certain embodiments is demonstrated.

[0010] Figure 4 This is a schematic diagram of components and data for providing intraocular navigation according to certain embodiments.

[0011] Figure 5 This is a process flowchart of a method for performing intraocular navigation according to certain embodiments.

[0012] Figure 6 An example computing device according to certain embodiments is shown, which at least partially implements one or more functions for providing guidance during ophthalmic surgery using illuminated ophthalmic surgical instruments.

[0013] For ease of understanding, the same reference numerals have been used where possible to refer to the same elements common to the figures. It is contemplated that elements and features of one embodiment can be advantageously combined in other embodiments without further description. Detailed Implementation

[0014] Figure 1 An example system 100 for providing intraocular navigation is shown. System 100 includes one or more three-dimensional imaging devices, such as at least two, for imaging the eye 102 of a patient receiving ophthalmic treatment. In the illustrated embodiment, system 100 includes an optical coherence tomography (OCT) imaging device 104 (“OCT 104”) and a three-dimensional (3D) camera 106. Camera 106 may be a visible light and / or infrared light camera. For example, camera 106 may be implemented as the NGENUITY 3D visualization system provided by Alcon Inc., Fort Worth, Texas. Other types of imaging devices may be used, such as scanning laser ophthalmoscopy (SLO), fundus autofluorescence (FAF) imaging devices, multispectral or hyperspectral imaging devices, or other types of imaging devices. In the following description, with reference to OCT 104 and camera 106, it should be understood that any of the above-described imaging devices may be used in a similar manner.

[0015] OCT 104 and camera 106 can be used separately or simultaneously to image eye 102. For example, beam splitter 108 can be used to allow light reflected from eye 102 to reach both camera 106 and OCT 104.

[0016] For some ophthalmic treatments, one or more cannulas 110 are placed in the eye 102, such as in the sclera of the eye 102. For example, cannulas 110. Cannulas 110 provide an entry point for instruments and help prevent damage to the tissues of the eye 102, and provide a seal to prevent contaminants from entering the eye 102. For some treatments, one or more additional cannulas 110 are provided to receive a light source and / or saline infusion.

[0017] In some embodiments, ophthalmic treatments are performed by or by means of a robotic arm 112 that holds surgical instruments 114. The robotic arm 112 can be remotely controlled by a human operator or can execute a predetermined treatment plan. The robotic arm 112 can dock with a cannula 110 through which instruments 114 are inserted. The robotic arm 112 can be implemented as a tandem robotic arm with five or more degrees of freedom. In other embodiments, instruments 114 are mounted on a handheld device held by a surgeon.

[0018] In some embodiments, the 3D position of each trocar cannula 110 is detected before and / or during ophthalmic treatment. One or more Local Positioning System (LPS) sensors 116 coupled to an LPS controller 118 can be used to sense the trocar cannula 110. The trocar cannula 110, LPS sensors 116, and LPS controller 118 can be implemented in various ways.

[0019] In the first example, the endotracheal cannula 110 includes one or more reference marks or other markings attached thereto, and the LPS sensor 116 is implemented as two or more cameras within its field of view. The position of the reference marks is detected in the image from the LPS sensor 116, and the position and orientation of the endotracheal cannula 110 are determined therefrom.

[0020] In the second example, the cannula 110 includes a transmitter, such as a radio frequency identifier (RFID) transmitter or other type of transmitter. The LPS sensor 116 detects signals from the transmitter and uses these signals to determine the 3D position of the cannula 110 and possibly its orientation. In the second example, the transmitter and LPS sensor 116 can be implemented using any method for detecting the position and possible orientation of a handheld controller in a virtual reality system or other LPS methods known in the art.

[0021] Figure 2This is a cross-sectional view of eye 102, showing an example positioning of various surgical instruments 114a, 114b, 114c and a trackable cannula 110 positioned therein. Eye 102 includes a transparent, spherical cornea through which light enters eye 102. Light passes through a pupil defined by iris 202. Light then passes through lens 204 contained within capsular bag 206. Capsular bag 206 is connected to ciliary body 208, which includes muscles for pulling capsular bag 206 to change the shape of lens 204. The area defined by cornea 200, iris, lens 204, and ciliary body 208 is called the anterior chamber and is occupied by a fluid called aqueous humor. Aqueous humor is filtered at the periphery of the anterior chamber by trabecular meshwork 210 and enters scleral venous sinus 212. Some glaucoma treatments improve aqueous humor drainage by placing an incision or shunt in the trabecular meshwork 210, such as using the illustrated instrument 114a, which is inserted via a cannula 110 through the limbus. The limbus is defined as the junction between the cornea 200 and the sclera 214.

[0022] Ophthalmic treatments for cataracts (including removal and replacement of lens 204) can also use an instrument 114c inserted through the limbus, possibly via a cannula 110, to remove lens 204 (phacoemulsification) and place an artificial lens (IOL).

[0023] Light passes through the lens 204 and a transparent gel called the vitreous body 216, and then onto the retina 218, which contains photoreceptor cells (i.e., rod cells and cone cells). The area between the lens 204 and the retina 218 is called the posterior chamber. Many conditions (such as floaters and some conditions of the retina 218) are treated with vitrectomy, in which an instrument 114b is inserted through a first cannula 110, and the vitreous body 216 is cut and aspirated from the posterior chamber. One or more additional instruments 114c, inserted through one or more second cannulas 110, can provide light to guide surgery, infuse fluids (e.g., saline), or perform other functions.

[0024] Ophthalmic treatments that repair retina 218 (such as performing retinal 218 repositioning surgery) can use a similar arrangement of cannula 110, which receives instruments 114b for grasping and manipulating retina 218 and instruments 114c for providing illumination.

[0025] OCT 104 can be used to achieve partial or complete visualization of eye 102. For example, OCT 104 can be configured to generate a 3D image of the displayed region 220, which includes a portion of the anterior chamber and possibly the posterior chamber. OCT 104 can also be configured to generate a 3D image of region 222, which includes a portion or all of the retina 218 and a portion of the vitreous body 216 adjacent to the retina 218. OCT 104 can penetrate a distance below the retina 218, thereby enabling visualization of the layers of the retina 218. In some embodiments, OCT 104 is configured to image the entire eye from the cornea to a depth below the surface of the retina 218. OCT 104 can image portions of eye 102 extending outward from the optical axis of the cornea 200 and lens 204, and may image the entire eyeball of eye 102. The surface of the retina 218 can also be visualized through the cornea 200 and lens 204 using camera 106.

[0026] Figure 3A and Figure 3B An example embodiment of a traceable cannula 110 is illustrated. The cannula 110 includes a cannula 300, which is a hollow tube that passes through the eyeball of eye 102 to reach the posterior chamber and provides a passage for instrument insertion into the posterior chamber. The cannula 300 is secured to a head 302 of the cannula 110, which extends outward from the cannula 300 and rests on the outer surface of the eye. The head 302 may define a valve 304 that allows instrument insertion through the head 302 and the cannula 300 while preventing contaminants from entering the posterior chamber and preventing fluid leakage from the posterior chamber. The valve 304 may be defined as a slit formed on the head 302 and may be made of a flexible polymer capable of deforming upon instrument insertion through the valve 304.

[0027] One or more trackers 306 are fixed or formed on the head 302 and / or cannula 300. For example, three trackers 306 may allow determination of both the position and orientation of the cannula 110, while a single tracker 306 may only be able to determine the position. Each tracker 306 may be implemented as a static visual indicator visible in visible or infrared wavelengths. The characteristics and range of the tracker 306 implemented as a visual indicator allow detection of the tracker 306 to determine both the position and orientation of the cannula 110. Each tracker 306 may be implemented as a transmitter that emits an identifiable signal. For example, each tracker 306 may be implemented as a radio frequency identifier (RFID) tracker that transmits a code in response to receiving an excitation signal. The tracker 306 may include or be coupled to a power supply that powers the transmitter, which outputs a signal having a known frequency, code, and possibly timing. The transmitter signal may be spontaneous or in response to a control signal received by the tracker 306 (e.g., from the LPS sensor 116 or LPS controller 118).

[0028] Figure 4 This is a schematic diagram of components and data used to provide intraocular navigation. The controller 400 receives data before and / or during ophthalmic treatment. The data may include one or more OCT images 402 from OCT 104, one or more 3D images 404 from 3D camera 106, 3D coordinates 406 of one or more cannulas 110 from LPS controller 118, and kinematic data 408 describing the state of the robotic arm 112. The kinematic data 408 may include the state of each joint of the robotic arm 112, including angular position, velocity, and some or all of acceleration.

[0029] The controller 400 can further receive a treatment plan 410. The treatment plan 410 can define the location for placing the shunt or incision within the trabecular meshwork 210. The treatment plan 410 can specify the path to be followed by the distal end of instrument 114a to remove the lens 204. The treatment plan 410 can specify the path to be followed by the distal end of instrument 114b to remove the vitrectomy 216. The treatment plan 410 can specify the actions to be performed to reposition the retina or perform other repairs on the retina 218 using instrument 114b. The treatment plan 410 can be defined for the anatomy of the eye 102 and can be automatically generated based on the three-dimensional geometry of the eye 102 determined using OCT 104 and possibly a 3D camera 106.

[0030] The controller 400 can retain a treatment history 412. The treatment history may include a record of where the instruments 114a, 114b are located in order to identify areas of the eye 102 that have been treated, such as the portion of the lens 204 that has been traversed by the instrument 114a configured to perform phacoemulsification, or the portion of the posterior chamber that has been traversed by the instrument 114b configured to perform vitrectomy.

[0031] The controller 400 can be connected to and display information on the display device 414, such as a representation of the eye 102 obtained from the OCT image 402 and the 3D image 404. Information can be overlaid on the representation of the eye 102, such as anatomical annotations, lines, shading or coloring of areas that still need to be treated according to the treatment plan 410 and the treatment history 412, areas that have been treated according to the treatment plan 410, or other information.

[0032] The controller 400 can be coupled to the actuator 416 of the robotic arm 112 to control the movement of the robotic arm 112 to perform automated actions based on the treatment plan 410 and the current state of the eye 102 obtained using OCT images 402 and possible 3D images 404.

[0033] Figure 5 This is a process flowchart for performing a method 500 for intraocular navigation. Method 500 can be executed by a controller 400 or other computing device. Method 500 can be executed before performing ophthalmic treatment or throughout the ophthalmic treatment, such as according to a fixed time period, in response to a detected event (e.g., the start of a step in treatment plan 410, movement of eye 102, etc.), or in response to instructions from a surgeon.

[0034] Method 500 includes receiving an OCT image 402 from the OCT 104 in step 502. The OCT image 402 may be a series of images representing different imaging planes within the eye 102, these imaging planes constituting a volumetric image of the eye 102 or regions 220, 222 within the eye at a certain point in time. The OCT image 402 may include one or more images of region 220, one or more images of region 222, or one or more images representing two regions 220, 222, or even possibly the entire eyeball.

[0035] Method 500 includes receiving one or more 3D images 404 from 3D camera 106 in step 504. The one or more 3D images 404 may be a set of stereo images acquired using 3D camera 106 at a certain point in time, a three-dimensional scene acquired from stereo images, a point cloud, or other representation of a 3D surface or volume image acquired using 3D camera 106.

[0036] Method 500 includes receiving 3D coordinates 406 of one or more cannulas 110 in step 506. The coordinates may include coordinates of a point in 3D space on each of the one or more cannulas 110, and possibly an indication of the orientation of each cannulas, in the form of coordinates of a second point on each cannulas or values ​​of one, two, or three angles relative to an axis defined for the 3D space. For example, receiving a cannulas 110 of an instrument 114a moving relative to the eye may result in displacement during the procedure. Accordingly, 3D coordinates 406 may include coordinates of cannulas 110 through which the instrument 114a passes. In step 506, coordinates of other cannulas may be omitted, or coordinates of other cannulas may also be received.

[0037] Method 500 includes receiving kinematic data 408 describing the state of the robotic arm 112 in step 508.

[0038] The OCT image 402, 3D image 404, 3D cannula coordinates 406, and kinematic data 408 used for iterations of method 500 may correspond to substantially the same time points, for example, being captured within 1 second, 0.1 seconds, or 10 milliseconds. In other embodiments, some steps 502 through 508 are performed more frequently than others. For example, kinematic data 408 may be refreshed at a much higher frequency because the robotic arm 112 is moving with high precision. In contrast, since the eye 102 is relatively (but not necessarily completely) stationary during ophthalmic treatment, the acquisition frequency of images 402, 404, and cannula coordinates 406 may be lower than that of kinematic data 408. Therefore, method 500 may be performed one or more iterations without re-performing some or all of the steps 502 through 508, wherein data from previous iterations of some or all of the steps 502 through 508 are reused.

[0039] Method 500 may include generating a 3D atlas of the eye 102 in step 510. Step 510 may include combining OCT images 402 of different regions 220, 222 into a single 3D OCT image. Therefore, step 510 may include transforming the coordinates within the 3D images 402 of regions 220, 222 into a common coordinate system, and combining the transformed images of regions 220, 222 into a single 3D image referred to herein as a 3D atlas. Step 510 may include adding information from the 3D image 404. For example, the voxels of image 402 may be monochromatic. Voxels in the 3D atlas that have corresponding voxels in the 3D image 404 may be assigned the color of the corresponding voxel.

[0040] Method 500 may include: in step 512, adding a representation of one or more cannula needles to a 3D atlas based on the coordinates received in step 506. For example, a 3D model of the cannula may be rendered at a location and possible orientation in the 3D atlas indicated by the coordinates received in step 506 (directly or in a separate overlay). As used herein, a separate overlay may be understood as a 3D image in which the voxels corresponding to the features represented by the separate overlay are non-zero. In the absence of an orientation, the 3D model may be oriented such that the cannula 300 is perpendicular to the point indicated by the coordinates, and the head 302 is flush with the surface of the eye 102.

[0041] Method 500 may include adding one or more representations of one or more devices 114a, 114b, 114c to a 3D atlas in step 514. Step 514 may include rendering a 3D model of each of the one or more devices 114a, 114b, 114c (e.g., at least a model of the portion extending into the eye) directly in the 3D atlas or as a separate overlay using the kinematic data from step 508 and the known dimensions of the one or more devices 114a, 114b, 114c.

[0042] Note that the instrument representation in step 514 can be used to update the treatment history 412. Specifically, the area in the 3D atlas corresponding to the end of the instrument can be added to the treatment history 412 to indicate that the area has been treated. For example, this area could be a portion of the lens 204 or vitreous 216 that has been removed. This area could also be a point within the trabecular meshwork 210 where a shunt has been placed or where an incision has been made.

[0043] Method 500 includes labeling one or more anatomical structures in a 3D atlas in step 516. Step 516 may include labeling the anatomical structures of the eye 102 in the 3D atlas, such as those discussed above. Figure 2 Any anatomical structures marked in the diagram. For the anterior chamber, other anatomical structures may include the ciliary body band, the stained trabecular meshwork, and the unstained trabecular meshwork to facilitate the placement of incisions or shunts for glaucoma treatment.

[0044] Other anatomical structures that can be labeled may include the layers of the eye 102, including some or all of the following: internal limiting membrane, nerve fiber layer, ganglion cell layer, internal plexiform layer, nuclear layer, middle limiting membrane, external plexiform layer, external nuclear layer, external membrane, retinal pigment epithelium (RPE), Bruch's membrane, choroid, or any group of adjacent layers of the above. Features of the retina that can be identified, such as the optic disc, avascular areas, fundus, capillaries, pathological membranes (e.g., epiretinal membrane (ERM)), fovea, or other retinal features. Anatomical structures may include features of the patient's face, such as the nose, eyebrows, or other features that can identify the nasal and temporal sides of the eye. Therefore, the representation of the cannula 110 added to the 3D atlas can be labeled as being on the nasal or temporal side of the eye.

[0045] Step 516 can be performed using one or more machine learning models trained to perform the task, machine vision algorithms, registration relative to previously labeled 3D images, or other methods. The machine learning model for identifying anatomical structures can be trained with images labeled with that anatomical structure, and multiple machine learning models trained to identify multiple anatomical structures may exist. The annotation in step 516 may include generating overlays where corresponding voxels are labeled to correspond to a specific anatomical structure. For example, non-zero pixels in the overlay corresponding to an anatomical structure indicate that the 3D atlas is estimated to correspond to a voxel corresponding to the anatomical structure. The annotation in step 516 may include assigning values ​​to the voxels of the 3D atlas itself. For each labeled anatomical structure or group of anatomical structures, there may be a single overlay or individual overlays.

[0046] The examples of anatomy structures listed above that can be annotated are merely illustrative; any other anatomy structures perceived using the OCT 104 and / or the 3D camera 106 can also be annotated.

[0047] Method 500 may include generating an instrument envelope in step 518. The instrument envelope may correspond to the volume of the eye 102 on which the permissible instruments 114a, 114b are located.

[0048] For example, in the case of glaucoma surgery, the instrument envelope may include the anterior chamber. The instrument envelope may be defined by and derived from the anatomical structures marked in step 516, such as surfaces offset from the anatomical structures used to define the instrument envelope. For example, in glaucoma surgery, the instrument envelope may be offset from the pouch 206 and iris 202, which should not interact with the instrument 114a.

[0049] In the context of cataract surgery, the instrument envelope may include the interior of the capsular bag 206 and the portion of the capsular bag 206 adjacent to the iris forming the opening. The instrument envelope may exclude features such as the iris 202 and the ciliary body 208.

[0050] In the case of vitrectomy, the instrument envelope may include a volume within the posterior chamber, wherein the surface of this volume has a safety margin of offset, for example, at least 0.1 mm, from the retina 218, the capsular bag 206, the ciliary body 208, and the choroidal layer. The instrument envelope may exclude sensitive anatomical structures such as blood vessels, the optic disc, the fovea, or other anatomical structures.

[0051] The examples above are merely illustrative. Additional instrument envelopes can be defined for other ophthalmic treatments based on the necessary movement of the instruments used.

[0052] Method 500 may include updating the device path in step 520. Step 520 may consider some or all of the following: treatment plan 410, treatment history 412, and the device envelope from step 518. The device path may be generated by a machine learning model trained to perform the task. For example, the machine learning model may take a 3D atlas and possible treatment plan 410, treatment history 412, or some other input as input. For example, the device path may include a path for aligning devices 114a, 114b, 114c with cannula 110 and inserting devices 114b, 114c into cannula 110. Similarly, during ophthalmic treatment, it may be necessary to move devices 114b, 114c to another different cannula 110. For example, the instrument pathway may include removing an instrument 114a, implemented as an phacoemulsification-vitrectomy tool, from the first cannula 110, aligning the instrument 114a with the second cannula 110, and inserting the phacoemulsification-vitrectomy tool into the second cannula 110 to remove portions of the vitreous that are inaccessible through the first cannula. Similarly, an instrument 114b providing fluid infusion may be moved from the second cannula 110 to the first cannula.

[0053] In the case of glaucoma surgery, the instrument path may include points on the trabecular meshwork 210 defined in the treatment plan that have not yet received a shunt or incision. Points on the trabecular meshwork 210 may be defined relative to a 3D atlas, i.e., taking into account any movement on the eye. Specifically, points on the trabecular meshwork 210 may be defined relative to the anatomical structures annotated in step 516 for the current iteration of method 500.

[0054] In the case of cataract surgery, the instrument path may include points within the instrument envelope as defined in step 518 for the capsular bag 206, which have not yet been traversed by the instrument 114a configured to perform phacoemulsification. The instrument path may traverse paths designed to efficiently cross the volume of the capsular bag, such as regular rows, spirals, or other shapes.

[0055] In the case of vitrectomy, the instrument path may include points within the instrument envelope defined for the posterior chamber in step 518 that have not yet been traversed by the instrument 114b configured to perform the vitrectomy. The instrument path may traverse a path designed to efficiently traverse the volume of the posterior chamber, such as a regular line, a spiral, or other shape.

[0056] In the example above, the previously treated area recorded in treatment history 412 can be transformed (rotated and / or translated) based on the rotation, translation, and / or deformation of the eye 102. Any rotation, translation, and / or deformation can be determined by comparing the changes in the position of the anatomical structures identified in step 516 from one iteration of method 500 to the next. Similarly, the changes in the coordinates of the cannula 110 from step 506 to one iteration of method 500 to the next can be used to estimate the rotation and / or translation of the eye 102.

[0057] In another example, the instrument path defines the location of the dissection steps for peeling a membrane (e.g., ILM or ELM) from the retina, such as in a spiral or zigzag shape. The portion of the retina to be peeled can be identified based on anatomical structures, such as the stained area of ​​the fovea and the avascular area from which the membrane is to be peeled.

[0058] In another example, the instrument path defines the path through which the therapeutic laser emits laser pulses to perform retinal reattachment. The path can be chosen to avoid sensitive anatomical structures, such as the blood vessels and anatomical structures identified in step 516. For example, the path could extend around the macula to avoid damaging it.

[0059] Method 500 may include a representation of any of the preceding steps of method 500 output in step 522. For example, a representation of a 3D atlas may be displayed, including a representation of an overlay of one or more cannulas, one or more instruments 114a, 114b, an instrument envelope, and instrument paths. In the case of a human surgeon instead of the robotic arm 112, step 522 may be performed to provide guidance to the surgeon. In particular, a representation of the instrument envelope can help avoid tissue that does not need to be treated. A representation of the instrument path can help the surgeon traverse the volume to be treated as efficiently as possible. In the case of a human surgeon, the instrument path may be represented as the area to be treated, rather than the specific path that the instruments 114a, 114b are to follow.

[0060] Figure 6 An example computing system 600 is shown. The OCT 104, 3D camera 106, LPS controller 118, and controller 400 may have some or all of the properties of computing system 600.

[0061] As shown in the figure, the computing system 600 includes a central processing unit (CPU) 602, one or more I / O device interfaces 604 that allow various I / O devices 614 (e.g., keyboard, display, mouse device, pen input, etc.) to be connected to the computing system 600, a network interface 606 through which the computing system 600 is connected to a network 690, a memory 608, a storage device 610, and interconnects 612.

[0062] CPU 602 can retrieve and execute programming instructions stored in memory 608. Similarly, CPU 602 can retrieve and store application data residing in memory 608. Interconnect 612 transfers programming instructions and application data between CPU 602, I / O device interface 604, network interface 606, memory 608, and storage device 610. CPU 602 can be used to represent a single CPU, multiple CPUs, a single CPU with multiple processing cores, etc.

[0063] Memory 608 represents volatile memory (such as random access memory) and / or non-volatile memory (such as non-volatile random access memory, phase-change random access memory, etc.). As shown, memory 608 may store input data 616 used according to method 500, including some or all of the following: one or more OCT images 402, one or more 3D images 404, 3D cannula needle coordinates 406, and kinematic data 408. Memory 608 may further store 3D atlases generated according to method 500, which may include storing any overlays generated according to method 500 and possible 3D atlases 618, and any overlays from one or more previous iterations of method 500. Memory 608 may further store the current treatment history 412 updated during the performance of ophthalmic treatment.

[0064] Storage device 610 may be a non-volatile memory, such as a disk drive, a solid-state drive, or a collection of storage devices distributed across multiple storage systems. Storage device 610 may optionally store treatment plan 410.

[0065] In a first example embodiment, a method includes: placing a cannula in a patient's eye; inserting a surgical instrument into the patient's eye; imaging the patient's eye using an imaging device; sensing the position of the cannula using a sensor; and having a controller perform the following operations: receiving one or more three-dimensional images from the imaging device; receiving coordinates of the cannula from the sensor; generating a three-dimensional atlas of the eye based on the one or more three-dimensional images and coordinates, the three-dimensional atlas including a representation of the cannula; generating guidance for performing ophthalmic surgery based on the three-dimensional atlas; and at least one of the following: (a) outputting the guidance to a display device; and (b) controlling an actuator coupled to a surgical instrument within the cannula based on the guidance.

[0066] In some embodiments of the first example embodiment, the imaging apparatus includes both an optical coherence tomography imaging apparatus and a three-dimensional camera.

[0067] In some embodiments of the first example embodiment, the method includes the controller performing the following operations: detecting one or more representations of one or more anatomical structures in a three-dimensional atlas; generating an instrument envelope based on the one or more representations; detecting regions of the eye traversed by surgical instruments; generating instrument paths representing portions of the eye still requiring treatment based on these regions and a treatment plan; and at least one of the following: (a) outputting representations of the surgical instruments and instrument paths to a display device; and (b) controlling actuators coupled to the surgical instruments based on the instrument envelope and instrument paths. Additional considerations

[0068] The foregoing description is provided to enable any person skilled in the art to practice the various embodiments described herein. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments. For example, changes can be made to the function and arrangement of the elements discussed without departing from the scope of this disclosure. Various procedures or components can be suitably omitted, substituted, or added to various examples. Furthermore, features described with respect to some examples can be combined in some other examples. For example, any number of aspects set forth herein can be used to implement an apparatus or method of practice. Additionally, the scope of this disclosure is intended to cover such apparatus or methods practiced using other structures, functions, or structures and functions other than or different from the aspects of this disclosure set forth herein. It should be understood that any aspect of this disclosure may be embodied by one or more elements of the claims.

[0069] As used herein, the phrase “at least one of a series of items” refers to any combination of those items, including a single member. For example, “at least one of a, b, or c” is intended to cover a, b, c, ab, ac, bc, and abc, as well as any combination of multiples of the same element (e.g., aa, aaa, aab, aac, abb, acc, bb, bbb, bbbc, cc, and ccc, or any other order of a, b, and c).

[0070] As used herein, the term "determine" encompasses a wide variety of actions. For example, "determine" can include calculation, operation, processing, derivation, investigation, searching (e.g., searching in a table, database, or other data structure), ascertainment, etc. Furthermore, "determine" can include receiving (e.g., receiving information), accessing (e.g., accessing data in memory), etc. Additionally, "determine" can include parsing, selecting, picking, building, etc.

[0071] The methods disclosed herein include one or more steps or actions for implementing the methods. The method steps and / or actions may be interchanged without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and / or use of specific steps and / or actions may be modified without departing from the scope of the claims. Furthermore, the various operations of the above methods can be performed by any suitable means capable of performing the corresponding functions. These means may include various hardware and / or software components and / or modules, including but not limited to circuits, application-specific integrated circuits (ASICs), or processors. Typically, where operations are illustrated in the figures, those operations may have corresponding means and functional components with similar numbering.

[0072] The various illustrative logic blocks, modules, and circuits described in connection with this disclosure may be implemented or executed using a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof, designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but alternatively, the processor may be any commercially available processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors incorporating a DSP core, or any other such configuration.

[0073] The processing system can be implemented using a bus architecture. Depending on the specific application and overall design constraints of the processing system, the bus can include any number of interconnect buses and bridges. The bus can link together various circuits, including processors, machine-readable media, and input / output devices. User interfaces (e.g., keypads, displays, mice, joysticks, etc.) can also be connected to the bus. The bus can also link various other circuits, such as timing sources, peripherals, voltage regulators, power management circuits, etc., which are well known in the art and therefore will not be described further. The processor can be implemented using one or more general-purpose and / or special-purpose processors. Examples include microprocessors, microcontrollers, DSP processors, and other circuit systems capable of executing software. Those skilled in the art will recognize how best to implement the described functions for the processing system, depending on the specific application and the overall design constraints imposed on the system as a whole.

[0074] If implemented in software, functionality can be stored or transmitted as one or more instructions or code on or through a computer-readable medium. Software should be interpreted broadly as instructions, data, or any combination thereof, whether referred to as software, firmware, middleware, microcode, hardware description language, or others. Computer-readable media includes both computer storage media and communication media (such as any medium that facilitates the transfer of computer programs from one place to another). The processor may be responsible for managing the bus and general processing, including the execution of software modules stored on the computer-readable storage medium. The computer-readable storage medium may be coupled to the processor, allowing the processor to read information from and write information to the storage medium. Alternatively, the storage medium may be integrated into the processor. For example, the computer-readable medium may include a transmission line, a carrier wave modulated by data, and / or a computer-readable storage medium on which instructions separate from the wireless node are stored, all accessible to the processor via a bus interface. Alternatively or additionally, the computer-readable medium or any portion thereof may be integrated into the processor, for example, in cases where it may have a cache and / or a general-purpose register file. Examples of machine-readable storage media may include RAM (random access memory), flash memory, ROM (read-only memory), PROM (programmable read-only memory), EPROM (erasable programmable read-only memory), EEPROM (electrically erasable programmable read-only memory), registers, disks, optical disks, hard disks, or any other suitable storage media, or any combination thereof. Machine-readable media may be embodied in computer program products.

[0075] Software modules can comprise a single instruction or a number of instructions, and can be distributed across several different code segments, across different programs, and across multiple storage media. Computer-readable media can include multiple software modules. Software modules include instructions that, when executed by a device such as a processor, cause the processing system to perform various functions. Software modules can include transmission modules and reception modules. Each software module can reside in a single storage device or be distributed across multiple storage devices. For example, when a triggering event occurs, a software module can be loaded from a hard disk drive into RAM. During the execution of a software module, the processor can load some instructions into a cache to improve access speed. Then, one or more cache lines can be loaded into a general-purpose register file for processor execution. When referring to the functionality of a software module, it should be understood that this functionality is implemented by the processor when executing the instructions from that software module. The following claims are not intended to be limited to the embodiments shown herein, but are given the full scope consistent with the language of the claims. In the claims, references to singular elements, unless specifically stated otherwise, are not intended to mean “one and only one”, but rather “one or more.” Unless otherwise specifically stated otherwise, the term “some” means one or more. No element of any claim shall be interpreted in accordance with 35 U.SC §112(f) unless such elements are expressly described using the phrase “means for…” or, in the case of a method claim, using the phrase “steps for…”. All structural and functional equivalents of the elements of the various aspects described throughout this disclosure that are known to or will be known hereafter by one of ordinary skill in the art are expressly incorporated herein by reference and are intended to be covered by the claims. Furthermore, nothing disclosed herein is intended for public disclosure, whether or not such disclosure is expressly stated in the claims.

Claims

1. A system comprising: An imaging device configured to perform three-dimensional imaging of at least a portion of a patient's eye; A sensor configured to sense the position of a cannula inserted in the patient's eye; as well as The controller is configured to: Receive one or more three-dimensional images from the imaging device; Receive the coordinates of the cannula needle insertion tube from the sensor; A three-dimensional map of the eye is generated based on the one or more three-dimensional images and the coordinates, the three-dimensional map including a representation of the cannula; Based on the three-dimensional atlas, a guide for performing ophthalmic surgery is generated; and At least one of the following: (a) outputting the guidance to a display device; and (b) controlling an actuator connected to a surgical instrument within the cannula according to the guidance.

2. The system as claimed in claim 1, wherein, The imaging device includes an optical coherence tomography imaging device.

3. The system as described in claim 1, wherein, The imaging device is a 3D camera.

4. The system as claimed in claim 1, wherein, The imaging device includes both an optical coherence tomography imaging device and a three-dimensional camera.

5. The system as claimed in claim 1, wherein, The sensor includes two or more cameras configured to detect one or more reference marks on the cannula.

6. The system of claim 1, wherein, The sensor includes a plurality of local positioning sensors configured to sense signals emitted from the cannula needle cannula.

7. The system as claimed in claim 1, wherein, The sensor includes a plurality of local positioning sensors configured to sense signals emitted from one or more radio frequency identifiers (RFID) devices in the cannula.

8. The system of claim 1, wherein, The controller is configured to: Detect one or more representations of one or more anatomical structures in the three-dimensional atlas; Generate an instrument envelope based on one or more of the representations; as well as At least one of the following: (a) outputting a representation of the instrument envelope to the display device; (b) Controlling the actuator connected to the surgical instrument according to the instrument envelope.

9. The system of claim 8, wherein, The controller is configured to use one or more machine learning models to detect one or more representations of the one or more anatomical structures.

10. The system of claim 8, wherein, The instrument envelope corresponds to the posterior chamber of the patient's eye.

11. The system of claim 8, wherein, The instrument envelope corresponds to the anterior chamber of the patient's eye.

12. The system of claim 8, wherein, The instrument envelope corresponds to the interior of the eye capsule of the patient.

13. The system of claim 1, wherein, The controller is further configured to: Detect the area of ​​the eye traversed by the surgical instruments; Based on the region and treatment plan, generate an instrument path representing the portion of the eye that still requires treatment; as well as At least one of the following: (a) outputting a representation of the instrument path to the display device; and (b) controlling the actuator connected to the surgical instrument according to the instrument path.

14. The system of claim 13, wherein, The controller is configured to use a machine learning model to generate the instrument path.

15. The system of claim 1, wherein, The controller is configured to execute (b), and the actuator is a robotic arm.

16. The system of claim 15, wherein, The ophthalmic surgery defines the placement of at least one of an incision and a shunt within the trabecular mesh of the eye to treat glaucoma.

17. The system of claim 15, wherein, The ophthalmic procedure described refers to phacoemulsification of the lens of the eye.

18. The system of claim 15, wherein, The ophthalmic surgery described refers to vitrectomy of the eye.

19. The system of claim 15, wherein, The ophthalmic procedure described specifies the removal of the membrane.

20. The system of claim 15, wherein, The ophthalmic surgery described herein is limited to performing retinal reattachment surgery.

Images