Toric intraocular lens alignment guide

Abstract

Certain embodiments disclosed herein provide an alignment guide for aligning a toric IOL during surgery. An image having a reference axis is acquired, such as from a digital microscope, and processed, such as using an autoencoder, to label alignment marks on the IOL and possibly other features of the IOL. The labels are processed, for example, using a logistic regression model, to estimate the IOL axis of the IOL that intersects with the alignment mark. An output image is generated from the image overlaid with a guide to the surgeon, such as a line representing the IOL axis, a rotation direction indicator, and a number or other representation of the difference between the reference axis and the IOL axis. Tracking of IOL features may be performed across multiple images to predict the location of features not represented in a particular image.

Description

[Technical Field]

[0001] The present disclosure relates generally to methods for the treatment of cataracts, and more particularly to treatments involving the use of intraocular lenses (IOLs). [Background technology]

[0002] Light entering the human eye passes through the clear cornea, which covers the iris and pupil of the eye. The light is transmitted through the pupil and focused by the lens, which is located behind the pupil in a structure called the lens capsule. The lens focuses the light onto the retina, which contains rods and cones that can generate nerve impulses in response to light.

[0003] The lens can become cloudy due to aging or disease, a condition known as a cataract. Cataracts are easily treated by removing the lens and inserting an artificial lens known as an intraocular lens (IOL). IOLs may also be fabricated to correct additional aberrations in the patient's eye, such as astigmatism. Because astigmatism is the result of ocular asymmetry, the IOL must be adjusted to accommodate the ocular asymmetry in order to correct the astigmatism. Therefore, IOLs are often provided with markers, such as a series of dots, around the periphery of the IOL that define axes that can be used to align the IOL. IOLs may be implemented as toric IOLs, which contain spring-like arms known as haptics that hold the IOL in place within the capsular bag. Traditionally, an imaging device, such as a digital marker microscope (DMM), is used to view the patient's eye during surgery. The image output by the imaging device is overlaid with reference axes corresponding to the desired orientation of the IOL's axes. Summary of the Invention [Problem to be solved by the invention]

[0004] Given the desirability of precise alignment of the IOL axis with the fiducials, techniques to facilitate this alignment would significantly improve patient outcomes. [Means for solving the problem]

[0005] The present disclosure generally relates to a system that provides an alignment guide for positioning a toric IOL in a patient's eye.

[0006] Certain embodiments disclosed herein provide a method and corresponding apparatus for providing alignment guidance during ophthalmic surgery. The method includes receiving, by a computing device, from an imaging device, an input image of a patient's eye having a toric intraocular lens (IOL) therein. The computing device obtains a reference axis of the patient's eye, the reference axis indicating a desired orientation of the toric IOL axis of the toric IOL. The input image is processed to obtain feature labels indicating locations of features of the toric IOL represented in the input image, the features including any of alignment dots defined on the toric IOL, the periphery of the toric IOL, and a portion of a haptic portion of the toric IOL. The feature labels are processed by the computing device to identify the orientation of the toric IOL axis. The method then includes calculating an angular difference between the toric IOL axis and the reference axis. The computing device then generates and outputs an output image including at least one indicator corresponding to the angular difference.

[0007] The following description and the related drawings set forth in detail certain illustrative features of the one or more embodiments.

[0008] The accompanying drawings depict certain aspects of one or more embodiments and therefore should not be considered as limiting the scope of the disclosure. [Brief explanation of the drawings]

[0009] [Figure 1] Figure 1 shows the anatomy of the human eye. [Figure 2] FIG. 2 shows a toric IOL. [Figure 3A-3B]3A and 3B illustrate an alignment guide system and a post-processor for positioning a toric IOL, respectively, according to certain embodiments. [Figure 4A] FIG. 4A illustrates a method for providing an alignment guide for positioning a toric IOL, according to certain embodiments. [Figure 4B] FIG. 4B is a method for performing post-processing, according to certain embodiments. [Figure 5A-5B] 5A and 5B illustrate providing an alignment guide on an image of a patient's eye during placement of a toric IOL, according to certain embodiments. [Figure 5C-5D] 5C and 5D illustrate providing an alignment guide on an image of a patient's eye during placement of a toric IOL, according to certain embodiments. [Figures 5E-5F] 5E and 5F illustrate providing an alignment guide on an image of a patient's eye while a toric IOL is being placed, according to certain embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0010] For clarity, where possible, the same reference numerals have been used to denote identical elements common to the figures, and it is contemplated that elements and features of one embodiment may be beneficially incorporated in other embodiments without specific indication.

[0011] Certain embodiments of the present disclosure provide an alignment guide for positioning a toric IOL in a patient's eye.

[0012] FIG. 1 illustrates portions of a human eye 100, which can be understood in terms of an anterior side, where light enters the eye, and a posterior side, opposite the anterior side. At the anterior side of the eye 100, a thin, transparent layer known as the cornea 102 is connected to the sclera 104, which forms the roughly spherical wall of the eye 100. The cornea 102 and sclera 104 are connected by a ring called the limbus. The iris 106, which is the color of the eye, and the opening defined by the iris, or pupil, are located behind the cornea and are visible due to the transparency of the cornea 102. The retina 108 is formed on the inner surface of the sclera 104, opposite the cornea 102 and iris 16. The volume defined by the sclera 104 is occupied by a transparent jelly called the vitreous body 110.

[0013] The crystalline lens 112 is a transparent, biconvex structure within the eye that, together with the cornea 102, helps refract light and focus it on the retina 108. By changing its shape, the crystalline lens 112 functions to change the focal length of the eye so that objects at different distances can be focused, thus allowing a sharp, real image of the object to be formed on the retina 108. This adjustment of the crystalline lens 112 is known as accommodation and is similar to the movement of a lens to adjust the focus of a photographic camera.

[0014] The lens 112 is positioned behind the iris 106 within a lens capsule 114. The lens capsule 114 is attached around its periphery to the suspensory ciliary ligament 116. The ciliary ligament 116 attaches the lens capsule 114 to the ciliary body 118. The ciliary body 118 is a ring-shaped muscle that attaches the ciliary ligament 116 to the sclera 104 and can contract or relax to change the shape of the lens 112.

[0015] Various diseases and disorders of the crystalline lens 112 may be treated with IOLs. By way of example, and not necessarily by way of limitation, IOLs according to embodiments of the present disclosure may be used to treat cataracts, large optical errors in myopic (near-sighted) eyes, hyperopic (far-sighted) eyes, and astigmatic eyes, ectopia lentis, aphakia, pseudophakia, and nuclear sclerosis. For purposes of explanation, however, IOL embodiments of the present disclosure are described with reference to cataracts, which often occur in the elderly population.

[0016] 2 illustrates an exemplary toric IOL 200. The toric IOL 200 includes a lens portion 202 that focuses light passing through the iris 106 onto the retina 108. The lens portion 202 may be surrounded by a peripheral ring 204 that is not used to focus light. Two or more haptics 206 may be secured to the peripheral ring 204. Each haptic 206 may include a base 208 secured to and extending outward from the peripheral ring 204. Spring arms 210 are secured to the base and extend circumferentially outward from and around the peripheral ring 204. In use, the spring arms 210 push outward against the lens capsule 114, holding the toric IOL 200 in a desired position.

[0017] Marks 212 may be formed on the peripheral ring 204. The marks 212 facilitate alignment of the IOL with the patient's eye 100. The marks 212 in the illustrated toric IOL 200 include two sets of dots (e.g., depressions or bumps), such as circular dots, formed on opposing peripheral rings 204. For example, each set may include two, three, or more dots. The dots in each set may be collinear with each other or with the dots in the other set. A line passing through one or both sets of dots (hereinafter, the "IOL axis") may also intersect and be perpendicular to the optical axis of the lens portion 202. In the illustrated toric IOL 200, the IOL axis also intersects the base 208 of the haptics 206. Some toric IOLs 200 are multifocal. The lens portion 202 may include a ring 214 that defines a boundary between regions of the lens portion 202 having different focal lengths. As discussed below, the marks 212 can be detected using a machine learning model and used to determine the IOL axis. Thus, the marks 212 need not intersect the IOL axis and can include any pattern visible in an image of the IOL 200. The machine learning model can be trained to identify marks 212 of any shape, arrangement, and number. Also, as discussed below, features other than the marks 212, such as the base 208 of the haptics, the periphery of the IOL 200, etc., can be used to determine the orientation of the IOL 200. The geometric relationship between any two or more features can be used to determine the orientation of the IOL 200 using a machine learning model, as outlined below.

[0018] 3A illustrates an alignment guide system 300a for positioning a toric IOL 200 within a patient's eye 100. The alignment guide system 300a may include an imaging device 302 for capturing images of the patient's eye 100 during implantation and alignment of the toric IOL 200. The imaging device 302 may be implemented as a digital three-dimensional microscope, such as the ALCON NGENUITY 1.5 (a digital three-dimensional digital marker microscope (DMM) with image guidance) or the ZEISS ARTEVO, or as an analog microscope with image guidance, such as the ALCON VERION digital marker or the ZEISS CALISTO. As known in the art, the imaging device 302 may be programmed with a treatment plan that includes preoperative images of the patient's eye 100, which may include images of the sclera 104, the iris 106, and optionally a portion of the retina 108. The imaging device 302 may be further configured to define a reference axis for one or more images of the patient's eye 100. The imaging device 302 can be programmed to perform registration of the patient's eye 100. Registration can be performed by capturing an image of the patient's eye 100 and matching ocular anatomical structures, such as the unique patterns of blood vessels on the sclera 104 and / or retina 108 and / or the unique patterns of the limbus and / or iris 106, depicted in the image with representations of corresponding ocular anatomical structures in one or more pre-operative images to determine the orientation of the eye 100. The imaging device 302 can then determine a transformation relating the orientation of the eye 100 in the image to the orientation of the eye in a reference image included in the treatment plan. The reference axes of the treatment plan can then be rotated and / or translated according to the transformation. References herein to reference axes superimposed on an image of the patient's eye should be understood as referring to the reference axes resulting from the transformation. The reference axis registration and corresponding transformation can be performed repeatedly on images (e.g., frames of video data) captured throughout surgery to account for movement of the patient's eye relative to the imaging device 302.

[0019] The components of system 300a other than the imaging device may be implemented using the computing power of imaging device 302 embodied as a digital microscope. Alternatively, the additional components of system 300a may be implemented by a separate computing device that receives images labeled with reference axes from imaging device 302. In yet other embodiments, the separate computing device may receive only images from imaging device 302 and perform registration to one or more pre-operative images as described above to obtain the reference axes.

[0020] The system 300a may include an autoencoder 304. The autoencoder 304 receives the image output by the imaging device 302, which may have a reference axis marked, and identifies features of the IOL 200 represented in the image. These features may include some or all of the marks 212, the ring 214, the base 208 of the haptics, and the circumference of the peripheral ring 204. The autoencoder 304 may label one or more of these features. The labels may be in the form of one or more pixel masks in which non-zero pixels correspond to pixels of the image found by the autoencoder 304 to represent one or more features or types of features. A separate pixel mask may be generated by the autoencoder 304 for each feature or each type of feature, such as a mask for the marks 212, a mask for the ring 214, a mask for the base 208 of the haptics, and a mask for the circumference of the peripheral ring 204. The autoencoder 304 may generate multiple masks, with one or more masks labeling multiple features or multiple types of features. Alternatively, a single mask may mark pixels corresponding to any of the features.

[0021] The autoencoder 304 may include an encoder 304a and a decoder 304b, such that an image is input to the encoder 304a and the output of the encoder 304a is input to the decoder 304b. The output of the decoder 304b may include one or more pixel masks as described above. The autoencoder 304 may be implemented using a convolutional neural network (CNN), a deep neural network (DNN), or other type of neural network. The autoencoder 304 may be replaced by any machine learning model known in the art that is trained to perform the tasks attributed to the autoencoder 304 herein.

[0022] Training data for the autoencoder 304 may include as input an image of the patient's eye 100 and as a desired output one or more pixel masks labeling features of the toric IOL 200 placed in the patient's eye 100. For example, during an implantation procedure, a video feed from a DMM may be captured, and multiple frames from the video feed may be labeled by a human (e.g., a human-generated pixel mask marking pixels corresponding to features represented by the pixel mask), such that each frame and its corresponding label becomes a training data entry. Such training data entries from multiple patients may then be used to train the autoencoder 304 by processing images for each training data entry with the autoencoder 304, receiving output from the autoencoder 304, comparing the autoencoder's output to one or more pixel masks of the training data entries (e.g., evaluating a loss function), and updating parameters of the autoencoder 304 according to the comparison.

[0023] The autoencoder 304 may include or be used in combination with an attention mechanism that designates one or more regions of the input image that contain or are likely to contain a feature of the container (e.g., the mark 212, the base 208 of the haptic portion 206, the periphery of the IOL 200, etc.). The attention mechanism can generate bounding boxes that contain or are likely to contain the feature. The attention mechanism may be another machine learning model, such as another autoencoder, or one or more layers of the autoencoder 304. The attention mechanism may be trained using training data entries, each of which includes an image as input and one or more bounding boxes obtained from a human or computerized labeler as desired output, where the one or more bounding boxes label one or more features, respectively. The training algorithm may then process each training data entry by processing the input image using the attention mechanism to obtain one or more estimated bounding boxes. The one or more estimated bounding boxes may then be compared to one or more bounding boxes of the training data entry. The training algorithm may then update the parameters of the attention mechanism according to the similarity between the one or more estimated bounding boxes and the one or more bounding boxes of the training data entry. The output of the attention mechanism may be an output image that is the input image with pixels that fall within one or more bounding boxes highlighted. Desired features are identified within the regions highlighted by the bounding boxes by subsequent layers of the autoencoder 304.

[0024] The labels output by the autoencoder 304 may be input to a post-processor 306. The post-processor 306 may identify the IOL axis from the label and determine the angular difference between the IOL axis and the reference axis. The post-processor 306 may generate an output image based on the image received from the imaging device 302 overlaid with information such as lines and arrows representing the reference and IOL axes, text, or other information indicating the amount of rotation required to align the IOL axis with the reference axis.

[0025] The output image generated by post-processor 306 may then be output to display device 308, which may be a screen integrated into imaging device 302 and viewable through the eyepieces of imaging device 302. Display device 308 may be implemented as a monitor in the room in which the toric IOL 200 is being implanted, a heads-up display worn by the implanting surgeon, or a separate display device.

[0026] 3B shows an exemplary implementation of the post-processor 306. The post-processor 306 may include a tracker 310. The output of the autoencoder 304 may be a series of labels for each frame of the video feed from the imaging device 302. The tracker 310 may receive these labels and possibly frames of the video feed and track the movement of features represented by the labels from frame to frame. Thus, the tracker 310 may predict the location of features in frames where the feature is not labeled because (a) it is obscured by a surgical instrument or a portion of the eye 100 (e.g., the iris 106) or (b) it was not successfully identified by the autoencoder 304. The tracker 310 may be implemented as a Markov chain, a flow detector, a Kalman filter, or any other tracking algorithm known in the art.

[0027] The output of the tracker 310 for a frame of the video feed may be a label having the same form as the label received from the autoencoder 304. For example, the output of the tracker 310 may be a pixel mask ("tracker mask") corresponding to each pixel mask ("original mask") output by the autoencoder 304. The tracker mask may include new non-zero pixels at pixel locations that are zero in the original mask, where the new non-zero pixels indicate predicted locations of features or portions of features that were not labeled in the original mask. The tracker 310 may also perform noise cancellation or smoothing to a degree that may cause pixel locations in the original mask and the tracker mask to have different values ​​(flipping from zero to non-zero or from non-zero to zero).

[0028] The labels output by the tracker 310, or the labels output by the autoencoder 304 if the tracker 310 is not used ("labels"), may be input to a line generator 312. The line generator 312 may be any machine vision algorithm trained or programmed to fit lines to the labels of the marks 212. In some implementations, the line generator 312 is a machine learning model, such as a neural network implementing a logistic regression model, trained to generate line parameters (e.g., slope and x- or y-intercept) based on the labels and, optionally, the images ("images") from which the labels were generated. As mentioned above, the labels themselves may include one or more pixel masks.

[0029] The labels may contain more information than is necessary to define the IOL axis. While only two points are required to define a line, the marks 212 may include four, six, or more points, and the bases 208 of the haptics may also be represented in the image. The machine learning model of the line generator 312 may advantageously use some or all of this information to accurately define the line representing the IOL axis. Training data entries for the machine learning model may include labels and optionally images corresponding to the labels as input, and may include human-generated parameters (slope and x- or y-intercept) describing the IOL axis of the toric IOL represented in the image as desired output. For each training data entry, the input may be processed using the machine learning model to obtain estimated line parameters. The estimated line parameters may be compared to the parameters of the training data entry (e.g., a loss function may be evaluated), and the parameters of the machine learning model may be updated according to the comparison. In use, the labels from the autoencoder 304 and optionally the images from which the labels were generated may be processed using the machine learning model to obtain line parameters that estimate the tilt and location of the IOL axis.

[0030] The line parameters describing the IOL axis may be processed using an angle calculator 314. The angle calculator 314 compares the line parameters describing the IOL axis with the parameters describing the reference axis and calculates the angular difference. For example, for two lines y1=m1*x+b1 and y2=m2*x+b1, the angular difference may be calculated as ATAN((m1-m2) / (1+m1*m2)). The angle may be adjusted (e.g., subtracted from 180 degrees or pi radians) to obtain the angular difference presented to the surgeon. The toric IOL 200 may, in some embodiments, be safely rotatable in only one direction, for example, clockwise relative to the illustrated toric IOL 200. Any technique may be used to calculate the angular difference between two lines.

[0031] The angle difference and possibly the label and image may be processed using the renderer 316. The renderer 316 may overlay some or all of the following onto the image to obtain an output image: -Representation of reference axes, -Representation of the IOL axis, a direction indicator (e.g., an arrow indicating the rotational direction to move the IOL axis to align with the reference axis), A numerical and / or graphical representation of the angular difference between the reference axis and the IOL axis. For example, a number may be shown indicating the number of degrees to rotate the toric IOL 200 to eliminate the angular difference. In some embodiments, colors are used: red represents a first range of angular difference, yellow represents a second range of angular difference below the first range, and green represents a third range of angular difference below the second range.

[0032] 4A and 4B illustrate a method 400 for providing an alignment guide for positioning a toric IOL 200, according to certain embodiments. Method 400 will be described with reference to the diagrams of FIGS. 5A-5F. Method 400 may be performed by a computing device incorporated within imaging device 302 or a separate computing device coupled to imaging device 302 and receiving images (e.g., a video feed) from imaging device 302. Such a computing device may include one or more processing devices (e.g., central processing units (CPUs)) and one or more memory devices coupled to the one or more processing devices, where the one or more memory devices store executable code including instructions that, when executed, cause the one or more processing devices to perform method 400.

[0033] Method 400 may begin after implantation of toric IOL 200 within capsular bag 114. The steps performed in preparation for implantation of toric IOL 200 and initial placement of toric IOL 200 within capsular bag 114 may be performed according to any technique known in the art.

[0034] Method 400 may include, at step 402, receiving an input image from imaging device 302. In the description of Figures 4A and 4B and 5A-5F, references to an item of anatomy, toric IOL 200, or portion thereof shall be understood to refer to a representation of that item of anatomy, toric IOL 200, or portion thereof in the image, unless otherwise stated. The image received at step 402 may include a patient's eye 100 and toric IOL 200. There may also be an incision 500 where the crystalline lens 112 has been removed and the toric lens IOL has been inserted.

[0035] The method 400 may include, at step 404, obtaining a label for the toric IOL 200 by processing the input image using the autoencoder 304. As shown in FIG. 5B , the label may include one or more pixel masks that include labeled (e.g., non-zero) pixels at pixel locations that correspond to features of the toric IOL 200. For example, the label may include a labeled pixel 502 for the mark 212, a labeled pixel 504 for the perimeter of the peripheral ring 204, and a labeled pixel 506 for one or more bases 208 of the haptics 206. Labeled pixels may also be present for the ring 214, the incision 500, or any item of anatomical structure of the eye 100 visible in the input image.

[0036] The method 400 may include, at step 406, obtaining an orientation of the toric IOL 200 from the labels or from both the labels and the input image using the autoencoder 304 as described above. The orientation may be in the form of line parameters (e.g., slope and x- or y-intercept) that describe the IOL axes of the toric IOL 200.

[0037] Method 400 may include acquiring a reference axis of the treatment plan in step 408. For example, the reference axis may be received from the imaging device 302 or may be acquired from a memory or storage device of the computing device performing method 400. The reference axis may be the result of a transformation performed during the registration step as described above. The reference axis acquired in step 408 may be in the form of line parameters (e.g., slope and x- or y-intercept) that describe the reference axis with respect to the input image. The reference axis may be acquired by performing a transformation of the reference axis defined in the treatment plan with respect to the reference image, as described above.

[0038] The method 400 may include generating an output image that includes the input image overlaid with a representation of some or all of the label, the reference axis, the IOL axis, the direction indicator, and a representation of the angular difference between the reference axis and the IOL axis, at step 410. The output image may then be displayed on the display device 308, at step 412.

[0039] For example, as shown in FIG. 5C , the output image generated in step 410 and displayed in step 412 may include some or all of a line 508 indicating the reference axis of the imaging device 302, a line 510 indicating the IOL axis, and a direction indicator 512 indicating the direction in which the toric IOL 200 can be rotated to align the IOL axis with the reference axis, i.e., the direction of the minimum angle between the IOL axis and the reference axis. Some toric IOLs 200 can only be safely rotated in one direction due to the shape of the haptics 206. Thus, the direction indicator 512 always points in that direction, e.g., clockwise for the illustrated toric IOL 200. The output image may further include an indicator 514 in the form of a number, text, or other symbolic representation (e.g., color, as described above) that conveys one or both of the amount of rotation required ("X1 degree") and the direction of rotation required (clockwise (CW) or counterclockwise (CCW)). In some implementations, the output image may include a number, text, or other symbolic representation 516 of the refractive error. The amount by which misalignment between the reference axis and the IOL axis affects the patient's vision is proportional to the patient's degree of astigmatism. Therefore, a representation 516 of refractive error can be calculated as a function of the degree of astigmatism and the angular difference and overlaid on the output image to inform the surgeon when the toric IOL 200 is sufficiently aligned.

[0040] 4A, method 400 may be repeated. Method 400 may be repeated periodically for every frame of the video feed from imaging device 302, or at larger intervals, such as every N frames, where N is a value greater than or equal to 1. The interval between frames at which method 400 is performed may be from 1 millisecond to 2 seconds. This interval may depend on the processing power available to perform method 400 and may be smaller or larger than the values ​​indicated herein.

[0041] For example, as shown in FIG. 5D, after displaying the output image shown in FIG. 5C, a surgeon using a surgical instrument 518, such as an IOL fixation hook or other instrument, may rotate the toric IOL 200 as instructed. Method 400 may be repeated, resulting in the updated result image shown in FIG. 5D. As can be seen, the IOL axis (represented by line 510) is now more closely aligned with the reference axis (represented by line 508).

[0042] Figure 5D further illustrates a scenario that may occur during alignment: the device 518 in Figure 5D obscures the mark 212 on one side of the peripheral ring 204 (see Figure 5A). In other scenarios, the mark 212 or the base 208 of the haptics may be located below the iris 106 and not visible in the input image.

[0043] Such a scenario may be handled in various ways, described below with reference to FIG. 4B . The process of generating an output image in step 410 may include receiving input data, which may include some or all of receiving an input image in step 414, receiving labels output by the autoencoder 304 in step 416, and receiving predicted labels output by the tracker 310 in step 418. In some implementations in which a tracker 310 is used, only the labels output by the tracker 310 are received in step 418. As mentioned above, the labels from the tracker 310 may predict the location of some features due to the mark 212 obscured by the instrument 518, the patient's iris 106, or some other cause.

[0044] The input data may then be processed in step 420, for example, using the line generator 312, which outputs line parameters describing the IOL axis in step 422. As described above, the line generator 312 may be embodied as a logistic regression model or other machine learning model. Because the labels (with or without predictions from the tracker 310) may contain sufficient information to define a line, the line generator 312 may use unambiguous features represented in the labels to estimate the toric IOL axis with greater accuracy than a human. For example, a set of marks 212 on only one side of the peripheral ring 204 may be sufficient, alone or in combination with only one base 208 of the haptics 206.

[0045] 5E illustrates an output image that may be displayed according to method 400 after an iteration of method 400 in which the angular difference between the IOL axis and the reference axis is within a predetermined tolerance. For example, the predetermined tolerance may be a predetermined angular difference, such as an angle between 1 and 0.1 degrees. The predetermined tolerance may be defined in terms of refractive error, and the tolerance is met if the angular difference and astigmatic refractive error of eye 100 are below a predetermined error threshold.

[0046] If the angular difference meets a predetermined tolerance, the output image may include an indicator 520 that conveys that no further rotation is required (NRR = No Rotation Needed). The output image may include or omit labeled pixels 502, 504, 506, line 508, and line 510 if the angular difference meets a predetermined tolerance.

[0047] If the angular difference meets a predetermined tolerance, as shown in FIG. 5F, the surgical instrument 518 may then be withdrawn and any post-operative procedures may be performed as known in the art.

[0048] The above description is provided to enable any person skilled in the art to practice the various embodiments described herein. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments. Accordingly, the claims are not intended to be limited to the embodiments shown herein, but are to be accorded the full scope consistent with the language of the claims.

Claims

1. 1. A method for providing alignment guidance during ophthalmic surgery, comprising: (a) receiving, by a computing device, from an imaging device, an input image of a patient's eye having a toric intraocular lens (IOL) in the patient's eye; (b) acquiring, by the computing device, a reference axis of the patient's eye, the reference axis indicating a desired orientation of a toric IOL axis of the toric IOL; (c) processing, by the computing device, the input image to obtain feature labels indicating locations of features of the toric IOL represented in the input image, the features including any of alignment dots defined on the toric IOL, a periphery of the toric IOL, and a portion of a haptic portion of the toric IOL; (d) processing, by the computing device, the feature labels to identify an orientation of the toric IOL axis; (e) calculating, by the computing device, an angular difference between the toric IOL axis and the reference axis; (f) generating, by the computing device, an output image including at least one indicator corresponding to the angular difference; (g) outputting the output image to a display device; A method comprising:

2. (h) orienting the toric IOL by a surgeon; (i) repeating (a) through (g); The method of claim 1 further comprising:

3. After carrying out (h) and (i), determining, by the computing device, that the angular difference meets a predetermined tolerance; In response to determining that the angular difference meets the predetermined tolerance, outputting, by the computing device, an indicator on the display device indicating that no further rotation of the toric IOL is required; and The method of claim 2 further comprising:

4. The method of claim 3 , wherein determining that the angular difference meets the predetermined tolerance comprises determining that a refractive error resulting from the angular difference meets the predetermined tolerance.

5. 5. The method of claim 4 , wherein (c) further comprises: processing the input image to obtain one or more bounding boxes that contain the feature; and using the one or more bounding boxes to obtain the feature label.

6. The method of claim 1 , wherein processing the feature labels to identify the orientation of the toric IOL axis comprises generating line parameters that describe a line passing through the alignment dots.

7. The method of claim 6 , wherein processing the feature labels to identify the orientation of the toric IOL axis further comprises processing the feature labels using a machine learning model.

8. The method of claim 7 , wherein the machine learning model is a logistic regression model.

9. The method of claim 1 , wherein the at least one indicator is one or more numbers representing the angular difference.

10. The method of claim 1 , wherein the at least one indicator is one or more numbers representing a refractive error corresponding to the angular difference.

11. receiving, by the computing device, a video feed from the imaging device, the video feed comprising a plurality of frames; performing (a)-(c) using each frame of the plurality of frames as the input image; tracking, by the computing device, the features of the plurality of frames using a tracking algorithm to obtain a predicted label for each frame of the plurality of frames, the predicted label for one or more frames of the plurality of frames including one or more representations of the features not represented in the feature labels obtained for the one or more frames of the plurality of frames; The method of claim 1 further comprising:

12. The method of claim 1 , wherein the imaging device is a digital microscope.

13. matching, by the computing device, the ocular anatomy represented in the input image to a treatment plan to identify an orientation of the patient's eye; determining, by the computing device, an orientation of the reference axis according to the treatment plan and the orientation of the patient's eye; The method of claim 1 further comprising:

14. 1. A system for providing alignment guidance during ophthalmic surgery, comprising: An imaging device; A display device; a computing device including one or more processing devices and one or more memory devices storing executable code, the executable code, when executed by the one or more processing devices, causing the one or more processing devices to: (a) receiving an input image of a patient's eye having a toric intraocular lens (IOL) in the patient's eye from the imaging device; (b) obtaining a reference axis of the patient's eye, the reference axis indicating a desired orientation of a toric IOL axis of the toric IOL; (c) processing the input image using a machine learning model to obtain feature labels indicating locations of features of the toric IOL represented in the input image, the features including any of alignment dots defined on the toric IOL, a periphery of the toric IOL, and a portion of a haptic portion of the toric IOL; (d) processing the feature labels to identify the orientation of the toric IOL axis; (e) calculating the angular difference between the toric IOL axis and the reference axis; (f) generating an output image including at least one indicator corresponding to said angular difference; (g) outputting the output image to the display device; A system that further enables the above.

15. The executable code, when executed by the one or more processing devices, causes the one or more processing devices to: receiving a video feed from the imaging device, the video feed including a plurality of frames; Repeating (a) to (g) by periodically using at least some of the frames as the input image; The system of claim 14 , further comprising:

16. The executable code, when executed by the one or more processing devices, causes the one or more processing devices to: using a tracking algorithm to track the features of the plurality of frames to obtain a predicted label for each frame of the plurality of frames, the predicted label for one or more frames of the plurality of frames including one or more representations of the features not represented in the feature labels obtained for the one or more frames of the plurality of frames. The system of claim 15 further comprising:

17. The executable code, when executed by the one or more processing devices, causes the one or more processing devices to: determining that the angular difference satisfies a predetermined tolerance; In response to determining that the angular difference meets the predetermined tolerance, outputting an indicator on the display device indicating that no further rotation of the toric IOL is required; and The system of claim 15 further comprising:

18. 15. The system of claim 14, wherein the machine learning model is further configured to identify one or more bounding boxes that highlight the feature and to obtain the feature label using the one or more bounding boxes.

19. the machine learning model is a first machine learning model; 15. The system of claim 14, wherein the executable code, when executed by the one or more processing devices, further causes the one or more processing devices to process the feature labels using a second machine learning model to identify the toric IOL axis orientation.

20. 20. The system of claim 19, wherein the second machine learning model is a logistic regression model.

Images