Toric intraocular lens alignment guidance

Abstract

Particular embodiments disclosed herein provide an alignment guide for aligning toric IOLs during surgery. An image having a reference axis is obtained, such as from a digital microscope, and processed, such as using an automatic encoder, to identify alignment marks on the IOL and possibly other features of the IOL. The identification is processed, such as using a logistic regression model, to estimate an IOL axis of the IOL that intersects the alignment mark. An output image is generated from an image on which guides to the surgeon (such as a line representing the IOL axis, a direction of rotation indicator, and a number representation of differences between the reference axis and the IOL axis, or other representation) are superimposed. Tracking of features of the IOL may be performed across multiple images to predict locations of features that are not represented in a particular image.

Description

Technical Field

[0001] The present disclosure relates generally to methods for treating cataracts and, more particularly, to treatments involving the use of an intraocular lens (IOL). Background Art

[0002] Light received by the human eye passes through the transparent cornea that covers the iris and pupil of the eye. The light travels through the pupil and is focused by the lens, which is located behind the pupil in a structure called the capsular bag. The light is focused by the lens onto the retina, which consists of rods and cones, which generate nerve impulses in response to light.

[0003] Due to age or disease, the lens may become cloudy, a condition known as a cataract. Cataracts are easily treated by removing the lens and inserting an artificial lens, known as an intraocular lens (IOL). An IOL can be made to additionally correct aberrations of the patient's eye, such as astigmatism. Since astigmatism is a result of eye asymmetry, the IOL must be aligned with the asymmetry of the eye to compensate for the astigmatism. Therefore, the IOL is provided with markings, such as a row of dots at the periphery of the IOL, which define an axis that can be used to align the IOL. The IOL can be implemented as a toric IOL, which includes a spring-like arm (known as a haptic) that holds the IOL in place within the capsular bag. In existing methods, an imaging device, such as a digital marking microscope (DMM), is used to observe the patient's eye during surgery. The image output by the imaging device has a reference axis superimposed thereon, which corresponds to the desired orientation of the IOL axis.

[0004] Since precise alignment of the IOL axis with the reference is desirable, methods for facilitating such alignment would greatly improve outcomes for the patient. Summary of the invention

[0005] The present disclosure generally relates to a system that provides an alignment guide for positioning a toric IOL in a patient's eye.

[0006] Specific embodiments disclosed herein provide a method and corresponding apparatus for providing alignment guidance during ophthalmic surgery. The method includes receiving, by a computing device, an input image of a patient's eye from an imaging device, the patient's eye having a toric intraocular lens (IOL). The computing device obtains a reference axis for the patient's eye, the reference axis indicating the desired orientation of the toric IOL axis of the toric IOL. The input image is processed to obtain a feature identifier, the feature identifier indicating the location of a feature of the toric IOL represented in the input image, the features including any of the following: an alignment point defined on the toric IOL, a perimeter of the toric IOL, and a portion of the haptic of the toric IOL. The feature identifier is processed by the computing device to determine the orientation of the toric IOL axis. Then, the method includes calculating an angular difference between the toric IOL axis and the reference axis. Then, the computing device generates and outputs an output image including at least one indicator corresponding to the angular difference.

[0007] The following description and the associated drawings set forth in detail certain illustrative features of the one or more embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Certain aspects of one or more embodiments are depicted in the accompanying drawings, and therefore should not be considered to limit the scope of the disclosure.

[0009] Figure 1 Shows the anatomy of the human eye.

[0010] Figure 2 A toric IOL is shown.

[0011] Figure 3A An alignment guide system for positioning a toric IOL is presented in accordance with certain embodiments.

[0012] Figure 3B A post-processor for an alignment guidance system according to certain embodiments is presented.

[0013] Figure 4A Methods for providing an alignment guide for positioning a toric IOL are presented in accordance with certain embodiments.

[0014] Figure 4B is a method for performing post-processing according to certain embodiments.

[0015] FIG. 5A to FIG. 5F Providing an alignment guide on an image of a patient's eye when performing placement of a toric IOL according to certain embodiments is demonstrated.

[0016] To facilitate understanding, identical reference numerals have been used, where possible, to designate identical elements that are common to the figures. It is contemplated that elements and features of one embodiment may be beneficially incorporated in other embodiments without further recitation. DETAILED DESCRIPTION

[0017] Particular embodiments of the present disclosure provide an alignment guide for positioning a toric IOL in a patient's eye.

[0018] Figure 1 1 is a diagram showing portions of a human eye 100, which can be understood relative to a front side (light enters the eye from the front side) and a back side opposite the front side. On the front side of the eye 100, a thin transparent layer called the cornea 102 is connected to the sclera 104, which forms a generally spherical wall of the eye 100. The cornea 102 and the sclera 104 are connected by a ring called the limbus. The iris 106 (the colored part of the eye) and the opening defined by it, the pupil, are located behind the cornea and are visible because the cornea 102 is transparent. The retina 108 is formed on the inner surface of the sclera 104 opposite the cornea 102 and the iris 16. The volume defined by the sclera 104 is occupied by the transparent jelly of the vitreous body 110.

[0019] The lens 112 is a transparent, biconvex structure in the eye that, along with the cornea 102, helps refract light so that it is focused on the retina 108. The lens 112 acts to change the focal length of the eye by changing its shape so that it can focus on objects at various distances, thereby allowing a sharp, real image of the object of interest to be formed on the retina 108. This adjustment of the lens 112 is called accommodation, and is similar to how a camera focuses by moving its lens.

[0020] The lens 112 is located behind the iris 106 in a capsular bag 114. The capsular bag 114 is attached at its perimeter to the suspensory ciliary ligaments 116. The ciliary ligaments 116 attach the capsular bag 114 to the ciliary body 118. The ciliary body 118 is a circular muscle that attaches the ciliary ligaments 116 to the sclera 104 and can contract or relax to change the shape of the lens 112.

[0021] IOLs may be used to treat a variety of diseases and conditions of the lens 112. By way of example, and not necessarily limitation, IOLs according to embodiments of the present disclosure may be used to treat cataracts, myopia (nearsightedness), hyperopia (hyperopia), and large optical errors in astigmatism, ectopia lentis, aphakia, pseudophakia, and nuclear sclerosis. However, for purposes of description, IOL embodiments of the present disclosure are described with reference to cataracts, which often occur in the elderly population.

[0022] Figure 2An exemplary toric IOL 200 is shown. The toric IOL 200 includes a lens portion 202 that focuses light passing through the iris 106 onto the retina 108. The lens portion 202 may be surrounded by a peripheral ring 204 that is not used to focus light. Two or more haptics 206 may be secured to the peripheral ring 204. Each haptic 206 may include a base 208 secured to and extending outwardly from the peripheral ring 204. Spring arms 210 are secured to the bases and extend outwardly from and circumferentially around the peripheral ring 204. In use, the spring arms 210 push outwardly against the capsular bag 114 and hold the toric IOL 200 in a desired position.

[0023] Markings 212 may be formed on the peripheral ring 204. Markings 212 facilitate alignment of the IOL with the patient's eye 100. Markings 212 in the illustrated toric IOL 200 include two groups of points (e.g., depressions or projections), such as dots, formed on the peripheral ring 204 that are opposite to each other. For example, each group may include two, three or more points. The points of each group may be collinear with each other and may be collinear with the points of the other group. A line passing through one or both groups of points (hereinafter referred to as the "IOL axis") may also intersect and be perpendicular to the optical axis of the lens portion 202. In the illustrated toric IOL 200, the IOL axis also intersects the base 208 of the haptic 206. Some toric IOLs 200 are multifocal. The lens portion 202 may include rings 214 that define boundaries between regions of the lens portion 202 that have different focal lengths. As discussed below, the markers 212 can be detected using a machine learning model and used to determine the IOL axis. Thus, the markers 212 need not intersect the IOL axis and can include any arbitrary pattern visible in an image of the IOL 200. The machine learning model can be trained to recognize any shape, arrangement, and number of markers 212. Also as discussed below, features other than the markers 212 (such as the base 208 of the haptics, the perimeter of the IOL 200) can be used to determine the orientation of the IOL 200. The geometric relationship between any two or more features can be used to determine the orientation of the IOL 200 using a machine learning model as outlined below.

[0024] Figure 3AAn alignment guide system 300a for positioning a toric IOL 200 in a patient's eye 100 is shown. The alignment guide system 300a may include an imaging device 302 for capturing images of the patient's eye 100 during implantation and alignment of the toric IOL 200. The imaging device 302 may be implemented as a digital 3D microscope, such as the ALCONNGENUITY 1.5 (a digital 3D digital marker microscope (DMM) with integrated image guidance) or the ZEISS ARTEVO, or as an analog microscope with image guidance, such as the ALCON VERION DIGITAL MARKER or the ZEISS CALISTATO. As is known in the art, the imaging device 302 may be programmed with a treatment plan including preoperative images of the patient's eye 100, which may include images of the sclera 104, the iris 106, and possibly an image of a portion of the retina 108. The imaging device 302 may be further configured to define a reference axis based on one or more images of the patient's eye 100. The imaging device 302 can be programmed to perform registration of the patient's eye 100. Registration can be performed by capturing an image of the patient's eye 100 and matching ocular anatomical structures (such as unique patterns of blood vessels on the sclera 104 and / or retina 108 and / or unique patterns of the limbus and / or iris 106 represented in the image) with representations of corresponding ocular anatomical structures in one or more preoperative images to determine the orientation of the eye 100. The imaging device 302 can then determine a transformation that relates the orientation of the eye 100 in the image to the orientation of the eye in a reference image included in the treatment plan. The reference axis of the treatment plan can then be rotated and / or translated according to the transformation. References herein to a reference axis superimposed on an image of the patient's eye should be understood to refer to a reference axis resulting from the transformation. Registration and corresponding transformations of the reference axis can be repeatedly performed for images (e.g., frames of video data) captured throughout the procedure to take into account movement of the patient's eye relative to the imaging device 302.

[0025] In addition to the imaging device, the components of system 300a may be implemented using the computing power of imaging device 302 implemented as a digital microscope. Alternatively, the additional components of system 300a may be implemented by a separate computing device that receives images with reference axes identified from imaging device 302. In still other embodiments, the separate computing device may only receive images from imaging device 302 and perform registration relative to one or more preoperative images to obtain the reference axes as described above.

[0026] The system 300a may include an autoencoder 304. The autoencoder 304 receives images output by the imaging device 302 (these images may be marked with reference axes) and identifies features of the IOL 200 represented in the images. These features may include some or all of the markers 212, the rings 214, the base 208 of the haptics, and the perimeter of the peripheral ring 204. The autoencoder 304 may identify one or more of these features. The identification may be in the form of one or more pixel masks, where non-zero pixels correspond to pixels of the image found by the autoencoder 304 to represent one or more features or feature types. The autoencoder 304 may generate a separate pixel mask for each feature or feature type, such as a mask for the markers 212, a mask for the rings 214, a mask for the base 208 of the haptics, and a mask for the perimeter of the peripheral ring 204. The autoencoder 304 may generate multiple masks, where one or more masks identify multiple features or feature types. Alternatively, a single mask may mark pixels corresponding to any one feature.

[0027] The auto encoder 304 may include an encoder 304a and a decoder 304b, so that the image is input to the encoder 304a, and the output of the encoder 304a is input to the decoder 304b. The output of the decoder 304b may include one or more pixel masks described above. The auto encoder 304 may be implemented using a convolutional neural network (CNN), a deep neural network (DNN), or other types of neural networks. The auto encoder 304 may be replaced with any machine learning model known in the art, which has been trained to perform the tasks attributed to the auto encoder 304 herein.

[0028] The training data for the autoencoder 304 may include, as input, an image of a patient's eye 100, and, as a desired output, one or more pixel masks identifying features of a toric IOL 200 located within the patient's eye 100. For example, during an implantation procedure, a video stream from a DMM may be captured, and a plurality of frames in the video stream may be labeled by a human (e.g., marking pixels corresponding to features represented by the pixel mask with a human-generated pixel mask), such that each frame and its corresponding identification become a training data entry. Such training data entries from a plurality of patients may then be used to train the autoencoder 304 by processing an image of each training data entry with the autoencoder 304, receiving an output from the autoencoder 304, comparing the output of the autoencoder to one or more pixel masks in the training data entry (e.g., evaluating a loss function), and updating the parameters of the autoencoder 304 based on the comparison.

[0029] The autoencoder 304 may include or be used in combination with an attention mechanism that specifies one or more regions in the input image that contain or may contain features (marker 212, base 208 of haptic 206, perimeter of IOL 200, etc.). The attention mechanism may generate bounding boxes that contain or may contain features. The attention mechanism may be another machine learning model, such as another autoencoder, or one or more layers of the autoencoder 304. The attention mechanism may be trained with training data entries, each of which includes an image as input and one or more bounding boxes obtained from a human or computerized identifier as a desired output, each of which identifies one or more features. The training algorithm may then process each training data entry by processing the input image using the attention mechanism to obtain one or more estimated bounding boxes. The one or more estimated bounding boxes may then be compared to the one or more bounding boxes of the training data entry. The training algorithm may then update parameters of the attention mechanism based on the similarity of the one or more estimated bounding boxes to the one or more bounding boxes of the training data entry. The output of the attention mechanism may be an output image that is the input image with pixels contained within the highlighted one or more bounding boxes. The desired features are identified by subsequent layers of the autoencoder 304 within the region highlighted by the bounding box

[0030] The identification output by the autoencoder 304 can be input to a post-processor 306. The post-processor 306 can determine the IOL axis based on the identification and determine the angular difference between the IOL axis and the reference axis. The post-processor 306 can generate an output image based on the image received from the imaging device 302, the output image having information superimposed thereon, such as lines representing the reference axis and the IOL axis, and arrows, text, or other information indicating the amount of rotation required to align the IOL axis with the reference axis.

[0031] The output image generated by post-processor 306 may then be output to display device 308, which may be a screen incorporated into imaging device 302 and viewable through an eyepiece of imaging device 302. Display device 308 may be implemented as a monitor in the room where toric IOL 200 implantation is being performed, a heads-up display worn by the surgeon performing the implant, or another display device.

[0032] Figure 3BAn exemplary implementation of a post-processor 306 is shown. The post-processor 306 may include a tracker 310. The output of the autoencoder 304 may be a series of identifications for each frame of a video stream from the imaging device 302. The tracker 310 may receive these identifications and possibly frames of the video stream and track the movement of features represented by the identifications from one frame to the next. Thus, the tracker 310 may predict the location of features in frames where the features are not identified due to (a) being obscured by a surgical instrument or a portion of the eye 100 (e.g., the iris 106), or (b) not being successfully identified by the autoencoder 304. The tracker 310 may be implemented as a Markov chain, a flow detector, a Kalman filter, or any other tracking algorithm known in the art.

[0033] The output of the tracker 310 for a frame of the video stream may be an identification in the same form as the identification received from the autoencoder 304. For example, the output of the tracker 310 may be a pixel mask ("tracker mask") corresponding to each pixel mask output by the autoencoder 304 ("raw mask"). The tracker mask may include new non-zero pixels at pixel locations that were zero in the raw mask, these new non-zero pixels indicating predicted locations for features or portions of features that were not identified in the raw mask. The tracker 310 may also perform some degree of noise removal or smoothing, which may cause pixel locations in the raw mask and the tracker mask to have different values ​​(zero flipped to non-zero or non-zero flipped to zero).

[0034] The identification output by the tracker 310, or the identification output by the autoencoder 304 if the tracker 310 is not used ("identification"), can be input to the line generator 312. The line generator 312 can be any machine vision algorithm that is trained or programmed to fit a line to the identification of the marker 212. In some embodiments, the line generator 312 is a machine learning model (such as a neural network implementing a logistic regression model) that is trained to generate line parameters (e.g., slope and x or y intercept) based on the identification and possibly based on the image ("image") that generated the identification. As described above, the identification itself can include one or more pixel masks.

[0035] The identification may include more information than is required to define the IOL axis. Only two points are required to define a line, but four, six or more points may be included in the marking 212, and the base 208 of the haptic may also be represented in the image. The machine learning model of the line generator 312 may advantageously use some or all of this information to accurately define a line representing the IOL axis. The training data entry for the machine learning model may include as input the identification and possibly an image corresponding to the identification, and as desired outputs the human-generated parameters (slope and x or y intercept) of the IOL axis of the toric IOL represented in the image. For each training data entry, the machine learning model may be used to process the input to obtain estimated line parameters. The estimated line parameters may be compared to the parameters of the training data entry (e.g., a loss function may be evaluated), and the parameters of the machine learning model may be updated based on the comparison. During use, the identification from the autoencoder 304, and possibly the image that generated the identification, may be processed using the machine learning model to obtain line parameters that estimate the slope and position of the IOL axis.

[0036] An angle calculator 314 may be used to process the line parameters describing the IOL axis. An angle calculator 314 compares the line parameters describing the IOL axis with the parameters describing the reference axis and calculates the angle difference. For example, for two lines y1=m1*x+b1 and y2=m2*x+b1, the angle difference may be calculated as ATAN((m1-m2) / (1+m1*m2)). The angle may be adjusted (e.g., subtracted from 180 degrees or Pi radians) to obtain the angle difference to be presented to the surgeon. In some embodiments, the toric IOL 200 may be safely rotated in only one direction, such as clockwise for the toric IOL 200 shown. Any method for calculating the angle difference between two lines may be used.

[0037] A renderer 316 may be used to process the angle difference and possible identification, as well as the image. The renderer 316 may overlay some or all of the following on the image to obtain an output image:

[0038] ·Representation of reference axis;

[0039] Representation of the IOL axis;

[0040] Direction indicator (e.g., an arrow describing the direction of rotation to move the IOL axis to align with the reference axis)

[0041] A numerical value and / or graphical representation of the angular difference between the reference axis and the IOL axis. For example, a number indicating the degree to which the toric IOL 200 should be rotated to eliminate the angular difference may be displayed. In some embodiments, colors are used: red represents a first range of angular differences, yellow represents a second range of angular differences below the first range, and green represents a third range of angular differences below the second range.

[0042] Figure 4A and Figure 4B A method 400 for providing an alignment guide for positioning a toric IOL 200 is shown in accordance with some embodiments. FIG. 5A to FIG. 5F 400 is described in FIG. The method 400 may be performed by a computing device incorporated within the imaging device 302, or by a different computing device coupled to the imaging device 302 and receiving images (e.g., video streams) from the imaging device 302. Such a computing device includes one or more processing devices (e.g., central processing units (CPUs)) and one or more memory devices coupled to the one or more processing devices, the one or more memory devices storing executable code including instructions that, when executed, cause the one or more processing devices to perform the method 400.

[0043] Method 400 may begin after implanting toric IOL 200 in capsular bag 114. The steps performed in preparation for implantation of toric IOL 200 and initial placement of toric IOL 200 in capsular bag 114 may be performed according to any method known in the art.

[0044] Method 400 may include receiving an input image from imaging device 302 at step 402. Figure 4A and Figure 4B as well as FIG. 5A to FIG. 5F In the description of FIG. 402, references to an anatomical item, a toric IOL 200, or a portion thereof, should be understood to refer to a representation of the anatomical item, toric IOL 200, or a portion thereof, in an image, unless otherwise stated. The image received at step 402 may include the patient's eye 100 and the toric IOL 200. There may also be an incision 500 through which the lens 112 is removed and the toric lens IOL is inserted.

[0045] The method 400 may include obtaining an identification of the toric IOL 200 by processing the input image using the autoencoder 304 at step 404. Figure 5BAs shown, the identification may include one or more pixel masks that include identified (e.g., non-zero) pixels at pixel locations corresponding to features of the toric IOL 200. For example, the identification may include identification pixels 502 of the marker 212, identification pixels 504 of the perimeter of the peripheral annulus 204, and identification pixels 506 of one or more bases 208 of the haptics 206. The identification pixels may also be present in the annulus 214, the incision 500, or any anatomical item of the eye 100 that is visible in the input image.

[0046] The method 400 may include obtaining the orientation of the toric IOL 200 based on the identification, or based on both the identification and the input image, using the autoencoder 304 as described above at step 406. The orientation may be in the form of line parameters (e.g., slope and x or y intercept) describing the IOL axis of the toric IOL 200.

[0047] The method 400 may include obtaining a reference axis for the treatment plan at step 408. For example, the reference axis may be received from the imaging device 302, or may be retrieved from a memory device or storage device in a computing device executing the method 400. The reference axis may be the result of a transformation performed during the registration step as described above. The reference axis obtained at step 408 may be in the form of line parameters (e.g., slope and x or y intercept) that describe the reference axis relative to the input image. As described above, the reference axis may be obtained by performing a transformation on a reference axis defined in the treatment plan relative to the reference image.

[0048] The method 400 may include generating an output image at step 410, the output image including an input image with some or all of the following superimposed thereon: a logo, a reference axis, an IOL axis, a direction indicator, and a representation of an angular difference between the reference axis and the IOL axis. Then, at step 412, the output image may be displayed on the display device 308.

[0049] For example, Figure 5CAs shown, the output image generated at step 410 and displayed at step 412 may include some or all of the following: a line 508 indicating the reference axis of the imaging device 302, a line 510 indicating the IOL axis, and a direction indicator 512 indicating the direction to rotate the toric IOL 200 so that the IOL axis is aligned with the reference axis (i.e., in the direction of the minimum angle between the IOL axis and the reference axis). Due to the shape of the haptics 206, some toric IOLs 200 can only be safely rotated in one direction. Therefore, the direction indicator 512 will always point to that direction, for example, clockwise for the toric IOL 200 shown. The output image may further include an indicator 514 in the form of a number, text, or other symbolic representation (e.g., a color as described above) that conveys one or both of the desired rotation amount ("X1 degrees") and the desired rotation direction (clockwise (CW) or counterclockwise (CCW)). In some embodiments, the output image may include a number, text, or other symbolic representation 516 of the refractive error. The amount by which the patient's vision is affected by the misalignment between the reference axis and the IOL axis is proportional to the patient's degree of astigmatism. Thus, a representation 516 of the refractive error can be calculated based on the degree of astigmatism and the angular difference and superimposed on the output image to inform the surgeon when the toric IOL 200 is adequately aligned.

[0050] like Figure 4A As shown, method 400 can be repeated. Method 400 can be repeated periodically for each frame of the video stream from imaging device 302, or can be repeated periodically at larger intervals (e.g., every N frames, where N is a value of 2 or greater). The interval between frames performed by method 400 can be from 1 ms to 2 seconds. The interval can depend on the processing power available to perform method 400, and can be less or greater than the values ​​indicated herein.

[0051] For example, Figure 5D As shown, in the display Figure 5C After the output image is shown, the surgeon using surgical instrument 518 (e.g., an IOL fixation hook or other instrument) can rotate the toric IOL 200 as instructed. Method 400 can be repeated to produce Figure 5D The updated output image is shown. It is apparent that the IOL axis (represented by line 510) is now more closely aligned with the reference axis (represented by line 508).

[0052] Figure 5D Further examples show what might happen during alignment. Note that Figure 5D The instrument 518 in FIG. 10 obscures the marking 212 on one side of the peripheral ring 204 (see FIG. 212 ). Figure 5A). In other cases, the marker 212 or base 208 of the haptic may be located below the iris 106 and not visible in the input image.

[0053] In this case, you can use the following reference Figure 4B The process of generating the output image at step 410 may include receiving input data, wherein receiving the input data may include some or all of the following: receiving the input image at step 414, receiving the identification output by the autoencoder 304 at step 416, and receiving the predicted identification output by the tracker 310 at step 418. In some embodiments using the tracker 310, only the identification output by the tracker 310 is received at step 418. As described above, the identification from the tracker 310 may predict the location of some features, such as a marker 212 that is obscured by the instrument 518, the patient's iris 106, or for some other reason.

[0054] The input data may then be processed at step 420, such as using a line generator 312, which outputs line parameters describing the IOL axis at step 422. As described above, the line generator 312 may be implemented as a logistic regression model or other machine learning model. Since the identification (with or without the prediction from the tracker 310) may include enough information to define a line, the line generator 312 may use the unobstructed features represented in the identification to estimate the toric IOL axis more accurately than a person. For example, a set of markers 212 on only one side of the peripheral ring 204 alone or in combination with the base 208 of only one haptic 206 may be sufficient.

[0055] Figure 5E An output image that can be displayed according to method 400 after an iteration of method 400 is shown, wherein the angular difference between the IOL axis and the reference axis is within a predetermined tolerance. For example, the predetermined tolerance can be a predetermined angular difference, such as an angle from 1 degree to 0.1 degrees. The predetermined tolerance can be defined based on the refractive error: the tolerance is met when the refractive error caused by the angular difference and the degree of astigmatism of the eye 100 is below a predetermined error threshold.

[0056] Where the angular difference meets a predetermined tolerance, the output image may include indicator 520 communicating that no further rotation is required (NRR = No Rotation Required). The output image may include or omit identification pixels 502, 504, 506, line 508, and line 510 when the angular difference meets a predetermined tolerance.

[0057] like Fig. 5F As shown, when the angular difference meets the predetermined tolerance, the surgical instrument 518 can then be withdrawn and any postoperative procedure known in the art can be performed.

[0058] The foregoing description is provided to enable any person skilled in the art to practice the various embodiments described herein. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments. Therefore, the claims are not intended to be limited to the embodiments shown herein, but are to be given the full scope consistent with the language of the claims.

Claims

1. A method for providing alignment guidance during ophthalmic surgery, the method comprising: (a) receiving, by a computing device, an input image of an eye of a patient from an imaging device, the eye of the patient having a toric intraocular lens (IOL) therein; (b) obtaining, by the computing device, a reference axis for the patient's eye, the reference axis indicating a desired orientation of a toric IOL axis of the toric IOL; (c) processing the input image by the computing device to obtain feature identifications indicating locations of features of the toric IOL represented in the input image, the features comprising any of: alignment points defined on the toric IOL, a perimeter of the toric IOL, and portions of haptics of the toric IOL; (d) processing the feature identifier by the computing device to determine the orientation of the toric IOL axis; (e) calculating, by the calculation device, the angular difference between the toric IOL axis and the reference axis; (f) generating, by the computing device, an output image, the output image comprising at least one indicator corresponding to the angular difference; as well as (g) Outputting the output image to a display device.

2. The method of claim 1, further comprising: (h) adjusting the orientation of the toric IOL by the surgeon; as well as (i) Repeat (a) to (g).

3. The method of claim 2, further comprising, after performing (h) and (i): determining, by the computing device, that the angular difference satisfies a predetermined tolerance; and In response to determining that the angular difference satisfies the predetermined tolerance, an indicator is output by the computing device on the display device indicating that no further rotation of the toric IOL is required.

4. The method of claim 3, wherein: Determining that the angular difference satisfies the predetermined tolerance includes determining that a refractive error resulting from the angular difference satisfies the predetermined tolerance.

5. The method of claim 4, wherein: (c) further comprises: processing the input image to obtain one or more bounding boxes including the feature, and using the one or more bounding boxes to obtain the feature identification.

6. The method of claim 1, wherein: Processing the feature identification to determine the orientation of the toric IOL axis includes generating line parameters describing a line passing through the alignment point.

7. The method of claim 6, wherein: Processing the feature identification to determine the orientation of the toric IOL axis further includes processing the feature identification using a machine learning model.

8. The method of claim 7, wherein: The machine learning model is a logistic regression model.

9. The method of claim 1, wherein: The at least one indicator is one or more numbers representing the angular difference.

10. The method of claim 1, wherein: The at least one indicator is one or more numbers representing the refractive error corresponding to the angular difference.

11. The method of claim 1, further comprising: receiving, by the computing device, a video stream comprising a plurality of frames from the imaging device; performing (a) to (c) using each of the plurality of frames as the input image; as well as The computing device tracks features of the multiple frames using a tracking algorithm to obtain a predicted identification for each of the multiple frames, the predicted identification for one or more of the multiple frames including representations of one or more features of the features that are not represented in the feature identifications obtained for one or more of the multiple frames.

12. The method of claim 1, wherein: The imaging device is a digital microscope.

13. The method of claim 1, further comprising: matching, by the computing device, the ocular anatomy represented in the input image with a treatment plan to determine an orientation of the patient's eye; as well as The orientation of the reference axis is determined by the computing device based on the treatment plan and the orientation of the patient's eye.

14. A system for providing alignment guidance during ophthalmic surgery, the system comprising: Imaging device; Display device; A computing device comprising one or more processing devices and one or more memory devices storing executable code that, when executed by the one or more processing devices, further causes the one or more processing devices to: (a) receiving an input image of a patient's eye from the imaging device, the patient's eye having a toric intraocular lens (IOL) therein; (b) obtaining a reference axis for the patient's eye, the reference axis indicating a desired orientation of a toric IOL axis of the toric IOL; (c) processing the input image using a machine learning model to obtain feature identifications that indicate locations of features of the toric IOL represented in the input image, the features comprising any of: alignment points defined on the toric IOL, a perimeter of the toric IOL, and portions of haptics of the toric IOL; (d) processing the feature identifier to determine the orientation of the toric IOL axis; (e) calculating the angular difference between the toric IOL axis and the reference axis; (f) generating an output image, the output image comprising at least one indicator corresponding to the angular difference; and (g) Outputting the output image to the display device.

15. The system of claim 14, wherein: The executable code, when executed by the one or more processing devices, further causes the one or more processing devices to: receiving a video stream from the imaging device, the video stream comprising a plurality of frames; and (a) to (g) are periodically repeated using each frame of at least a part of the plurality of frames as the input image.

16. The system of claim 15, wherein: The executable code, when executed by the one or more processing devices, further causes the one or more processing devices to: Features of the multiple frames are tracked using a tracking algorithm to obtain a predicted identification for each of the multiple frames, the predicted identification for one or more of the multiple frames including a representation of one or more of the features that are not represented in the feature identification obtained for one or more of the multiple frames.

17. The system of claim 15, wherein: The executable code, when executed by the one or more processing devices, further causes the one or more processing devices to: determining that the angular difference satisfies a predetermined tolerance; and In response to determining that the angular difference satisfies the predetermined tolerance, an indicator is output on the display device indicating that no further rotation of the toric IOL is required.

18. The system of claim 14, wherein: The machine learning model is further configured to identify one or more bounding boxes that highlight the feature and to obtain the feature identification using the one or more bounding boxes.

19. The system of claim 14, wherein: The machine learning model is a first machine learning model; and The executable code, when executed by the one or more processing devices, further causes the one or more processing devices to process the feature identification using a second machine learning model to determine the orientation of the toric IOL axis.

20. The system of claim 19, wherein: The second machine learning model is a logistic regression model.

Images