High-speed retina tracking

Abstract

Tracking of a patient's retina or eye movement is useful in treating the patient's eye. Tracking is performed using an imaging system and two tracking processes. The first tracking process may determine eye movement based on a comparison between full frames of an image. The second tracking process may determine eye movement based on a partial frame or strip of an image. The first tracking process waits until a full frame is received, while the second tracking process may be faster since only a portion of an image needs to be received.

Description

[Technical field]

[0001] This application claims priority to Canadian Patent Application No. 3,135,405, entitled "High Speed ​​Retina Tracking," filed October 22, 2021, the entirety of which is incorporated herein by reference.

[0002] This disclosure relates to retinal tracking. More specifically, in some embodiments, this application relates to systems and methods for tracking retinal movement across multiple frames using full-frame and sub-frame tracking. [Background technology]

[0003] Imaging of the eye is important for identifying and possibly treating eye conditions. A variety of imaging techniques can be used to obtain images of the internal compartments of the eye. For example, scanning laser ophthalmoscopy (SLO) imaging can provide two-dimensional images of a portion of the eye, such as the retina or cornea. Optical coherence tomography (OCT) imaging can provide three-dimensional and / or cross-sectional images of a portion of the retina or cornea. Other imaging techniques can be used to obtain images of at least a portion of the fundus.

[0004] Imaging of the eye is used to identify eye conditions requiring treatment, which may be performed using a laser, and specific target locations of the laser beam or pulses are determined from the acquired images.

[0005] Although patient eye movement during treatment is minimized, even slight movement of the patient's eye may cause the laser to be misdirected to the intended treatment location. Retinal tracking technology exists that can be used to track the patient's eye movements, but tracking technology can be relatively slow or errors may accumulate.

[0006] Additional, alternative, and / or improved retinal tracking methods are desirable. Summary of the Invention

[0007] According to the present disclosure, there is provided a method for tracking eye movement of a patient using a scan-based imaging device, the method including receiving a current image strip of a current image frame captured from a scan-based imaging device, determining whether the current image frame is complete, if it is determined that the current image frame is not complete, processing the current image strip to track movement between the current image strip and a corresponding image strip of a previously processed image frame and providing a relative frame transformation of the current image frame for transforming a position of the current image frame to a corresponding position of the previously processed image frame, setting a current transformation based on a combination of the relative frame transformation of the current image frame and an absolute frame transformation of the previously processed image frame for transforming a position of the previously processed image frame to a corresponding position of the initial image frame, and if it is determined that the current image frame is complete, processing the current image frame to track movement between the current image frame and the initial image frame and providing an absolute frame transformation of the current image frame for transforming a position of the current image frame to a corresponding position of the initial image frame according to the tracked movement, setting the current transformation based on the absolute frame transformation of the current image frame.

[0008] In a further embodiment of the method, processing the current image frame to track movement includes determining one or more translations and rotations that transform a position of the current image frame to a corresponding position of the initial image frame using one or more of feature tracking and phase correlation.

[0009] In a further embodiment of the method, processing the current image strip to track movement includes determining one or more translations that transform a position of the current image frame to a corresponding position of a previously processed image frame using one or more of feature tracking and phase correlation.

[0010] In a further embodiment of the method, the method further includes registering a treatment plan including one or more treatment locations of the patient's eye to the initial image frame, applying the current transformation to a next treatment location of the treatment plan to provide an adjusted next treatment location, and treating the next treatment location according to the treatment plan.

[0011] In a further embodiment of the method, treating the next treatment location includes adjusting one or more targeting and focusing elements of the laser delivery system to target the adjusted next treatment location and firing the laser delivery system.

[0012] In a further embodiment of the method, the current image strip comprises a predetermined number of pixel columns captured by a scan-based imager.

[0013] In a further embodiment of the method, the number of pixel columns of the current image strip is determined dynamically.

[0014] In a further embodiment of the method, the number of pixel columns of the current image strip is dynamically determined by receiving a next pixel column of the current image strip, processing the current image strip to provide a trial relative frame transformation, determining whether the trial relative frame transformation is reliably determined, and if the trial relative frame transformation is reliably determined, using the trial relative frame transformation as the relative frame transformation.

[0015] According to the present disclosure there is further provided a non-transitory computer readable medium having stored thereon instructions which, when executed by a processor of a computing device, cause the computing device to perform a method according to any one of the above method embodiments.

[0016] According to the present disclosure there is further provided a computing device including a processor for executing instructions and a memory having stored thereon instructions which, when executed by the processor, cause the computing device to perform a method according to any one of the above method embodiments.

[0017] Further features and advantages of the present disclosure will become apparent from the following detailed description taken in conjunction with the accompanying drawings. [Brief description of the drawings]

[0018] [Figure 1] FIG. 1 shows an imaging and laser treatment system incorporating retinal tracking.

[0019] [Diagram 2] FIG. 2 shows an exemplary image frame and strip according to the present method.

[0020] [Diagram 3] FIG. 3 illustrates a method for retinal tracking and eye treatment.

[0021] [Figure 4] FIG. 4 illustrates a further method of retinal tracking and treatment.

[0022] [Diagram 5] FIG. 5 shows an example timeline of the tracking process.

[0023] [Figure 6] FIG. 6 is a block diagram illustrating an embodiment of a computer hardware system that executes software that implements one or more embodiments of the health testing and diagnostic systems, methods, and apparatus disclosed herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] Although certain preferred embodiments and examples are disclosed below, the subject matter of the invention extends beyond the specifically disclosed embodiments to other alternative embodiments and / or uses, as well as modifications and equivalents thereof. Thus, the scope of the claims appended hereto is not limited to any of the specific embodiments described below. For example, in any method or process disclosed herein, the acts or operations of the method or process may be performed in any suitable order, and are not necessarily limited to the specific disclosed order. Although various operations may be described in sequence as multiple separate operations to aid in understanding a particular embodiment, the order of description should not be construed to imply that these operations are order dependent. Furthermore, structures, systems, and / or devices described herein may be embodied as integrated components or as separate components. For purposes of comparing various embodiments, certain aspects and advantages of these embodiments are described. Not necessarily all such aspects or advantages are achieved by a particular embodiment. Thus, for example, various embodiments may be implemented in a manner that achieves or optimizes one advantage or group of advantages taught herein, but not necessarily achieves other aspects or advantages that may be taught or suggested herein.

[0025] Certain exemplary embodiments will now be described to provide a general understanding of the principles of the structure, function, manufacture, and use of the devices and methods disclosed herein. One or more examples of these embodiments are illustrated in the accompanying drawings. Those skilled in the art will appreciate that the devices and methods specifically described herein and illustrated in the accompanying drawings are non-limiting exemplary embodiments, and the scope of the invention is defined only by the claims. Features illustrated or described in connection with one exemplary embodiment may be combined with features of other embodiments. Such modifications and variations are intended to be within the scope of the technology.

[0026] High-speed retinal tracking methods can be used with linear scanning imagers that capture image frames as multiple scanlines of an image target such as SLO and / or OCT. High-speed retinal tracking includes full-frame tracking, which can track motion between full image frames, and sub-frame tracking, which can track motion across a strip of frames or stream of image data. Full-frame tracking can wait until a complete new frame is captured from the scanning imager and can determine motion between the complete new frame and a reference frame, such as an initial captured frame of the patient's eye, used to register a treatment plan to the patient's eye position. The full-frame tracking process can provide an accurate determination of the patient's eye motion, including accounting for various translations and rotations of the eye.

[0027] However, full-frame tracking alone can be relatively slow, not only because the speed of full-frame tracking depends on when the full frame is available, but also because feature detection, matching, and transformation determination can be computationally expensive when performed on the entire frame.

[0028] The strip tracking process or sub-frame tracking process may track the movement of the eye, retina, vitreous floaters, pupil, lens, sclera, cornea, and / or other positions or structures of the eye by identifying the movement of one or more features across a strip of a currently captured frame and a corresponding strip of a previously captured frame. Strip tracking may process an image strip by identifying one or more features of the strip of a currently captured frame and matching the one or more features with corresponding one or more features of a corresponding strip of a previously captured frame to determine the movement of the one or more features across the strip of a previously captured frame and the currently captured frame. The strip tracking process may determine the translation of the matching of one or more features across the strip or may use other techniques including correlation techniques such as phase correlation to determine the matching movement of one or more features between corresponding strips of different frames. Computing the translation using strip tracking or sub-frame tracking requires less time than computing the translation using full-frame tracking. This is because strip tracking or sub-frame tracking uses fewer degrees of freedom to compute the translation than full-frame tracking. However, due to the fewer degrees of freedom and image strips used to calculate the translation, strip or sub-frame tracking may introduce more errors or be less accurate than full-frame tracking.

[0029] The use of full-frame tracking in combination with strip or sub-frame tracking can significantly increase the speed at which eye movement is tracked using a linear scanning imager. For example, a linear scanning imager can capture a full frame every 32 milliseconds and a strip every 2 milliseconds. Thus, strip or sub-frame tracking can be used to track eye movement approximately every 2 milliseconds. To reduce or eliminate errors in strip or sub-frame tracking, full-frame tracking can be used to compare a full frame to the first frame, eliminating or reducing errors in strip or sub-frame tracking every 32 milliseconds. Although strip or sub-frame tracking and full-frame tracking are described with reference to particular speeds, it should be understood that the particular speeds are merely examples and do not limit the scope of the disclosure.

[0030] Without strip or sub-frame tracking, tracking retinal motion is limited to the speed at which the linear scanning imager can capture a full frame, and without full-frame tracking, errors from strip or sub-frame tracking can cause subsequent treatment systems to become less accurate over time.

[0031] The high-speed retinal tracking can be performed by a computer system connected to or included in the imaging and laser delivery device. The high-speed retinal tracking can be performed by a graphics processing unit (GPU) or custom hardware specifically configured to perform high-speed retinal tracking, which increases the speed at which the high-speed retinal tracking can be performed and decreases the time required to perform the high-speed retinal tracking.

[0032] The high-speed retinal tracking process can be used in a variety of applications, such as, for example, tracking eye movement to determine where to target a laser in a laser eye treatment system. High-speed retinal tracking can additionally or alternatively stabilize or align a stream of captured image frames by determining movement of stationary features within the eye and automatically aligning the stationary features throughout the stream of captured images. Stabilizing or aligning the stream of captured images can improve detection or movement of moving structures throughout the stream of captured images, such as floaters. The high-speed retinal tracking process can be used to calculate movement or displacement of one or more features within the eye to compensate for eye movement or one or more features within the eye throughout the stream of captured images. The calculated movement or displacement can be used as an automatic safety device. The automatic safety device can automatically turn off one or more components of the laser eye treatment system or stop one or more functions of one or more components of the laser eye treatment system. For example, if the calculated movement or displacement exceeds a threshold, the laser treatment system can prevent or stop firing a laser. Although described specifically with reference to a laser eye treatment system, the high speed retinal tracking process can be used in any application where an image of the eye is captured using a scanning imager.

[0033] FIG. 1 illustrates an imaging and laser treatment system 100 incorporating retinal tracking. The system 100 can include an imaging and laser delivery device 102. The imaging and laser delivery device 102 can include an SLO imaging component 104, an OCT imaging component 106, and / or a therapeutic laser delivery component 108. In some embodiments, the SLO imaging component 104, the OCT imaging component 106, and the therapeutic laser delivery component 108 can include a focusing component, a light generating component, an image sensor, and / or any other components. The SLO imaging component 104, the OCT imaging component 106, and / or the therapeutic laser delivery component 108 can be controlled by a device controller 110. Light generated by the SLO imaging component 104, the light generated by the OCT imaging component 106, and / or the light or laser generated by the therapeutic laser delivery component 108 can be delivered to an eye 112 to be imaged and / or treated, or to other subjects as the case may be. Light generated by the SLO imaging component 104 and / or the OCT imaging component 106 can be reflected from a portion of the eye 112, such as the retina, back to a detector of the corresponding imaging component.

[0034] The device controller 110 can be in wired or wireless communication with the computing device 114, and the device controller interfaces between the imaging and laser delivery device 102 and the computing device 114. In some embodiments, the computing device 114 operates the imaging and laser delivery device 102 via a system control function 116. Although the computing device 114 is shown as a separate computing device 114, in some embodiments the computing device 114 may be part of or integrated into the imaging and laser delivery device 102. In some embodiments, the SLO imaging component 104 and / or the OCT imaging component 106 can transmit data to the device controller 110. The data can include position data, one or more coordinates, image data, depth data, orientation of one or more mirrors of the SLO imaging component 104 and / or the OCT imaging component 106, or other data.

[0035] The computing device 114 may include one or more processing devices (not shown) for executing instructions and one or more storage devices (not shown) for storing data and instructions. The one or more processing devices may execute instructions to operate the imaging and laser delivery device 102 via a system control function 116. In some embodiments, the one or more processing devices may include a graphics processing unit (GPU), a central processing unit (CPU), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a microcontroller (MCU), and / or other hardware processing devices. The system control function 116 may include a graphical user interface (GUI) function 118 that provides a GUI for operating the imaging and laser delivery device.

[0036] As shown, the GUI functionality 118 may include a zoom registration functionality 120. A user may use the zoom registration functionality 120 to zoom in on the SLO imaging component 104 and / or the OCT imaging component 106 and maintain registration of points on the zoomed-in image with corresponding points on the other imaging components or the laser delivery device 108. In some embodiments, the SLO imaging component 104, the OCT imaging component 106, and / or the laser delivery device 108 may use one or more coordinate systems. The zoom registration functionality 120 may automatically perform a transformation between the coordinate systems to provide registration across the SLO imaging component 104, the OCT imaging component 106, and the laser delivery device 108. For example, a first transformation may map the zoomed-in coordinates of the SLO imaging component 104 to the zoomed-out coordinates of the SLO imaging component 104, and a second transformation may map the zoomed-out coordinates of the SLO imaging component 104 to the coordinates of the OCT imaging component 104. The GUI functionality 118 can display to the user a zoomed-in view of the image captured by the SLO imaging component 104 and transform points in the zoomed-in view to corresponding points in a display view of the image captured by the OCT imaging component 106 by applying a first transformation and a second transformation to the coordinates of the zoomed-in view of the image captured by the SLO imaging component 106. In some embodiments, the transformation can be a transformation that is pre-determined based on the zoom level of the SLO imaging component 104. Based on the zoom level of the SLO imaging component 104 and / or the OCT imaging component 106, the zoom registration functionality 120 can dynamically determine the transformation in substantially real-time. In some embodiments, the zoom registration functionality 120 can maintain registration of points on the zoomed-in SLO image with corresponding points of the laser delivery device 108 and / or the zoom registration functionality 120 can maintain registration of points on the zoomed-in SLO image with corresponding points of the laser delivery device 108.

[0037] In some embodiments, the system control functionality 116 can include a calibration functionality 122. The calibration functionality 122 can align and correlate the SLO imaging component 104, the OCT imaging component 106, and the therapeutic laser delivery component 108 so that images captured by the SLO imaging component 104 and locations within the images captured by the OCT imaging component are aligned and the laser delivery device 108 can accurately target the location. In some embodiments, the calibration functionality 122 can align and correlate the SLO imaging component 104, the OCT imaging component 106, and the therapeutic laser delivery component 108 using image processing techniques, or the calibration functionality 122 can physically align two or more of the SLO imaging component 104, the OCT imaging component 106, and the therapeutic laser delivery component 108 using various sensors and actuators.

[0038] In some embodiments, the system control functionality 116 can include a planning functionality 124. The planning functionality 124 can generate a treatment plan to treat the ocular condition. The planning functionality 124 can determine the treatment plan using the GUI functionality 118 and / or image data received from the SLO imaging component 104 and the OCT imaging component 106. In some embodiments, the planning functionality 104 can display the image data via the GUI functionality 118, and the planning functionality 104 can include one or more user controls and / or user inputs that enable a user to select one or more treatment locations in the image data. In some embodiments, the planning functionality 104 can automatically detect one or more ocular conditions in the image data using artificial intelligence and / or machine learning. The locations of the one or more ocular conditions in the image data can be one or more treatment locations.

[0039] In some embodiments, the system control functionality 116 can include a therapy functionality 126. The therapy functionality can control the SLO imaging component 104, the OCT imaging component 106, and / or the therapeutic laser delivery component 108 to direct the therapeutic laser delivery component 108 to one or more treatment locations. The therapy functionality 126 can include a tracking functionality 128 that can track the movement of the patient's eye to accurately direct the therapeutic laser delivery component 108 to one or more treatment locations as the patient's eye moves.

[0040] In some embodiments, the GUI functionality 118 can display the generated GUI 132 on the display device 130. Although shown as a separate display, in some embodiments, the display device 130 can be part of or integrated into the imaging and laser delivery device 102. In some embodiments, one or more portions of the generated GUI can change depending on the information that needs or is desired to be displayed to the user. FIG. 1 illustrates a GUI 132 that may be displayed during treatment. For example, the GUI 132 can display an SLO image 134 and an OCT image 136. The SLO image 134 can include an indication of the location of the cross-section of the OCT image 136. In some embodiments, the SLO image 134 and / or the OCT image 136 can include an indication of one or more treatment locations that have not yet been treated and treatment locations that have been treated. In some embodiments, the GUI 130 can include one or more graphic elements 138 to advance the treatment plan, such as starting / stopping the treatment plan or advancing to the next treatment location, as well as treatment plan information that may be relevant to the user.

[0041] The laser delivery device 102 and system 100 shown in FIG. 1 broadly includes optical hardware, control electronics, and software. The components of the laser delivery device 102 and system 100 are described in more detail below. The system 100 can be used to image the eye to identify treatment target areas and perform treatment. The imaging and possible treatments can target a wide range of different eye conditions, including, for example, symptomatic vitreous opacities or vitreous floaters (SVO), age-related macular degeneration (AMD), vitreomacular traction syndrome (VTS), diabetic retinopathy, cataracts, choroidal neovascularization, microaneurysms, glaucoma, epiretinal membrane (REM), retinal breaks and detachments, central or branch vein occlusion.

[0042] 2 illustrates exemplary image frames and strips according to the systems and methods disclosed herein. As described above, high speed retinal tracking can be used to determine transformations or changes throughout the stream of captured images using full frame tracking and real-time or substantially real-time sub-frame tracking through one or more strips of a full frame as the system receives one or more strips. As shown, a first image frame 202a, a second image frame 202b, and a partial image frame 202c can include multiple individual strips 204a-204f.

[0043] Each of the individual strips 204a-204f may include multiple column scans. The individual strips 204a-204f of the image frames 202a-202c are shown as being the same size or number of columns from the scanning device. It will be understood that the size of the individual strips 204a-204f may vary not only within the same frame but also between different frames of the stream of captured images. In some embodiments, the size of the individual strips 204a-204f may be dynamically determined by the computer system based on various factors such as the processing speed and / or processing load of the computer system, which may determine, for example, the time it takes to process the strip, the capture rate of the imaging component capturing the scan rows, the area of ​​the eye covered by the strip, the features within the area of ​​the eye covered by the strip, etc. The corresponding strip of the previous frame may be determined as the strip of the previous frame having the same or similar size and position as the size and position of the strip of the current frame. As shown in the partial image frame 202c, the strip is a column or portion of an image frame captured by the imaging component. However, a strip may be a portion of a train of image frames captured by the imaging component, or may be an entire train of image frames captured by the imaging component, hi some embodiments, a strip may be a continuous data stream of image data captured by the imaging component.

[0044] FIG. 2 shows a current strip 206 being captured. Although the current image frame 202c is shown being captured from the top of the current image frame 202c to the bottom of the current image frame 202c, the imaging component can capture the frame from bottom to top or side to side. In some embodiments, the imaging component can capture alternating image frames in opposite directions. For example, the imaging component can capture a first image frame from top to bottom, a second image frame from bottom to top, a third image frame from top to bottom, etc. In this way, the imaging component does not have to move through an entire image frame to begin capturing the next frame. In some embodiments, the imaging component can capture each point of the image strip sequentially, or the imaging component can capture the entire image strip at once. The image frames can capture images of the patient's eye and can be processed to identify movement of features 208a, 208b, 208c across the frames, as described below with reference to FIGS. 3 and 4.

[0045] In some embodiments, the first image frame 202a may be captured by the imaging component at a first time. In some embodiments, the imaging component may capture the entire first image frame 202a at the first time, or the imaging component may capture one or more of the multiple individual strips 204a-204f at different times. For example, the imaging component may capture the individual strips 204a of the first image frame 202a at a first time and capture the individual strips 204b of the first image frame 202a at a second time after the imaging component has captured the individual strips 204a of the first image frame 202a. In another example, the imaging component may capture the individual strips 204a of the first image frame 202a and the individual strips 204b of the first image frame 202a at a first time, and the imaging component may capture one or more of the uncaptured individual strips 204c-204f of the first image frame 202a at a second time. In some embodiments, after the imaging component captures a first image frame 202a, the imaging component may capture subsequent image frames 202b-202c.

[0046] In some embodiments, multiple imaging components can capture multiple individual strips 204a-204f simultaneously. For example, multiple imaging components can be positioned or aimed such that each of the multiple imaging components can simultaneously capture individual strips or portions of individual strips of the multiple individual strips 204a-204f of the first image frame 202a. In this manner, a majority of the first image frame 202a or the entire first image frame 202a can be captured simultaneously or substantially simultaneously.

[0047] In some embodiments, the imaging component can transmit image data for multiple individual strips 204a-204f and / or the entire image frame 202a-202b to the computer system and / or the computer system's GPU. The imaging component can transmit the image data to the computer system after capturing the entire individual strips 204a-204f, or the imaging component can transmit image data for the portions of the individual strips 204a-204f to the computer system in real time or substantially real time as it captures the portions of the individual strips 204a-204f.

[0048] In some embodiments, the computer system is capable of processing the received image data and performing high speed retinal tracking, as described below with reference to FIGS.

[0049] FIG. 3 illustrates a method of retinal tracking and eye treatment. In some embodiments, the method 300 can be performed by an imaging and laser treatment system as described herein with reference to FIG. 1. In some embodiments, the method 300 can be performed by the imaging and laser treatment system after the imaging and laser treatment system and / or a user determine a treatment plan for the patient. The treatment plan can include one or more treatment locations identified prior to an eye condition in a previously captured image of the patient's eye. The method 300 begins at step 302, where the imaging and laser treatment system can capture an initial image of the patient's eye. At step 304, the imaging and laser treatment system can register an orientation of the patient's eye based on the initial image by identifying a treatment location in the initial image, whereby the imaging and laser treatment system can direct a treatment laser delivery component or treatment laser to a predetermined location for treatment of the eye condition. Once the imaging and laser treatment system is registered to the orientation of the patient's eye, the imaging and laser treatment system can track retinal movement of the patient's eye at step 306 and can use the retinal movement to determine an updated treatment location and update the registration of the imaging and laser treatment system at step 308. The tracking and updating in steps 306 and 308 can be performed continuously so that when performing treatment, the imaging and laser therapy system can target a current location of the ocular condition based on the movement of the retina tracked by the imaging and laser therapy system. The current location of the ocular condition can be the updated treatment location determined in step 308. In step 310, after updating the registration of the imaging and laser therapy system in step 308, the imaging and laser therapy system can treat the ocular condition at the updated treatment location, for example, by firing a laser at the updated treatment location.

[0050] As shown in FIG. 3, the imaging and laser therapy system can use high-speed retinal tracking to track retinal movement of the patient's eye in step 306. High-speed retinal tracking can include full-frame tracking and / or strip or sub-frame tracking. The imaging and laser therapy system can include a scanning imager (e.g., SLO imager, OCT imager, etc.) that generates or captures a full image frame from multiple individual strips or scans, as described above with reference to FIG. 2. In some embodiments, each strip or scan can be a full frame row or a portion of a full frame row. The imaging and laser therapy system can process the strips using high-speed retinal tracking, with each image strip including multiple rows captured by the scanning imager. That is, a full image frame can be formed by multiple strips, and a strip can be formed by multiple row scans. The number of strips within a full frame and the size of the strips can be predefined or dynamically adjusted by the imaging and laser therapy system based on various factors, such as the processing speed and / or processing load of the imaging and laser therapy system's computer system or the computer system's GPU, the capture rate of the scanning imager that captures the scan rows, the area of ​​the eye covered by the strips, and features within the area of ​​the eye covered by the strips.

[0051] In some embodiments, the imaging and laser therapy system can perform high speed retinal tracking using steps 306a-306d. In step 306a, the computer system can receive a strip of image frames. As described above with reference to FIG. 2, the computer system can receive the strip of image frames after the scanning imager captures the entire strip, or the computer system can receive the portion of the strip in real time or substantially real time as the scanning imager captures the portion of the strip.

[0052] In step 306b, the computer system can automatically determine whether the computer system received a complete image frame or a strip of image frames in step 306a. The computer system can receive position data from the scanning imager to determine whether the computer system received a complete image frame or a strip of image frames. In some embodiments, the position data can include a position of a mirror of the scanning imager. The mirror of the scanning imager can direct the light or laser of the scanning imager to a patient's eye position. The computer system can determine whether the patient's position corresponds to a position corresponding to a full image frame. In some embodiments, the position corresponding to a full image frame can be any corner or edge of the image frame. In some embodiments, the position corresponding to a full image frame can be any predetermined position of the image.

[0053] If the computer system receives a full image frame in step 306a, the imaging and laser therapy system may track retinal movement of the patient's eye using full-frame tracking in step 306c, as described below with reference to Figure 4. If the computer system receives a strip of image frames in step 306a, the imaging and laser therapy system may track retinal movement of the patient's eye using sub-frame tracking in step 306d, as described below with reference to Figure 4. In step 306c, the imaging and laser therapy system may track retinal movement of the patient's eye after the initial image is captured in step 302 by comparing the full frame to the initial image and / or one or more image frames previously captured by the scanning imager.

[0054] In step 306d, the imaging and laser therapy system can track retinal movement of the patient's eye by comparing the strip of image frames to the initial image, the corresponding strip of the initial image, one or more image frames previously captured by the scanning imaging device, and / or the corresponding strip of one or more image frames previously captured by the scanning imaging device. As described further below with reference to FIG. 4, in step 306d, the corresponding strip of one or more image frames previously captured by the scanning imaging device can be a full image frame or a portion of one or more image frames already captured by the scanning imaging device when the computer system receives the strip of image frames. The retinal movement of the patient's eye, whether determined from full frame tracking or strip tracking, can be used to update the registration of the imaging and laser therapy system in step 308.

[0055] 4 illustrates a further method of retinal tracking and treatment 400. The method 400 can include a full-frame tracking process 402 and / or a sub-frame tracking process 404. In some embodiments, the sub-frame tracking process 404 can be a strip tracking process.

[0056] At step 406, the computer system may receive the image strip or image data for the image strip. At step 408, the computer system may automatically determine whether a full frame has been received by the computer system. As described above with reference to FIG. 3, the computer system may receive position data from the scanning imager to determine whether the computer system has received a full image frame or a strip of image frames. In some embodiments, the position data may include a position of a mirror of the scanning imager. The mirror of the scanning imager may direct the light or laser of the scanning imager to a position of the patient's eye. The computer system may determine whether the position of the patient corresponds to a position corresponding to a full image frame. In some embodiments, the position corresponding to a full image frame may be any corner or edge of the image frame. In some embodiments, the position corresponding to a full image frame may be any predetermined position of the image. If the computer system determines at step 408 that a full frame has been received, the computer system may use the full frame tracking process 402.

[0057] In some embodiments, the first step in the full frame tracking process 402 is step 410. In step 410, the computer system may pre-process the full frame or a portion of the full frame. The computer system may apply one or more adjustments or transformations to the full frame or a portion of the full frame, such as, for example, sharpening, adjusting white balance, color, contrast, or other image characteristics, removing lens distortion, etc.

[0058] After the computer system pre-processes the full frame in step 410, the computer system may detect one or more features of the patient's eye. The computer system may analyze the full frame in step 412 to determine the location of one or more features of the patient's eye in the full frame. The computer system may determine the location of one or more veins or other features of the retina of the patient's eye. The computer system may use a variety of feature detection techniques or methods including, for example, one or more of edge detection, corner detection, blob detection, and ridge detection.

[0059] After the computer system detects one or more features of the patient's eye in step 412, the computer system may match one or more features of the patient's eye with corresponding one or more features of the patient's eye in an initial frame in step 414. In some embodiments, the initial frame may be a first frame captured by the scanning imager, or the initial frame may be a previously received full image frame. After the computer system matches one or more features of the patient's eye with corresponding features of the patient's eye in the initial frame in step 414, the computer system may determine whether a threshold number of pairs of the one or more features and the corresponding one or more features have been matched in step 416. In some embodiments, the threshold number may be a predetermined number of pairs. The predetermined number of pairs may be a number of pairs necessary to accurately match a full image frame with the initial frame. In some embodiments, the computer system may dynamically determine the threshold number of pairs depending on how much the computer system transformed the previously captured frame, the capture speed of the full frame, the processing power of the computer system, etc. Once the computer system determines that sufficient pairs of features have been matched in step 416, the computer system may determine an absolute transformation in step 418. An absolute transformation is a translation, rotation, resizing, and / or warping applied to the full image frame such that one or more features of the full image frame are aligned with or in the same position as the corresponding one or more features of the initial image frame.

[0060] After the computer system determines the absolute transformation in step 418, the computer system may determine whether the absolute transformation was successfully calculated in step 420. If the computer system determines that the absolute transformation was successfully calculated in step 420, the computer system may evaluate the absolute transformation in step 422 and calculate an absolute transformation threshold in step 424. The transformation threshold may be a translation, rotation, resizing, and / or warping that, if applied to the treatment laser, would cause the treatment laser to fire as a line rather than a point. A transformation that exceeds the transformation threshold may cause unsafe firing of the treatment laser and damage to the patient's eye. If the computer system determines in step 424 that the absolute transformation is below the transformation threshold, the computer system may store the absolute transformation and / or the results of the full frame tracking in the computer system's memory in step 426.

[0061] If the computer system determines in step 416 that a threshold number of one or more feature pairs corresponding to one or more features are not matched, the absolute transform is not successfully calculated in step 420, or the absolute transform exceeds a transform threshold in step 424, the computer system may indicate a tracking failure in step 428. In some embodiments, a tracking failure may prevent firing of the treatment laser.

[0062] If, at step 408, it is determined that a full frame has not been received when the computer system receives the image strip or image data for the image strip, the computer system may use a sub-frame tracking process 404. In some embodiments, the first step of the sub-frame tracking process 404 is step 430. At step 430, the computer system may retrieve a corresponding strip from a previously received full image frame that is stored in the computer system's memory. At step 432, the computer system may determine whether the corresponding strip was successfully retrieved. If the computer system determines that the corresponding strip was successfully retrieved, at step 434, the computer system may pre-process the image strip. The computer system may apply one or more adjustments or transformations to the image strip, such as sharpening, adjusting white balance, color, contrast, or other image characteristics, removing lens distortion, etc.

[0063] After the computer system pre-processes the image strip at step 434, the computer system can calculate the relative transformation at step 436. The computer system can calculate the relative transformation by analyzing the image strip to determine the location of one or more features of the patient's eye in the image strip. The computer system can use various feature detection techniques or methods to determine the location of the one or more features of the patient's eye in the image strip. This can include, for example, one or more of edge detection, corner detection, blob detection, and ridge detection. After the computer system detects the one or more features of the patient's eye, the computer system can match the one or more features of the patient's eye with the corresponding one or more features of the patient's eye in the corresponding image strip. After the computer system matches the one or more features of the patient's eye with the corresponding feature of the patient's eye in the corresponding image strip, the computer system can determine whether a threshold number of pairs of the one or more features and the corresponding one or more features have matched. In some embodiments, the threshold number is a predetermined number of pairs. The predetermined number of pairs is the number of pairs required to accurately match the image strip with the initial image strip. In some embodiments, the computer system may dynamically determine the threshold number of pairs depending on how much the computer system has transformed a previously captured image strip, the capture rate of the image strip, the processing power of the computer system, etc. If the computer system determines that enough feature pairs have been matched, the computer system may determine a relative transformation. The relative transformation may be a translation, rotation, resizing, and / or warping applied to the image strip frame such that one or more features of the image strip are aligned with or located at the same position as the corresponding one or more features of the corresponding image strip. In some embodiments, the relative transformation may be a simplified transformation compared to an absolute transformation. For example, in some embodiments, the relative transformation may include only a translation. In this manner, the computer system may calculate the relative transformation in less time than it would take to calculate an absolute transformation.A relative transformation can be less accurate than an absolute transformation.

[0064] In some embodiments, the computer system may evaluate the relative transform in step 438 and determine in step 440 whether an error in the relative transform is within an acceptable error range and / or whether the relative transform is below a transform threshold. The computer system may determine the error in the relative transform by comparing a position of one or more features of a corresponding image strip to a position of one or more features of the image strip after the relative transform has been applied to the image strip. If the computer system determines in step 404 that the error in the relative transform is within an acceptable error range and / or the relative transform is below a transform threshold, the computer system may determine a sub-tracking absolute transform in step 443. The sub-tracking absolute transform is a combination of the relative transform and the sub-tracking absolute transform that the previous image strip uses for the sub-tracking process 404, or the sub-tracking absolute transform is a combination of the relative transform and the absolute transform determined in the previous full-frame tracking process 410. After the computer system determines the sub-tracking absolute transform in step 442, the computer system may store the results of the sub-tracking absolute transform and / or the sub-frame tracking in the computer system's memory in step 426.

[0065] If the computer system determines that the corresponding strip was not successfully acquired in step 432, or if the computer system determines that the error in the relative transformation is not within an acceptable error range and / or the relative transformation exceeds a transformation threshold, the computer system may indicate a tracking failure in step 432. In some embodiments, a tracking failure may prevent firing of the treatment laser.

[0066] After the computer system saves the absolute transformation or the sub-tracking absolute transformation in step 426, the computer system may set the absolute transformation or the sub-tracking absolute transformation to the latest transformation in step 446. In some embodiments, the computer system may apply the absolute transformation or the sub-tracking absolute transformation to the laser treatment coordinates of the laser target in the current frame to transform the position of the laser target. After the computer system applies the absolute transformation or the sub-tracking absolute transformation to the laser treatment coordinates, the computer system may apply the absolute transformation or the sub-tracking absolute transformation to the coordinates of the OCT imager in step 450. Once the laser target is transformed, the treatment laser is positioned and fired in step 452.

[0067] Although not shown, in some embodiments, method 400 is a feedback loop, and the computer system may continually execute method 400 as image strips are received. The continuous tracking of eye movement in the feedback loop ensures that the treatment laser is positioned or aimed at the correct treatment location on the patient's eye as the patient's eye moves. Although steps 426-452 are described with respect to a treatment laser, it should be understood that method 400 may be used to ensure that a scanning device, such as an OCT imager, is positioned or aimed at the correct location.

[0068] FIG. 5 illustrates an exemplary timeline of the tracking process. As shown, an initial frame is received (502). Although not shown in the figure, the initial frame may be captured or imaged as several strips, as well as additional frames. As shown in the figure, each frame 1-5 may be captured or imaged as a series of strip AFs. The scanning imager may capture each strip as a series of points or may capture the entire strip at one time. Once the computer system receives one of the series of strip AFs, as indicated by arrows 506a-506d and described above with reference to FIG. 4, the computer system may perform sub-frame tracking using the strip AF and the corresponding strip AF of the previous frame. The corresponding strip AF of the previous frame may be the corresponding strip of the last matched full image frame.

[0069] Although strips AF of frames 1 and 2 received before frame 1 was matched are not shown as being matched with the initial frame 502, the strips AF may be matched with corresponding strips of the initial frame.

[0070] As shown, the computer system may take longer to perform full-frame tracking 504a-504d than it would take to receive each of the multiple strip AFs and perform sub-frame tracking 506a-506d using the multiple strip AFs. As shown, after frame 1 is matched using full-frame tracking, strip F of frame 2 is compared to strip F of frame 1 using sub-frame tracking, and strips A and E of frame 3 are compared to corresponding strips A-E of frame 1 using sub-frame tracking, as indicated by arrow 506a. Similarly, each strip is compared to corresponding strips of previously matched frames, as indicated by arrows 506b, 506c, and 506d. The computer system can perform sub-tracking on a series of strip AFs while simultaneously performing full-frame tracking using a recently captured full frame.

[0071] The above process provides high speed retinal tracking where potential eye movements are updated as each strip is received. As explained above with reference to FIG. 4, the sub-tracking process can be significantly faster than full-frame tracking, but the sub-tracking can include errors that can accumulate with each sub-tracking absolute transformation applied. Once a full frame is received and the computer system is able to perform full-frame tracking, the absolute transformations calculated during the full-frame tracking process can be applied to reduce or eliminate error accumulation because movements in the full-frame tracking process are determined using the full frame rather than image strips.

[0072] Although a computer system is described above with reference to FIG. 5, any portion of the computer system, as shown in FIGS. 2-5, may perform any of the methods described herein. In some embodiments, a GPU may perform the methods described herein to improve the speed of execution of high speed retinal tracking. The GPU may perform high speed retinal tracking using parallel processing. In some embodiments, a custom processor may be used to perform any of the methods described herein. The custom processor may include one or more GPUs, CPUs, FPGAs, ASICs, and / or any other hardware. In some embodiments, multiple GPUs and / or multiple custom processors may perform the methods described herein simultaneously on different full frame images or image strips to further improve processing speed.

[0073] FIG. 6 is a block diagram illustrating an embodiment of a computer hardware system that executes software to implement one or more embodiments disclosed herein.

[0074] In some embodiments, the systems, processes, and methods described herein are implemented using a computer system such as that shown in Figure 6. The exemplary computer system 602 communicates with one or more computer systems 20 and / or one or more data sources 622 via one or more networks 618. Although Figure 6 illustrates an embodiment of a computer system 620, it will be recognized that the functionality provided in the components and modules of computer system 620 may be integrated into fewer components and modules or further separated into additional components and modules.

[0075] Computer system 602 may include modules 614 that perform the functions, methods, operations, and / or processes described herein. Modules 614 are executed on computer system 602 by central processing unit 606, which is described further below.

[0076] In general, the term "module" as used herein refers to logic embedded in hardware or firmware, or a collection of software instructions with entry and exit points. Modules are written in a programming language such as JAVA, C or C++, Python, etc. Software modules may be compiled or linked into executable programs, installed in dynamic link libraries, or written in interpreted languages ​​such as BASIC, PERL, LUA, Python, etc. Software modules may be called by other modules or by themselves, or in response to detected events or interrupts. Hardware-implemented modules include connected logic units such as gates and flip-flops, and may include programmable units such as programmable gate arrays and processors.

[0077] In general, modules as described herein refer to logical modules that may be combined with other modules or divided into sub-modules, regardless of physical organization or storage. Modules may be executed by one or more computer systems, stored on or in any suitable computer-readable medium, or implemented in whole or in part in specially designed hardware or firmware. Any of the methods, calculations, processes, or analyses described above may be readily performed using a computer, although not all calculations, analyses, and / or optimizations require the use of a computer system. Additionally, in some embodiments, process blocks described herein may be modified, rearranged, combined, and / or omitted.

[0078] The computer system 602 includes one or more processing units (CPUs) 606, which may include a microprocessor. The computer system 602 further includes physical memory 610, such as random access memory (RAM) for temporary storage of information, read only memory (ROM) for permanent storage of information, and mass storage 604, such as a backup store, hard drive, rotating magnetic disk, solid state disk (SSD), flash memory, phase change memory (PCM), 3D XPoint memory, diskette, or optical media storage device. Alternatively, the mass storage device may be implemented in an array of servers. Typically, the components of the computer system 602 are connected to the computer using a standard-based bus system. The bus system may be implemented using a variety of protocols, such as peripheral component interconnect (PCI), microchannel, SCIC, industrial standard architecture (ISA), and extended ISA (EISA) architectures.

[0079] The computer system 602 includes one or more input / output (I / O) devices and interfaces 612, such as a keyboard, a mouse, a touchpad, a printer, etc. The I / O devices and interfaces 612 may include one or more display devices, such as a monitor, that may visually display data to a user. More specifically, the display devices provide for the display of a GUI, for example, application software data, multimedia presentations, etc. The I / O devices and interfaces 612 may also provide communication interfaces to various external devices. The computer system 602 may include one or more multimedia devices 608, such as, for example, speakers, a video card, a graphics accelerator, a microphone, etc.

[0080] The computer system 602 may run on a variety of computer devices, such as a server, a Windows server, a structured query language server, a Unix server, a personal computer, a laptop computer, etc. In other embodiments, the computer system 602 may run on a cluster computer system, a mainframe computer system, and / or other computer systems suitable for controlling and / or communicating with large databases, performing high volume transaction processing, and generating reports from large databases. The computer system 602 is typically controlled and coordinated by operating system software, such as Windows XP, Windows Vista, Windows 7, Windows 8, Windows 10, Windows 11, Windows Server, Unix, Linux (and its derivatives Debian, Linux Mint, Fedora, Red Hat), SunOS, Solans, Blackberry OS, z / OS, iOS, macOS, or other operating systems, including proprietary operating systems. The operating system provides a variety of functions, such as controlling and scheduling the execution of computer processes, performing memory management, providing file system, network, and I / O services, and providing a user interface, such as a graphical user interface (GUI).

[0081] The computer system 602 shown in FIG. 6 is connected to a network 618, such as a LAN, a WAN, or the Internet, via a communication link 616 (wired, wireless, or a combination thereof). The network 618 communicates with various computer apparatus and / or other electronic devices. The network 618 communicates with one or more computer systems 620 and one or more data sources 622. The module 614 may access and be accessed by the computer systems 620 and / or data sources 622 via a web-enabled user access point. The connection may be a direct physical connection, a virtual connection, and other connection types. The web-enabled user access point may include a browser module that allows the user to view and interact with data over the network 618 using text, graphics, audio, video, and other media.

[0082] Access to the module 614 of the computer system 602 by the computer system 620 and / or the data source 622 may be through a web-enabled user access point such as a personal computer, mobile phone, smart phone, laptop, tablet computer, e-reader device, audio player, or other device capable of connecting to the network 618 of the computer system 620 or data source 622. Such a device may include a browser module implemented as a module that displays data using text, graphics, audio, video, and other media and allows interaction with the data over the network 618.

[0083] The output module may be implemented as a combination of all-points addressable displays such as cathode ray tubes (CRT), liquid crystal displays (LCD), plasma displays, or other types and / or combinations of displays. The output module may be implemented to communicate with input devices 612 and may also include software with an appropriate interface that allows a user to access data using stylized screen elements such as menus, windows, dialog boxes, toolbars, and controls (e.g., radio buttons, checkboxes, sliding scales, etc.). Additionally, the output module may communicate with a set of input and output devices to receive signals from a user.

[0084] Input devices may include keyboards, rollerballs, pens and styluses, mice, trackballs, voice recognition systems, or pre-designated switches or buttons. Output devices may include speakers, display screens, printers, or voice synthesizers. Additionally, touch screens can function as hybrid input / output devices. In another embodiment, the user may interact with the system more directly, such as through a system terminal connected to the score generator without communication over the Internet, WAN, LAN, or similar network.

[0085] In some embodiments, system 602 may include a physical or logical connection established between a remote microprocessor and a mainframe host computer for the express purpose of uploading, downloading, or displaying interactive data and databases online in real time. The remote microprocessor may be operated by an entity operating computer system 602, including a client-server system or a main server system, and / or may be operated by one or more data sources 622 and / or one or more computer systems 620. In some embodiments, a micro-mainframe link may be joined using terminal emulation software on the microprocessor.

[0086] In some embodiments, computer system 620 internal to the entity operating computer system 602 may access module 614 internally as an application or process executed by CPU 606 .

[0087] In some embodiments, one or more features of the systems, methods, and apparatus described herein may use URLs and / or cookies, for example, to store and / or transmit data or user information. A Uniform Resource Locator (URL) may include a web address and / or a reference to a web resource stored in a database and / or a server. A URL may specify the location of a resource on a computer and / or a computer network. A URL may include a mechanism for obtaining a network resource. A source of a network resource may receive a URL, identify the location of the web resource, and return the web resource to the requester. A URL may be translated into an IP address, and a Domain Name System (DNS) may look up the URL and its corresponding IP address. URLs may be references to web pages, file transfers, email, database access, and other applications. A URL may include strings that identify paths, domain names, file extensions, hostnames, queries, fragments, schemes, protocol identifiers, port numbers, usernames, passwords, flags, objects, resource names, and the like. The systems disclosed herein may generate, receive, transmit, apply, parse, serialize, render, and / or act on URLs.

[0088] Cookies, also known as HTTP cookies, web cookies, Internet cookies, and browser cookies, can include data sent by a website or stored on a user's computer. This data is stored by the user's web browser while the user is browsing. Cookies can include useful information that allows a website to remember previous browsing information, such as shopping carts for online stores, button clicks, login information, and / or records of previously visited web pages or network resources. Cookies can also include information entered by a user, such as name, address, password, credit card information, etc. Cookies can also perform computer functions. For example, an authentication cookie can be used by an application (e.g., a web browser) to identify whether a user has already logged in (e.g., to a website). Cookie data can be encrypted to provide security to consumers. Tracking cookies can be used to compile an individual's past browsing history. The system disclosed herein can generate and use cookies to access an individual's data. The system can also generate and use JSON Web Tokens to store credibility information, HTTP authentication as an authentication protocol, IP addresses tracking session or identity information, URLs, etc.

[0089] The computer system 602 may include one or more internal and / or external data sources (eg, data source 622). In some embodiments, one or more of the data repositories and data sources described above may be implemented using relational databases, such as Sybase, Oracle, CodeBase, DB2, PostgreSQL, Microsoft® SQL Server, as well as other types of databases, such as NoSQL databases (such as Couchbase, Cassandra, MongoDB), flat file databases, entity-relationship databases, object-oriented databases (such as InterSystems Cache), cloud-based databases (such as Amazon® RDS, Azure SQL, Microsoft Cosmos DB, Azure Database for MySQL, Azure Database for MariaDB, Azure Cache for Redis, Azure Managed Instance for Apache Cassandra, Google® Bare Metal Solution for Oracle on Google Cloud, Google Cloud SQL, Google Cloud Spanner, Google Cloud Big Table, Google Firestore, Google Firebase Realtime Database, Google Memorystore, Google MongoDB Atlas, Amazon Aurora, Amazon DynamoDB, Amazon Redshift, Amazon ElastiCache, Amazon MemoryDB for Redis, Amazon DocumentDB, Amazon Keyspaces, etc.).

[0090] The computer system 602 may also have access to one or more databases 622. The databases 622 are stored in a database or data repository. The computer system 602 may access the one or more databases 622 over a network 618 or directly access the databases or data repository via the I / O devices and interfaces 612. The data repository that stores the one or more databases 622 may reside within the computer system 602.

[0091] In the foregoing specification, the system and process have been described with reference to specific embodiments thereof. It will be apparent, however, that various modifications and changes can be made thereto without departing from the broader spirit and scope of the embodiments disclosed herein. The specification and drawings are therefore to be regarded in an illustrative sense, rather than a restrictive sense.

[0092] Indeed, while the systems and processes are disclosed in the context of certain embodiments and examples, those skilled in the art will appreciate that various embodiments of the systems and processes extend beyond the specifically disclosed embodiments to other alternative embodiments and / or uses of the systems and processes, as well as obvious modifications and equivalents thereof. Moreover, while several variations of the system and process embodiments have been shown and described in detail, other modifications that are within the scope of the present disclosure will be readily apparent to those skilled in the art based on the present disclosure. It is also contemplated that various combinations or subcombinations of specific features and aspects of the embodiments may be made and still fall within the scope of the present disclosure. It is understood that various features and aspects of the disclosed embodiments can be combined with or substituted for one another to form various modes of embodiments of the disclosed systems and processes. The methods disclosed herein need not be performed in the order described. Thus, it is intended that the scope of the systems and processes disclosed herein should not be limited by the specific embodiments described above.

[0093] It will be understood that the disclosed systems and methods each have several innovative aspects, no single one of which is solely responsible or necessary for the desirable attributes disclosed herein. The various features and processes described above can be used independently of one another or can be combined in various ways. All possible combinations and subcombinations are intended to be within the scope of the present disclosure.

[0094] Certain features that are described herein in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, even if features are described above as functioning in a particular combination and are initially claimed as such, one or more features of the claimed combination may, in some cases, be deleted from the combination, and the claimed combination may be directed to a subcombination or variation of the subcombination. No single feature or group of features is required or essential to all embodiments.

[0095] It will also be understood that conditional language used herein, e.g., "can," "may," "might," "might," "for example," and the like, is generally intended to convey that certain embodiments include certain features, elements, and / or steps, but not other embodiments, unless otherwise indicated or understood within the context in which it is used. Thus, such conditional language is not generally intended to suggest that the features, elements, and / or steps are somehow required for one or more embodiments, or that one or more embodiments necessarily include logic that determines whether those features, elements, and / or steps are included or performed in a particular embodiment, with or without author input or prompting. Terms such as "include," "comprise," "having," and the like are synonymous and are used in an inclusive, open-ended manner and do not exclude additional elements, features, acts, operations, etc. Additionally, the term "or" is used in its inclusive (not exclusive) sense, so that, for example, when used to connect a list of elements, the term "or" may mean one, some, or all of the elements in the list. Additionally, the articles "a," "an," and "the," as used in this application and the appended claims, are to be construed to mean "one or more" or "at least one," unless otherwise specified. Similarly, although operations may be depicted in the drawings in a particular order, it should be recognized that such operations need not be performed in the particular order or sequential order depicted, or that not all of the operations depicted need to be performed, to achieve desirable results. Additionally, the drawings may generally depict one or more exemplary processes in the form of a flow chart. However, the exemplary methods and processes depicted generally may incorporate other operations that are not depicted. For example, one or more additional operations may be performed before, after, simultaneously with, or between the depicted operations. Additionally, operations may be rearranged or reordered in other embodiments. In certain circumstances, multitasking and parallel processing may be advantageous.Moreover, the separation of various system components in the above embodiments should not be understood as requiring such separation in all embodiments, and it will be understood that the program components and systems described may generally be integrated into a single software product or packaged into multiple software products. Moreover, other embodiments are within the scope of the following claims. In some cases, desirable results may be achieved even when the actions recited in the claims are performed in a different order.

[0096] Furthermore, the methods and apparatus described herein are susceptible to various modifications and alternative forms, specific examples of which are shown in the drawings and described in detail herein. However, it will be understood that the embodiments are not limited to the specific forms or methods disclosed, but rather, the embodiments are intended to cover all modifications, equivalents, and alternatives within the spirit and scope of the various described embodiments and the appended claims. Furthermore, the disclosure herein of an embodiment or an embodiment-related specific features, aspects, methods, properties, characteristics, qualities, attributes, elements, etc., can be used with any other embodiment or embodiments described herein. The methods described herein do not have to be performed in the order described. The methods described herein may include specific actions taken by the practitioner, but may also include, explicitly or implicitly, third-party instructions regarding those actions. The ranges described herein include all overlaps, subranges, and combinations thereof. Words such as "up to," "at least," "greater than," "less than," "between," and the like, include the numbers described. Numbers preceded by terms such as "about" or "approximately" are inclusive of the stated number and should be interpreted in the context (e.g., as precisely as reasonably possible under the circumstances, e.g., ±5%, ±10%, ±15%, etc.). For example, "about 3.5 mm" includes "3.5 mm". Phrases preceded by terms such as "substantially" are inclusive of the stated phrase and should be interpreted in the context (e.g., as precisely as reasonably possible under the circumstances). For example, "substantially constant" includes "constant". Unless otherwise specified, all measurements are made at standard conditions, including temperature and pressure.

[0097] As used herein, a phrase referring to "at least one" of a list of items refers to any combination of those items, including single members. For example, "at least one of A, B, or C" is intended to cover "A," "B," "C," "A and B," "A and C," "B and C," and "A, B and C." Conjunctions such as "at least one of X, Y, and Z" will be understood to be used in the context of common usage to convey that an item, term, etc. may be at least one of X, Y, or Z, unless otherwise indicated. Thus, such conjunctions are not intended to suggest that a particular embodiment requires that "at least one of X," "at least one of Y," and "at least one of Z," respectively, be present. Headings provided herein, if any, are for convenience only and do not necessarily affect the scope or meaning of the apparatus and methods disclosed herein.

[0098] Thus, the claims are not intended to be limited to the embodiments shown herein but are to be accorded the widest scope consistent with the present disclosure, the principles and novel features disclosed herein.

Claims

1. 1. A method for tracking eye movement of a patient using a scan-based imager, comprising: receiving a current image strip of a current image frame captured from the scan-based imager; determining whether the current image frame is complete; If it is determined that the current image frame is not complete, processing the current image strip to track movement between the current image strip and a corresponding image strip of a previously processed image frame, and providing a relative frame transformation of the current image frame to transform a position of the current image frame to a corresponding position of the previously processed image frame; setting a current transformation based on a combination of a relative frame transformation of the current image frame and an absolute frame transformation of the previously processed image frame that transforms positions of the previously processed image frame to corresponding positions of an initial image frame; If the current image frame is determined to be complete, processing the current image frame to track motion between the current image frame and an initial image frame, and providing an absolute frame transformation of the current image frame that transforms a position of the current image frame to a corresponding position of the initial image frame according to the tracked motion; setting the current transformation based on an absolute frame transformation of the current image frame; A method comprising:

2. processing the current image frame to track the movement includes determining one or more translations and rotations that transform the position of the current image frame to the corresponding position of the initial image frame using one or more of feature tracking and phase correlation; The method of claim 1.

3. processing the current image strip to track the movement includes determining one or more translations that transform the position of the current image frame to the corresponding position of the previously processed image frame using one or more of feature tracking and phase correlation; The method of claim 1.

4. registering a treatment plan including one or more treatment locations of the patient's eye to the initial image frame; applying the current transformation to a next treatment position in the treatment plan to provide an adjusted next treatment position; treating the next treatment location according to the treatment plan. The method of claim 1.

5. Treating the next treatment area: adjusting one or more targeting and focusing elements of a laser delivery system to target the adjusted next treatment location; firing the laser delivery system. The method of claim 4.

6. the current image strip having a predetermined number of pixel columns captured by the scan-based imager; 6. The method according to any one of claims 1 to 5.

7. the number of pixel columns of the current image strip is dynamically determined; The method of claim 1.

8. The number of pixel columns in the current image strip is receiving a next row of pixels of the current image strip; processing the current image strip to provide a trial relative frame transform; determining whether the trial relative frame transformation was reliably determined; if the trial relative frame transformation is reliably determined, using the trial relative frame transformation as the relative frame transformation; is dynamically determined by The method of claim 7.

9. A non-transitory computer-readable medium that, when executed by a processor of a computing device, causes the computing device to: receiving a current image strip of a current image frame captured from a scan-based imager; determining whether the current image frame is complete; If it is determined that the current image frame is not complete, processing the current image strip to track movement between the current image strip and a corresponding image strip of a previously processed image frame, and providing a relative frame transformation of the current image frame to transform a position of the current image frame to a corresponding position of the previously processed image frame; setting a current transformation based on a combination of a relative frame transformation of the current image frame and an absolute frame transformation of the previously processed image frame that transforms positions of the previously processed image frame to corresponding positions of an initial image frame; If the current image frame is determined to be complete, processing the current image frame to track motion between the current image frame and an initial image frame, and providing an absolute frame transformation of the current image frame that transforms a position of the current image frame to a corresponding position of the initial image frame according to the tracked motion; setting the current transformation based on an absolute frame transformation of the current image frame; storing instructions for performing the method, Computer-readable medium.

10. processing the current image frame to track the movement includes determining one or more translations and rotations that transform the position of the current image frame to the corresponding position of the initial image frame using one or more of feature tracking and phase correlation; 10. The computer-readable medium of claim 9.

11. Processing the current image strip to track motion includes determining one or more translations that transform positions of the current image frame to corresponding positions of the previously processed image frame using one or more of feature tracking and phase correlation.

10. The computer-readable medium of claim 9.

12. The method further comprises: registering a treatment plan including one or more treatment locations of the patient's eye to the initial image frame; applying the current transformation to a next treatment position in the treatment plan to provide an adjusted next treatment position; treating the next treatment location in accordance with the treatment plan.

10. The computer-readable medium of claim 9.

13. Treating the next treatment area: adjusting one or more targeting and focusing elements of a laser delivery system to target the adjusted next treatment location; firing the laser delivery system. The computer-readable medium of claim 12.

14. the current image strip having a predetermined number of pixel columns captured by the scan-based imager; 14. The computer-readable medium of any one of claims 9 to 13.

15. the number of pixel columns of the current image strip is dynamically determined; 10. The computer-readable medium of claim 9.

16. The number of pixel columns in the current image strip is receiving a next row of pixels of the current image strip; processing the current image strip to provide a trial relative frame transform; determining whether the trial relative frame transformation was reliably determined; if the trial relative frame transformation is reliably determined, using the trial relative frame transformation as the relative frame transformation; is dynamically determined by 16. The computer-readable medium of claim 15.

17. 1. A computing device including a processor for executing instructions and a memory on which the instructions are stored, when executed by the processor, The instructions may cause a computing device to: receiving a current image strip of a current image frame captured from a scan-based imager; determining whether the current image frame is complete; If it is determined that the current image frame is not complete, processing the current image strip to track movement between the current image strip and a corresponding image strip of a previously processed image frame, and providing a relative frame transformation of the current image frame to transform a position of the current image frame to a corresponding position of the previously processed image frame; setting a current transformation based on a combination of a relative frame transformation of the current image frame and an absolute frame transformation of the previously processed image frame that transforms positions of the previously processed image frame to corresponding positions of an initial image frame; If the current image frame is determined to be complete, processing the current image frame to track motion between the current image frame and an initial image frame, and providing an absolute frame transformation of the current image frame that transforms a position of the current image frame to a corresponding position of the initial image frame according to the tracked motion; setting the current transformation based on an absolute frame transformation of the current image frame; performing a method, Computer equipment.

18. processing the current image frame to track the movement includes determining one or more translations and rotations that transform the position of the current image frame to the corresponding position of the initial image frame using one or more of feature tracking and phase correlation; 18. The computer device of claim 17.

19. Processing the current image strip to track motion includes determining one or more translations that transform positions of the current image frame to corresponding positions of the previously processed image frame using one or more of feature tracking and phase correlation.

18. The computer device of claim 17.

20. The method further comprises: registering a treatment plan including one or more treatment locations of the patient's eye to the initial image frame; applying the current transformation to a next treatment position in the treatment plan to provide an adjusted next treatment position; treating the next treatment location in accordance with the treatment plan.

18. The computer device of claim 17.

21. Treating the next treatment area: adjusting one or more targeting and focusing elements of a laser delivery system to target the adjusted next treatment location; firing the laser delivery system.

21. The computer device of claim 20.

22. the current image strip having a predetermined number of pixel columns captured by the scan-based imager; 22. Computer apparatus according to any one of claims 17 to 21.

23. the number of pixel columns of the current image strip is dynamically determined; 18. The computer device of claim 17.

24. The number of pixel columns in the current image strip is receiving a next row of pixels of the current image strip; processing the current image strip to provide a trial relative frame transform; determining whether the trial relative frame transformation was reliably determined; if the trial relative frame transformation is reliably determined, using the trial relative frame transformation as the relative frame transformation; is dynamically determined by 24. The computer device of claim 23.

Images