Eye tracker with hardware filtering and software sorting

Abstract

In certain embodiments, an ophthalmic surgical system for tracking eye movement includes a laser device, a camera, and a computer. The laser device directs a laser beam at the eye, and the camera captures an image of the eye. The computer includes hardware and software. The hardware identifies objects in the image of the eye and determines whether each object is a candidate object according to one or more filtering criteria. The candidate object represents a candidate pupil image of the pupil of the eye. The hardware notifies the software of the candidate object. The software sorts the candidate objects according to one or more sorting criteria and identifies the candidate objects as pupil images according to the sorted candidate objects. The software tracks the movement of the eye using the pupil image.

Description

Technical Field

[0001] The present disclosure generally relates to ophthalmic surgical systems, and more particularly to an eye tracker having hardware filtering and software sorting.

Background Art

[0002] In certain ophthalmic surgeries, a laser beam is directed at an eye to treat the eye. For example, in laser in-situ keratomileusis (LASIK) surgery, laser pulses are directed at the eye in a specific pattern to reshape the cornea and remove tissue. To effectively treat the eye, the laser beam should be accurately directed at a specific point on the eye. Accordingly, certain systems utilize an eye tracker to monitor the movement of the eye in order to direct the laser beam to the appropriate location.

Summary of the Invention

Means for Solving the Problems

[0003] In certain embodiments, an ophthalmic surgical system for tracking the movement of an eye includes a laser device, a camera, and a computer. The camera captures an image of the eye. The computer includes hardware and software. The hardware identifies objects in the image and determines whether each object is a candidate object according to one or more filtering criteria. The candidate objects represent candidate pupil images of the pupil of the eye. The hardware notifies the software of the candidate objects. The software sorts the candidate objects according to one or more sorting criteria and identifies the candidate objects as pupil images according to the sorted candidate objects. The software tracks the movement of the eye using the pupil image. The laser device directs a laser beam at the tracked eye.

[0004] The embodiments may not include any of the following features, or may include one, several, or all of them. The filtering criteria include square bounding box filtering criteria. The hardware is to establish a bounding box around the following object, the bounding box having a first side of a first length and a second side of a second length, and the bounding box being substantially square, for example, when the first length differs from the second length by at most 30%, identifying the object as a candidate object, thereby identifying whether the object is a candidate object. The filtering criteria include bounding box area filtering criteria. The hardware is to establish a bounding box around the following object and, when the area of the bounding box is within a range corresponding to the area of an average pupil's bounding box, identifying the object as a candidate object, thereby identifying whether the object is a candidate object. The filtering criteria include object area filtering criteria. The hardware is to calculate the area of the following object and, when the area of the object is within a range corresponding to the area of an average pupil, identifying the object as a candidate object, thereby identifying whether the object is a candidate object. The filtering criteria are represented as a logical or mathematical combination of filtering criteria. The hardware identifies whether the object is a candidate object by identifying whether the object meets the combination of filtering criteria. The hardware identifies the features of a previous object identified as a pupil image in a previous image and refines one of the one or more filtering criteria according to the features. The sorting criteria include object area sorting criteria. The software sorts the candidate objects by identifying the area of each candidate object and sorting the candidate objects according to the area, such that candidate objects with larger areas are given a higher priority than candidate objects with smaller areas.The sorting criteria include the center position sorting criterion. The software is to identify the proximity of each candidate object to the center of the image and sort the candidate objects according to the proximity, such that candidate objects closer to the center of the image are given a higher priority than candidate objects farther away from the center of the image, and thereby sort the candidate objects. The sorting criteria include the symmetry sorting criterion. The software is to identify the symmetry of each candidate object and sort the candidate objects according to the symmetry, such that candidate objects with higher symmetry are given a higher priority than candidate objects with lower symmetry, and thereby sort the candidate objects. The sorting criteria include the circularity sorting criterion. The software is to identify the circularity of each candidate object and sort the candidate objects according to the circularity, such that candidate objects with higher circularity are given a higher priority than candidate objects with lower circularity, and thereby sort the candidate objects. The sorting criteria include the density sorting criterion. The software is to identify the density of each candidate object and sort the candidate objects according to the density, such that candidate objects with higher density are given a higher priority than candidate objects with lower density, and thereby sort the candidate objects. The sorting criteria include the tracking sorting criterion. The software identifies the location of a previous object identified as a pupil image in a previous image. The software is to identify the proximity of each candidate object to the location and sort the candidate objects according to the proximity, such that candidate objects with higher proximity are given a higher priority than candidate objects with lower proximity, and thereby sort the candidate objects. The sorting criteria are represented as the order in which the sorting criteria are applied. The software sorts the candidate objects by applying the sorting criteria according to the order. The sorting criteria are represented as a mathematical combination of the sorting criteria. The software sorts the candidate objects by applying the sorting criteria according to the combination.The software identifies the features of previous objects identified as pupil images in previous images and refines the sorting criteria according to the features. If there is only one candidate object, the software identifies the candidate object as a pupil image, and identifies the candidate object as a pupil image according to the sorted candidate objects. Until a candidate object is selected as a pupil image, the software selects the next highest-priority candidate object among the following sorted candidate objects, and if the selected candidate object meets one or more selection criteria, the software identifies the candidate object as a pupil image, thereby identifying the candidate object as a pupil image according to the candidate objects sorted by priority. The selection criteria may comprise a more stringent version of the filtering criteria or may specify limitations of the sorting criteria. The selection criteria may be expressed as a logical or mathematical combination of selection criteria, and the software may identify the candidate object as a pupil image if the candidate object meets the combination of selection criteria. The software may identify the features of previous objects identified as pupil images in previous images and refine the selection criteria according to the features. The computer instructs the laser device to direct the laser beam towards the eye according to the tracked eye movement.

Brief Description of the Drawings

[0005]

Figure 1

Figure 2

Figure 3

Figure 4

DETAILED DESCRIPTION OF THE INVENTION

[0006] Here, exemplary embodiments of the disclosed devices, systems, and methods are shown in detail with reference to the description and the drawings. The description and the drawings are not intended to be exhaustive or to limit the claims to the particular embodiments shown in the drawings and disclosed in the description. The drawings represent possible embodiments, but the drawings are not necessarily to scale and certain features may be simplified, exaggerated, deleted, or partially divided to better illustrate the embodiments.

[0007] A particular eye tracker tracks eye movement by monitoring the movement of eye features (e.g., the pupil) in a series of images. Since the images can be generated at a high speed, such as 1500 Hz, the eye tracker needs to quickly and efficiently identify features within the images. Thus, the eye tracker presented herein includes hardware that filters images for objects that are likely candidates for features and transmits these candidates to software for sorting and identifying the features. The filtering performed by the hardware enables the software to easily identify the features, resulting in more rapid tracking of eye movement.

[0008] 1. Example of an Ophthalmic Surgical System FIG. 1 shows an example of an ophthalmic surgical system that includes an eye tracker for tracking eye movement according to a particular embodiment. In the illustrated embodiment, the ophthalmic surgical system 10 includes a laser device 12, a camera 14, a control computer 16, and an eye tracker 18 that are coupled as shown. The laser device 12 includes a laser source 20, a scanner 22, one or more optical elements 24, and / or a focusing objective lens 26 that are coupled as shown. The computer 16 includes logic 30, a memory 32 (which stores a computer program 34), and a display 36 that are coupled as shown. The eye tracker 18 includes the camera 14 and a portion of the logic 36. For ease of explanation, the following xyz coordinate system is used. That is, the z-axis is defined by the propagation direction of the laser beam when centered according to the laser device 12, and the xy plane is perpendicular to the propagation direction. Other suitable xyz coordinate systems may be used.

[0009] Generally, the eye tracker 18 tracks eye movement. The camera 14 captures an image of the eye. The computer 16 includes hardware and software. The hardware identifies an object in the image and determines whether the object is a candidate object that is likely to represent the pupil. The software sorts the candidate objects according to the likelihood that the candidate object represents the pupil and identifies the candidate object as a pupil image. The software tracks eye movement according to the pupil image. The computer 16 instructs the laser device 12 to direct the laser beam at the eye according to the tracked eye movement.

[0010] Referring to a part of system 10, laser device 12 directs laser pulses towards the eye according to a laser shot pattern. Laser source 20 generates a laser beam including laser pulses. Laser source 20 may be an excimer laser source, a solid-state laser source, a femtosecond laser source, or any other suitable laser source that generates a laser beam of any suitable wavelength, for example, an infrared wavelength or an ultraviolet wavelength. The laser shot pattern specifies the coordinates of the positions (e.g., xy or xyz coordinates) where the laser pulses should be directed. For example, the laser shot pattern may represent an ablation profile indicating a specific location on the cornea where tissue is to be removed.

[0011] Scanner 22 directs the focus of the laser beam in the lateral and / or longitudinal directions. The lateral direction means a direction perpendicular to the beam propagation direction, i.e., the x and y directions. Examples of lateral scanners include galvanometer mirrors tilted about axes perpendicular to each other and electro-optic crystals that steer the laser beam electro-optically. The longitudinal direction means the laser beam propagation direction, i.e., the z direction. Examples of longitudinal scanners include longitudinally adjustable lenses, lenses with variable refractive power, and deformable mirrors that can control the z position of the focus.

[0012] One (or more) optical elements 24 direct the laser beam towards focusing objective lens 26. Optical element 24 can act on the laser beam (e.g., transmit, reflect, refract, diffract, collimate, condition, shape, focus, modulate, and / or act otherwise). Examples of optical elements include lenses, prisms, mirrors, diffractive optical elements (DOEs), holographic optical elements (HOEs), and spatial light modulators (SLMs). In an embodiment, optical element 24 is a mirror. Focusing objective lens 26 focuses the focus of the laser beam towards a point on the eye. In an embodiment, focusing objective lens 26 is an objective lens.

[0013] Camera 14 records an image of the eye and generates image data representing the recorded eye image. The image data may be provided to computer 16 for analysis and also provided to display 35 to present the eye image. Camera 14 may record images at any suitable frequency, such as a high frequency of 1500 Hz or higher. Examples of camera 14 include a charge-coupled device (CCD), video, complementary metal-oxide semiconductor (CMOS) sensor (e.g., active pixel sensor (APS)), line sensor, and optical coherence tomography (OCT) camera.

[0014] Computer 16 controls components of system 10 (e.g., laser 12, camera 14, eye tracker 18, laser source 20, scanner 22, optical element 24, and / or focusing objective lens 26) to treat the eye. In an embodiment, computer 16 identifies the location and position of the eye from eye tracker 18 and aligns the laser shot pattern with the tracked eye. Computer 16 then instructs laser device 12 to direct laser pulses towards the eye according to the laser shot pattern.

[0015] Eye tracker 18 tracks the movement of the eye. Camera 14 sends image data representing the recorded eye image to logic 30 of computer 16 that performs image processing on the image data to analyze the image. Logic 30 includes hardware and software. The hardware filters the image for objects that are likely candidates for the pupil and sends these candidates to the software for pupil sorting and identification. The filtering performed by the hardware enables the software to easily identify the pupil, resulting in faster monitoring of eye movement. An embodiment of eye tracker 18 will be described in more detail with reference to FIGS. 2 and 3.

[0016] 2. Eye Tracker Embodiment Figure 2 shows an example of an eye tracker 18 that may be used by the system 10 of FIG. 1 to track eye movement according to a particular embodiment. In the example, the eye tracker 18 includes a camera 14 and a portion of the logic 36. The camera 14 includes a camera sensor 40, and the logic 36 includes hardware 44 and software 46. The camera sensor 40 converts input light into an electrical signal and may be, for example, a charge coupled device or other suitable sensor. The hardware 44 may be an integrated circuit and may be programmable or application specific, such as a field programmable gate array (FPGA), a complex programmable logic device (CPLD), or an application specific integrated circuit (ASIC). The software 46 may be stored in the memory 32.

[0017] 2.1 Hardware Filtering More specifically, the hardware 44 identifies objects in the eye image and determines, according to filtering criteria, whether each object is a candidate object representing a candidate pupil image, and notifies the candidate objects to the software 46. The hardware 44 may identify objects in any suitable way. For example, the hardware may identify objects using edge detection. Edge detection identifies object boundaries using discontinuities in image luminance. The hardware 44 may select candidate objects likely to be pupil images using any suitable filtering criteria. Examples of filtering criteria include the following.

[0018] 2.1.1 Object Area Filtering Criterion. This criterion selects objects having an area that satisfies an area range (a "reference area range") corresponding to an area similar to the average pupil area, for example, an average pupil area (e.g., 0.8 square millimeters (mm 2 ) to 80 mm 2 )). The reference area range may be the same as the average pupil area or may be wider or narrower (e.g., ±10 mm) to result in a larger or smaller set of candidate objects2 Even just doing this is fine. In one embodiment, the hardware 44 identifies the boundary of the object and then calculates the area of the object. The hardware 44 then notifies the software 46 of the object by transmitting the coordinates of the points within the object.

[0019] 2.1.2. Bounding Box. FIG. 3 shows an example of a bounding box 50 disposed around an object 52 that may be used to select a candidate object according to a particular embodiment. In an embodiment, the bounding box 50 is the smallest rectangle that encloses the object 52. For example, the box 50 is the minimum surrounding bounding box. In an example, the boundaries of the box 50 include the vertical lines x = a and x = b and the horizontal lines y = p and y = q, where x and y are related to the xy plane orthogonal to the z direction of the laser beam. The vertical lines may be longer, equal to, or shorter than the horizontal lines. The coordinates of the boundaries may be used to describe the box 50 and may also be used to identify the object enclosed by the box 50. For example, the hardware 44 may notify the software 46 of the object 52 by transmitting the boundary coordinates of the bounding box 50 of the object 52.

[0020] 2.1.2.1 Square Bounding Box Filtering Criterion. The pupil is approximately circular, so an object having a bounding box that is approximately square is more likely to represent the pupil. Therefore, this criterion selects the object 52 when the bounding box 50 of the object 52 is approximately square, for example, when the length of the horizontal side is approximately the same as the length of the vertical side (e.g., at most 10, 15, 20, or 30% different).

[0021] 2.1.2.2 Bounding Box Area Filtering Criterion. This criterion is based on the average bounding box area of the pupil, for example, 1 square millimeter (mm 2 ) to 64 mm2 Select the object with a bounding box having an area corresponding thereto (the "bounding box area"). The reference area range may be the same as the average pupil bounding box area, or may be wider or narrower (e.g., ±10 mm 2 only) so as to yield a larger or smaller set of candidate objects.

[0022] 2.1.3. Combination of filtering criteria. In certain embodiments, a combination of filtering criteria may be used to select candidate objects. For example, the combination may be represented as a logical combination of filtering criteria F1, F2, ··· Fj, e.g., "F1 AND F2" or "F1 OR F2". In order to satisfy "F1 AND F2", the object must satisfy both F1 and F2. As another example, the combination may be represented as a mathematical combination of filtering criteria F1, F2, ··· Fj, e.g., "F1 / F2", where F1 represents the pupil area and F2 represents the pupil bounding box area.

[0023] 2.1.4. Refinement of filtering criteria. In certain embodiments, the filtering criteria may be refined according to pupil characteristics identified from previous images, e.g., location, size, shape, area, brightness, density, bounding box characteristics, symmetry, or circularity. For example, the area of an object representing the pupil in a previous image may be identified. Accordingly, for example, the area range of the object area filtering criteria may be refined to filter a narrow range (e.g., ±10 mm 2 ) around the pupil area.

[0024] 2.2 Software Sort Returning to FIG. 2, software 46 sorts candidate objects according to a sorting criterion and identifies candidate objects as pupil images according to the sorted candidate objects. Software 46 may sort candidate objects according to any suitable sorting criterion according to the likelihood that a candidate object is a pupil image. A higher priority may be given to candidate objects that are more likely to be pupil images. Examples of sorting criteria include the following.

[0025] 2.2.1 Object region sorting criterion. Since the pupil tends to be the largest object in an eye image, larger objects are more likely to be the pupil. In one embodiment, software 46 calculates the area of an object and sorts the objects according to the area. Objects with a larger area are given a higher priority than objects with a smaller area, where the highest priority is given to the object with the largest area.

[0026] 2.2.2 Central position sorting criterion. Since the pupil is usually at the center of an eye image, objects closer to the center of the image are more likely to be the pupil. In one embodiment, software 46 identifies the locations of objects within the image and sorts the objects according to those locations. Objects closer to the center of the eye image are given a higher priority than objects that are farther away, where the highest priority is given to the object closest to (and potentially at) the center of the image. In certain embodiments, the proximity of an object to the image center may be specified by the proximity of the center of the object (e.g., the centroid of the object) to the image center.

[0027] 2.2.3 Symmetry sorting criteria. Since the pupil is generally circular, objects that are more symmetric are more likely to be the pupil image. In one embodiment, software 46 identifies the symmetry of the objects and sorts the candidates according to their symmetry. Higher priority is given to more symmetric objects than to less symmetric objects, where the highest priority is given to the most symmetric object. In certain embodiments, a more symmetric object may be an object having more symmetry lines or a smaller rotational symmetry angle.

[0028] 2.2.4 Circularity sorting criteria. Since the pupil is generally circular, objects that are more circular are more likely to be the pupil image. In one embodiment, software 46 identifies the circularity of the objects and sorts the objects according to their circularity. Higher priority is given to more circular objects than to less circular objects, where the highest priority is given to the most circular object. In certain embodiments, circularity is measured by the ratio between the inscribed circle and the circumscribed circle of the object, where a rounder object has a higher ratio.

[0029] 2.2.5 Density sorting criteria. Since the pupil is usually the darkest object in the image, objects with higher density are more likely to be the pupil image. In one embodiment, software 46 identifies the density of the objects and sorts the objects according to their density. Higher priority is given to objects with higher density than to objects with lower density, where the highest priority is given to the object with the highest density. In certain embodiments, density may be measured by the percentage of darker pixels (e.g., pixels that are at least 80% black), where a higher percentage indicates a higher density.

[0030] 2.2.6 History / Tracking Sorting Criteria. Eye features do not move much, especially relative to other features of the eye. Therefore, if an object in a previous image was identified as a pupil image, an object closer to the location of the identified object in subsequent images is more likely to be the pupil image. In one embodiment, software 46 identifies the location of the object representing the pupil image. In subsequent images, objects closer to that location are given a higher priority than objects farther from that location, where the highest priority is given to the closest object (which may be the object at that location).

[0031] 2.2.7 Combining Sorting Criteria. In certain embodiments, combinations of sorting criteria may be used to sort candidate objects. In some cases, the combination may be represented as the order in which sorting criteria S1, S2, ··· Sj are applied to sort candidate objects. For example, according to the order "first S1, then S2", objects are first sorted according to S1, and then objects having the same value for S1 are sorted according to S2. In other cases, the combination may be represented as a mathematical combination of weighted combinations of sorting criteria, such as w1S1, w2S2, ···, wjSj, where wi represents the weight assigned to criterion Si. For example, objects may be scored according to "w1S1 + w2S2" and then sorted according to their scores.

[0032] 2.2.8 Refining Sorting Criteria. In certain embodiments, the sorting criteria may be refined according to the features of the pupil identified from previous images (as described with reference to refining filtering criteria). For example, the location of the object representing the pupil in a previous image may be identified. Accordingly, the center position sorting criterion may be refined to give a higher priority to objects closer to the location of the pupil rather than the center of the image.

[0033] 2.3 Examples of Selection Criteria Software 46 identifies the candidate object as a pupil image in any suitable way. For example, if there is only one candidate object, that candidate object is identified as the pupil image. If there are multiple candidate objects, software 46 may perform the following until a candidate object is selected as the pupil image. Select one of the candidate objects sorted according to the priority (for example, a higher priority is selected before a lower priority), and if the selected candidate object meets one or more selection criteria, the candidate object is identified as the pupil image. If there is no candidate object, computer 16 may search for a candidate object in subsequent images.

[0034] 2.3.1 Selection Criteria from Filtering Criteria. In a particular embodiment, the selection criteria may be a more stringent version of the filtering criteria. For example, if the filtering criteria have a specific range, the selection criteria use a more restrictive range to select the pupil image. Examples of such selection criteria include the following.

[0035] (1) Object Area: The area of the object is between 2 square millimeters (mm 2 ) and 40 mm 2 .

[0036] (2) Bounding Box Area: The area of the bounding box is between 4 square millimeters (mm 2 ) and 50 mm 2 .

[0037] (3) Bounding Box Square: The bounding box is approximately square, where the first length differs from the second length by at most 10%.

[0038] 2.3.2 Selection criteria from sorting criteria. In certain embodiments, the selection criteria may specify limitations of the sorting criteria. For example, if the sorting criteria is sorting by feature, the selection criteria uses feature limitations to select pupil images. Examples of such selection criteria include the following.

[0039] (1) Center position: The object is within 2 millimeters (mm) from the center of the image.

[0040] (2) Symmetry: The object has at least two axes of symmetry or a rotational symmetry angle of less than 45 degrees.

[0041] (3) Circularity: Object a has a circularity ratio between the inscribed circle and the circumscribed circle of the object that exceeds 0.8.

[0042] (4) Density: The object has a percentage of darker pixels that exceeds 90%.

[0043] 2.3.3 Combination of selection criteria. In certain embodiments, a logical or mathematical combination of selection criteria may be used to select pupil images in a manner similar to that for combinations of filtering or sorting criteria. In certain embodiments, the selection criteria may be applied in a specific order to select pupil images in a manner similar to that for ordering sorting criteria.

[0044] 2.3.4 Refinement of selection criteria. In certain embodiments, the selection criteria may be refined according to the characteristics of the pupil identified from previous images (as described with reference to refining filtering and sorting criteria). For example, the location of the object representing the pupil image in the previous image may be identified. Accordingly, the center position selection criteria may be refined to select an object closer to the location of the pupil rather than the center of the image.

[0045] 3. Examples of eye tracking method Figure 4 shows an example of a method for identifying features of an eye (e.g., a pupil) to track eye movement that may be performed by the eye tracker of FIG. 2 according to a particular embodiment. The method begins at step 110 where camera 14 captures an image of the eye. Steps 112-116 are performed by hardware 44. An object within the image is identified at step 112. The hardware 44 may identify the object, for example, by edge detection. The hardware 44 determines at step 114 whether each object is a candidate object according to filtering criteria. Examples of filtering criteria include a square bounding box, a bounding box, an object area, and / or other suitable filtering criteria. The hardware 44 notifies software 46 of the candidate objects at step 116. For example, the hardware may transmit the coordinates of the object and / or the bounding box surrounding the object to software 46.

[0046] Steps 120 and 122 are performed by software 46. The software 46 sorts the candidate objects according to sorting criteria at step 120. Examples of sorting criteria include an object area, a center position, a symmetry, a circularity, a density, and / or other suitable sorting criteria. The software 46 identifies the candidate objects as pupil images according to the sorted candidate objects at step 122. For example, if there is only one candidate object, that candidate object is identified as the pupil image. If there are multiple candidate objects, the software 46 may perform the following until a candidate object is selected as the pupil image. Select one of the candidate objects sorted according to the priority (e.g., a higher priority is selected before a lower priority), and if the selected candidate object meets one or more selection criteria, the candidate object is identified as the pupil image. The software 46 tracks the eye using the pupil image at step 124.

[0047] When the position of the eye is tracked, at step 126, computer 16 instructs laser device 12 to direct the laser beam towards the eye in accordance with the tracked eye movement. The method ends.

[0048] The components (such as control computer 16) of the systems and devices disclosed herein may include an interface, logic, and / or memory, and any of these may include computer hardware and / or software. An interface can receive inputs to and / or transmit outputs from a component, and is typically used to exchange information between, for example, software, hardware, peripheral devices, users, and combinations thereof. A user interface is a type of interface that can be utilized by a user to communicate with a computer (e.g., send inputs to and / or receive outputs from the computer). Examples of user interfaces include displays, graphical user interfaces (GUIs), touchscreens, keyboards, mice, gesture sensors, microphones, and speakers.

[0049] Logic may execute the operations of a component. Logic may include one or more electronic devices that process data, e.g., execute instructions to generate an output from an input. Examples of such electronic devices include computers, processors, microprocessors (e.g., central processing units (CPUs)), and computer chips. Logic may include computer software that encodes instructions executable by an electronic device to perform operations. Examples of computer software include computer programs, applications, and operating systems.

[0050] The memory can store information and may include a tangible computer-readable and / or computer-executable storage medium. Examples of memory include computer memory (e.g., random access memory (RAM) or read-only memory (ROM)), mass storage media (e.g., hard disk), removable storage media (e.g., compact disc (CD) or digital video or versatile disc (DVD)), databases, network storage (e.g., server), and / or other computer-readable media. Certain embodiments may be directed to memory encoded with computer software.

[0051] While the present disclosure has been described with respect to particular embodiments, modifications to the embodiments (e.g., variations, substitutions, additions, omissions, and / or other modifications) should be apparent to those skilled in the art. Accordingly, modifications may be made to the embodiments without departing from the scope of the invention. For example, modifications may be made to the systems and apparatuses disclosed herein. As will be apparent to those skilled in the art, the components of the systems and apparatuses may be integrated or separated, or the operations of the systems and apparatuses may be performed by more, fewer, or other components. As another example, modifications may be made to the methods disclosed herein. As will be apparent to those skilled in the art, the methods may include more, fewer, or other steps, and the steps may be performed in any suitable order.

[0052] To assist the Patent Office and the reader in construing the claims, the applicant notes that unless the words "means for" or "step for" are expressly used in a particular claim, no claim or claim element is intended to invoke 35 U.S.C. § 112(f). The use of any other term within the claims (e.g., "mechanism," "module," "device," "unit," "component," "element," "member," "apparatus," "machine," "system," "processor," or "controller") is understood by the applicants to refer to structures known to those of ordinary skill in the art and is not intended to invoke 35 U.S.C. § 112(f).

Claims

1. An ophthalmic surgical system for tracking eye movements, comprising: a laser device configured to direct a laser beam towards the eye; a camera configured to capture a plurality of images of the eye; a computer, hardware, identifying a plurality of objects in the plurality of images of the eye, determining whether each object of the plurality of objects is a candidate object according to one or more filtering criteria such that the candidate object represents a candidate pupil image of the pupil of the eye, resulting in one or more candidate objects; notifying the one or more candidate objects to software; hardware configured as such; software, sorting the one or more candidate objects according to one or more sorting criteria; identifying the candidate object as the pupil image of the pupil according to the sorted one or more candidate objects; tracking the movement of the eye using the pupil image; software configured as such; a computer comprising the same; an ophthalmic surgical system comprising the same.

2. The one or more filtering criteria comprise a square bounding box filtering criterion, and the hardware is configured to: establish a bounding box around the object, the bounding box having a first side of a first length and a second side of a second length; and determine that the object is a candidate object if the bounding box is substantially square and the first length differs from the second length by at most 30%, whereby to determine whether the object is a candidate object. The ophthalmic surgical system according to Claim 1.

3. The one or more filtering criteria comprise a bounding box area filtering criterion, and the hardware is configured to: establish a bounding box around the object, the bounding box having an area; and determine that the object is a candidate object if the area of the bounding box is within a range corresponding to the area of an average pupil bounding box, whereby to determine whether the object is a candidate object. The ophthalmic surgical system according to Claim 1.

4.

5.

6.

7.

8.

9.

10.

11.

12.

13.

14.

15.

16.

17.

18.

19.

20.

21.

22.

23.

24.

25.

26.

27.

28.

29.

30.

31.

32.

33.

34.

35.

36.

37.

38.

39.

40.

41.

42.

43.

44.

45.

46.

47.

48.

49.

50.

51.

52.

53.

54.

55.

56.

57.

58.

59.

60.

61.

62.

63.

64.

65.

66.

67.

68.

69.

70.

71.

72.

73.

74.

75.

76.

77.

78.

79.

80.

81.

82.

83.

84.

85.

86.

87.

88.

89.

90.

91.

92.

93.

94.

95.

96.

97.

98.

99.

100.

101.

102.

103.

104.

105.

106.

107.

108.

109.

110.

111.

112.

113.

114.

115.

116.

117.

118.

119.

120.

121.

122.

123.

124.

125.

126.

127.

128.

129.

130.

131.

132.

133.

134.

135.

136. ]

137. ]

138. ]

139. ]

140. ]

141. ]

142. ]

143. ]

144. ]

145. ]

146. ]

147. ]

148. ]

149. ]

150. ]

151. ]

152. ]

153. ]

154. ]

155. ]

156. ]

157. ]

158. ]

159. ]

160. ]

161. ]

162. ]

163. ]

164. ]

165. ]

166. ]

167. ]

168. ]

169. ]

170. ]

171. ]

172. ]

173. ]

174. ]

175. ]

176. ]

177. ]

178. ]

179. ]

180. ]

181. ]

182. ]

183. ]

184. ]

185. ]

186. ]

187. ]

188. ]

189. ]

190. ]

191. ]

192. ]

193. ]

194. ]

195. ]

196. ]

197. ]

198. ]

199. ]

200. ]

201. ]

202. ] [[ The one or more filtering criteria include an object area filtering criterion, The hardware is the following, calculating the area of the object, identifying the object as a candidate object when the area of the object is within a range corresponding to an average pupil area, configured to identify whether an object having an area is a candidate object by: The ophthalmic surgical system according to claim 1.

5. The one or more filtering criteria are expressed as a logical or mathematical combination of the filtering criteria, The hardware is configured to identify whether the object is a candidate object by identifying whether the object satisfies the combination of the filtering criteria, The ophthalmic surgical system according to claim 1.

6. The hardware is identifying features of a previous object identified as the pupil image in a previous image, configured to refine one of the one or more filtering criteria according to the features, The ophthalmic surgical system according to claim 1.

7. The one or more sorting criteria include an object area sorting criterion, The software is the following, identifying the area of each of the one or more candidate objects, sorting the candidate objects according to the area, wherein a candidate object having a larger area is given a higher priority than a candidate object having a smaller area, configured to sort the one or more candidate objects each having an area by: The ophthalmic surgical system according to claim 1.

8. The one or more sorting criteria include a center position sorting criterion, The software is the following, identifying the proximity of each of the one or more candidate objects to the center of the image, sorting the candidate objects according to the proximity, wherein a candidate object closer to the center of the image is given a higher priority than a candidate object farther from the center of the image, configured to sort the one or more candidate objects of the plurality of images in which each candidate object has the proximity to the center of the image The ophthalmic surgical system according to claim 1.

9. The one or more sorting criteria include a symmetry sorting criterion, The software is as follows, identifying the symmetry of each candidate object of the one or more candidate objects, sorting the candidate objects according to symmetry, wherein a higher symmetry candidate object is given a higher priority than a lower symmetry candidate object, configured to sort the one or more candidate objects in which each candidate object has the symmetry The ophthalmic surgical system according to claim 1.

10. The one or more sorting criteria include a circularity sorting criterion, The software is as follows, identifying the circularity of each candidate object of the one or more candidate objects, sorting the candidate objects according to circularity, wherein a higher circularity candidate object is given a higher priority than a lower circularity candidate object, configured to sort the one or more candidate objects in which each candidate object has the circularity The ophthalmic surgical system according to claim 1.

11. The one or more sorting criteria include a density sorting criterion, The software is as follows, identifying the density of each candidate object of the one or more candidate objects, sorting the candidate objects according to density, wherein a higher density candidate object is given a higher priority than a lower density candidate object, configured to sort the one or more candidate objects in which each candidate object has the density The ophthalmic surgical system according to claim 1.

12. The one or more sorting criteria include a tracking sorting criterion, The software identifies the location of a previous object identified as the pupil image in a previous image and is as follows, identifying the proximity of each candidate object of the one or more candidate objects to the location Sorting the candidate object according to the proximity, such that a candidate object with a higher proximity is given a higher priority than a candidate object with a lower proximity; configured to sort the one or more candidate objects thereby; The ophthalmic surgery system according to claim 1.

13. The one or more sorting criteria are represented as an order in which the sorting criteria are applied; The software is configured to sort the candidate objects by applying the sorting criteria according to the order; The ophthalmic surgery system according to claim 1.

14. The one or more sorting criteria are represented as a mathematical combination of the sorting criteria; The software is configured to sort the candidate objects according to the combination; The ophthalmic surgery system according to claim 1.

15. The software is identifying features of a previous object identified as the pupil image in a previous image, configured to refine sorting criteria according to the features; The ophthalmic surgery system according to claim 1.

16. The software is configured to identify a candidate object as the pupil image according to the sorted one or more candidate objects, by identifying the one candidate object as the pupil image when the one or more candidate objects include only one candidate object; The ophthalmic surgery system according to claim 1.

17. The software is, until a candidate object is selected as the pupil image, selecting the next highest-priority candidate object among the sorted one or more candidate objects, identifying the candidate object as the pupil image when the selected candidate object meets one or more selection criteria, configured to identify a candidate object as the pupil image according to the sorted one or more candidate objects by performing the above, wherein the candidate objects are sorted by priority; The ophthalmic surgery system according to claim 1.

18. One of the one or more selection criteria comprises a more stringent version of a filtering criterion. The ophthalmic surgery system according to claim 17.

19. The ophthalmic surgical system according to claim 17, wherein one of the one or more selection criteria specifies a restriction of the sorting criterion.

20. The one or more selection criteria are represented as a logical or mathematical combination of the selection criteria, and the software is configured to identify the candidate object as the pupil image when the candidate object satisfies the combination of the selection criteria. The ophthalmic surgical system according to claim 17.

21. The software is configured to identify features of a previous object identified as the pupil image in a previous image, and refine the selection criteria according to the features. The ophthalmic surgical system according to claim 17.

22. The computer is configured to instruct the laser device to direct the laser beam at the eye according to the tracked movement of the eye, as claimed in claim 1.

Images