Eye tracking instrument with multiple cameras

Abstract

In certain embodiments, an ophthalmic system tracks movement of an eye region and includes a camera system and a computer. A camera system has cameras that produce an image portion of an eye region, where each camera images at least a portion of the eye region. The camera system has a system axis and a system field of view. The eye region includes one or two eyes, and each eye has an eye center and an eye axis. The computer receives the image portion from the camera system and tracks movement of the at least one eye from the image portion.

Description

Technical Field

[0001] This disclosure generally relates to ophthalmic systems, and more particularly to an eye tracker having multiple cameras. Background Art

[0002] Certain ophthalmic systems utilize an eye tracker to monitor eye movements. For example, in a laser-assisted in situ keratomileusis (LASIK) procedure, laser pulses are directed onto the eye in a specific pattern to ablate tissue to reshape the cornea. To effectively treat the eye, the laser beam should be accurately directed to a specific point on the eye, even when the eye is moving. Accordingly, an eye tracker is used to monitor eye movements. Summary of the Invention

[0003] In some embodiments, an ophthalmic system tracks the movement of an eye region and includes a camera system and a computer. The camera system has cameras that produce image portions of the eye region, where each camera images at least a portion of the eye region. The camera system has a system axis and a field of view. The eye region includes one or both eyes, and each eye has an eye center and an axis. The computer receives the image portions from the camera system and tracks the movement of at least one eye based on the image portions.

[0004] Embodiments may not include the following features or may include one, some, or all of the following features:

[0005] * The computer tracks the movement of at least one eye in two dimensions.

[0006] * The computer tracks the movement of at least one eye in three dimensions to allow 6D tracking.

[0007] * The cameras include a set of stereo cameras symmetrically arranged about the system axis.

[0008] * The cameras include coaxial cameras aligned with the system axis.

[0009] * The cameras include asymmetrically arranged cameras that lack corresponding cameras symmetric about the system axis.

[0010] * The cameras include high-speed cameras that generate images at a rate greater than 400 frames per second.

[0011] * The cameras include high-resolution cameras that generate images having greater than 4 megapixels.

[0012] * At least one camera detects the visible light range from the eye region to produce the image portions.

[0013] * At least one camera detects the infrared light range from the eye region to produce the image portions.

[0014] *At least one camera detects an ultraviolet light range from the eye region to generate an image portion.

[0015] *A light projector directs a light pattern toward at least one eye of the eye region. At least one camera detects the light pattern reflected by the at least one eye.

[0016] *A computer aligns the image portions to generate a combined image of the eye region and tracks the movement of at least one eye based on the combined image.

[0017] In some embodiments, a method for tracking the movement of an eye region includes providing an image portion of the eye region by a camera system of a camera. The camera system has cameras that generate image portions of the eye region, where each camera images at least a portion of the eye region. The camera system has a system axis and a system field of view. The eye region includes one or two eyes, and each eye has an eye center and an eye axis. A computer receives the image portions from the camera system and tracks the movement of at least one eye of the eye region based on the image portions.

[0018] The embodiments may not include the following features or may include one, some, or all of the following features:

[0019] *The method further includes tracking the movement of at least one eye in two dimensions.

[0020] *The method further includes tracking the movement of at least one eye in three dimensions to allow 6D tracking.

[0021] *The method further includes generating images at a rate greater than 400 frames per second.

[0022] *The method further includes generating images having greater than 4 megapixels.

[0023] *The method further includes directing a light pattern toward at least one eye by a light projector. At least one camera detects the light pattern reflected by the eye. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 An example of an ophthalmic system with an eye tracker according to some embodiments is shown;

[0025] Figure 2A and Figure 2B An example of the field of view (FOV) of a camera system according to some embodiments is shown; Figure 1 of

[0026] Figure 3A and Figure 3B An example of Figure 1Example of a camera system tracking an example eye region;

[0027] Figure 4 Shows an example of a stereo arrangement of cameras of a camera system according to certain embodiments; Figure 1 of a camera system;

[0028] Figure 5 Shows an example of a stereo coaxial arrangement of a camera system according to certain embodiments; Figure 1 of a camera system;

[0029] Figure 6 Shows an example of an asymmetric arrangement of cameras of a camera system according to certain embodiments; and Figure 1 of a camera system;

[0030] Figure 7 Shows an example of a method that can be performed by an Figure 1 ophthalmic system according to certain embodiments. DETAILED DESCRIPTION

[0031] Reference is now made to the specification and the drawings to illustrate in detail example embodiments of the disclosed devices, systems, and methods. The specification and the drawings are not intended to be exhaustive or to otherwise limit the claims to the specific embodiments shown in the drawings and disclosed in the specification. Although the drawings represent possible embodiments, the drawings are not necessarily drawn to scale, and certain features may be simplified, exaggerated, removed, or partially sectioned to better illustrate the embodiments.

[0032] In some eye trackers, a light projector directs light towards the eye at a known angle, and a camera generates an image showing the light reflection on the eye. Assumptions based on a standard eye model are used to determine eye movement from the camera image. However, these assumptions may not accurately describe the eyes of a particular patient, resulting in less accurate tracking.

[0033] The eye trackers described herein do not require eye model assumptions and can therefore provide more accurate tracking. The eye tracker includes a camera system having cameras that image the eye from different directions (e.g., coaxially and obliquely). Based on the known orientation of the cameras, eye movement can be determined from the resulting images. The tracker can track translational movement and / or rotational motion, for example, in the x, y, and / or z directions. In certain embodiments, the cameras can record infrared (IR), visible light, and / or other light and can record images at a higher speed and / or higher resolution. The eye tracker can be used in an ophthalmic diagnostic and / or treatment system (e.g., in refractive surgery or cataract surgery).

[0034] Figure 1Shows an example of an ophthalmic system 10 with an eye tracker 12 according to certain embodiments. The eye tracker monitors an eye region 14 that includes one or both eyes of a patient. Generally, the eye tracker 12 monitors one or more features of the eye in an image (e.g., pupil, iris, blood vessels, limbus, sclera, eyelashes, and / or eyelids) to track eye movement.

[0035] For ease of explanation, certain eye features are used to define an example coordinate system 16 (x, y, z) of the eye. For example, the eye has a center (e.g., pupil center, highest point, vertex) and an eye axis 15 (e.g., optical axis or pupil axis), which can define the z-axis of the eye coordinate system 16, and the z-axis in turn defines the xy plane of the coordinate system 16. The eye region 14 has a region axis 17. If the eye region 14 has one eye, the region axis 17 can be substantially coincident with the eye axis 15. If the eye region 14 has two eyes, the region axis 17 can pass through the midpoint between the two eyes.

[0036] As an overview of the example system, the ophthalmic system 10 includes an eye tracker 12, an ophthalmic device 22, a display 24, and a computer 26 (which includes logic 27 and a memory 28), coupled as shown. The eye tracker 12 includes a camera system 20 and a computer 26, coupled as shown. In certain embodiments, the eye tracker 13 includes a light projector 30 to allow tracking in the z direction. As an example of an operational overview, the camera system 20 of the eye tracker 12 has a camera that produces an image portion of the eye region 14. Each camera is located in a known orientation (e.g., relative to each other and / or a known position and / or orientation relative to the eye region 14) and records at least a portion of the eye region 14 to produce an image portion. As described in more detail below, the known orientation allows for the calculation of eye movement. The computer 26 receives the image portion from the camera system 20 and tracks the movement of at least one eye based on the image portion.

[0037] Turning to the components of this example, the eye tracker 12 can track eye movement in six dimensions (6D), i.e., "6D tracking". These six dimensions include translational movement in the x, y, and z directions relative to the eye coordinate system 16, rotational movement, x-roll movement, and / or y-roll movement. In certain embodiments, the x, y, and z translational movements can be translational movements in the x, y, and z directions, respectively. The rotational movement can be a movement about the eye axis 15. The x-roll movement and y-roll movement can be rotational movements about the x-axis and y-axis, respectively. In a particular embodiment, 6D tracking can track some or all of the 6D movements.

[0038] In some embodiments, the eye tracker 12 includes a camera system 20 that generates an image of the eye region 14. The camera system 20 has a field of view (FOV) that covers the eye region 14 (described in more detail with respect to Figure 2A and Figure 2B ). The FOV has a known relationship to the coordinate system of the camera system 20, which in turn has a known relationship to the coordinate system used by the ophthalmic device 22 to treat and / or diagnose the eye. The eye tracker 12 tracks the movement of the eye by tracking the movement of the eye relative to the FOV. The ophthalmic device 22 can use the eye tracking information to treat and / or diagnose the eye.

[0039] In an embodiment, the camera system 20 includes a camera. For ease of explanation, the "orientation" of the camera relative to the eye region 14 can describe the distance between the camera and the eye region 14 and the direction of the camera axis relative to the region axis 17. The camera detects light from an object and generates a signal in response to the light. The signal carries image data that can be used to generate an image of the eye. The image data is provided to a computer 26 for eye tracking (and optionally other analysis), and can also be provided to a display 24 to present an image of the eye. Examples of cameras include charge-coupled devices (CCDs), video, complementary metal oxide semiconductor (CMOS) sensors (e.g., active pixel sensors (APS)), line sensors, and optical coherence tomography (OCT) cameras.

[0040] The camera detects light having any suitable spectral range, such as a range of infrared (IR), ultraviolet (UV), and / or visible (VIS) wavelength light, where the range can include a portion or all of the wavelengths. For example, the camera can detect visible light, infrared light, or other visible and infrared light from the eye region 14 to produce an image portion. Some cameras can capture features of the eye (e.g., the pupil, iris structure, blood vessels, limbus, etc.) better than others. For example, an infrared camera typically provides more stable pupil tracking and better contrast for the iris structure. Accordingly, an IR camera can be used to monitor lateral movement by tracking the pupil and / or to monitor eye rotation by tracking the iris structure. As another example, a visible range camera produces better blood vessel images, so a visible range camera can be used to monitor translational and / or rotational movement by tracking the blood vessels.

[0041] The camera can record images at any suitable frequency or resolution. A high-speed camera can record images at a rate greater than, for example, 400 to 1500 frames per second (such as greater than 500, 750, or 1000 frames per second). A high-resolution camera can produce images with a resolution greater than, for example, 4 to 24 megapixels (such as greater than 5, 10, 15, or 20 megapixels). Generally, higher-resolution images and higher-speed image acquisition can provide more accurate tracking, but these two features may require more computational time, so there may be a trade-off between resolution and speed. Accordingly, the speed and / or resolution of the camera can be selected for a specific purpose. In some embodiments, a high-speed camera can track eye features that move faster and / or can be identified at a lower resolution, and a high-resolution camera can be used to track eye features that require a higher resolution for identification and / or move more slowly. For example, a lower-resolution, higher-speed camera can track the pupil (which does not require high resolution) to detect xy movement. As another example, a higher-resolution, lower-speed camera can track the blood vessel / iris structure to detect rotation and z movement.

[0042] The ophthalmic device 22 can be a system for diagnosing and / or treating the eye. Examples include refractive surgery systems, cataract systems, topographers, OCT measurement devices, and wavefront measurement devices. The display 24 provides images to the user of the system 10, for example, image portions and / or combined images. Examples of the display 24 include computer monitors, 3D monitors, projectors (projector / beamer), TV monitors, binocular monitors, glasses with displays, virtual reality displays, augmented reality displays, and mixed reality displays.

[0043] The light projector 30 directs a light pattern towards the eye region 14 and uses the reflection of the light to track the eye. The light projector 30 can include one or more light sources that generate the light pattern. The light projection can be used in any suitable manner. For example, the light can be directed at a known angle, which can be used to align the image portions. As another example, the curvature of the eye causes line projection distortion, so the line distortion can help identify the boundary of the curvature change between the cornea and the sclera. As yet another example, symmetric projection can be used to identify the vertex or highest point of the eye. As yet another example, a stripe projector can project lines at an angle onto the eye, so these lines appear curved at the cornea and change curvature as the eye moves. Any suitable pattern can be used, for example, lines (such as stripes), crosses, and / or line arrays and / or point arrays.

[0044] Computer 26 controls components of system 10 (e.g., camera system 20, ophthalmic device 22, display 24, and / or light projector 30) to track the eyes. In this example, computer 16 receives image portions from camera system 20 and tracks movement of at least one eye based on the image portions. In some embodiments, computer 26 aligns the image portions to produce a combined image of eye region 14, and tracks movement of at least one eye based on the combined image.

[0045] Figure 2A and Figure 2B Demonstrating some embodiments of the Figure 1 4. An example of a field of view (FOV) 40 of a camera system 20 of FIG. 40. The cameras of the camera system 20 have a field of view (FOV) that detects light from the eye region 14 to produce an image portion 45 of some or all of the eye region 14. Different cameras may have different FOVs that detect light from different portions of the eye region at different directions, and the different FOVs may overlap. The combined FOVs from the cameras produce the system FOV 40. In general, more cameras at different locations (positions and orientations) may improve the accuracy of detection and tracking of eye features.

[0046] In this example, the camera system 20 has a system FOV 40, a system axis 42, and a system coordinate system 44 (x', y', z'). The system axis 42 can have any suitable position, for example, the axis 42 can be substantially orthogonal to the system FOV 40 and can pass through the center of the system FOV 40. The system axis 42 and the system coordinate system 44 (x', y', z') can be related in any suitable manner. In this example, the system axis 42 defines the z' axis of the system coordinate system 44. In this example, the system FOV 40 is generally planar and images numbers 1 to 9. The camera system 20 includes a camera A with a FOV A and a camera B with a FOV B. FOV A covers the system FOV 40 (i.e., image numbers 1 to 9), and FOV B covers only a portion of the system FOV 40 (i.e., image numbers 4 to 9). Camera A produces image portion A, and camera B produces image portion B.

[0047] In some embodiments, computer 26 aligns and combines image portions 45 to produce a combined image 46. The image portions 45 can be aligned in any suitable manner. For example, each camera has a known orientation (such as position (e.g., distance from system FOV 40 and / or eye region 14) and orientation (e.g., camera optical axis relative to system axis 42 and / or eye axis 15, or viewing angle)) as well as dimensions and imaging characteristics. Based on this information, computer 26 can determine the positions of the image portions 45 to align them within the combined image 46. As another example, each camera generates an image of a calibration pattern (e.g., checkerboard), and the orientation of the camera is determined based on the image. As yet another example, when viewed through the camera, the user calibrates the image portions 45 by manually aligning portion 45. Computer 26 records the positions of the aligned portions.

[0048] The image portions 45 can be combined in any suitable manner. For example, the image portions 45 can be combined to produce a two-dimensional (2D) image to allow 2D tracking, and / or the image portions 45 (e.g., from a stereo camera) can be combined to produce a three-dimensional (3D) image to allow 3D tracking.

[0049] Eye tracker 12 tracks one or both eyes of eye region 14 based on the image portions and / or combined image 46. For example, computer 26 identifies target eye features (e.g., pupil, iris structure, or blood vessels) in the uncombined or combined image portions, and tracks the movement of the feature relative to system FOV 40 to track the eye. Computer 26 can use the image portions 45 from the camera with a better quality image that is more likely to produce the feature to identify the feature. For example, the camera can have a FOV, wavelength, resolution, and / or speed that is more likely to image the feature. Examples of cameras with such characteristics imaging specific features are presented throughout this specification.

[0050] Figure 3A and Figure 3B shows an example of Figure 1 camera system 20 tracking eye region 14 according to certain embodiments. In Figure 3A , eye region 14 includes one eye. In this example, the eye axis 15 of the eye can first be substantially aligned with the system axis 42 of camera system 20. As the eye moves relative to camera system 20, the eye axis 15 moves relative to the system axis 42.

[0051] In Figure 3BIn it, the eye region 14 includes two eyes. The system axis 42 of the camera system 20 is substantially aligned with the midpoint between the two eyes. The camera system 20 includes cameras that image one or both eyes to produce image portions and / or a combined image of both eyes simultaneously, so the camera system 20 can track both eyes simultaneously and independently of each other. In some embodiments, the camera system 20 includes a pair of stereo cameras that can each image both eyes to provide three-dimensional image information, including z-depth information of both eyes.

[0052] Figure 4 An example of a stereo arrangement of cameras of a camera system 20a according to some embodiments is shown. Cameras A-L and A-R are arranged to be mirror-symmetrical about the system axis 14, i.e., spatially separated on opposite sides of the system axis 14 at equal viewing angles. The images can be stereoscopically reconstructed to track the position and orientation of the eyes in three dimensions. The greater the angle and / or distance between the cameras, the better the accuracy in the z-direction. This can help position the patient's head.

[0053] Figure 5 An example of a stereo coaxial arrangement of cameras of a camera system 20b according to some embodiments is shown. Cameras A-L and A-R are arranged stereoscopically, and cameras B-L and B-R are also arranged stereoscopically. Camera C is arranged coaxially, i.e., aligned with the system axis 14.

[0054] Figure 6 An example of an asymmetric arrangement of cameras of a camera system 20c according to some embodiments is shown. Cameras A and B are arranged asymmetrically at different viewing angles, i.e., these cameras are not mirror-symmetrical with respect to the system axis 14. The cameras in the asymmetric arrangement lack corresponding cameras that are symmetrical about the system axis 14. In this example, neither camera A nor camera B has a corresponding camera that is symmetrical about the system axis 14, so they are asymmetric cameras.

[0055] Figure 7 An example of a method that can be performed by an Figure 1 ophthalmic system 10 according to some embodiments is shown. The method begins at step 110, in which the camera system 20 records an image portion 45 of the eye region 14. The image portion can show features of the eyes and, in some embodiments, can show a light pattern projected onto the eyes.

[0056] In step 114, computer 26 receives image portion 45 from camera system 20. In step 116, computer 26 aligns image portion 45. For example, computer 26 can determine the relative positions of the image portions based on the orientation of the cameras, based on an image of a calibration pattern, or based on user calibration. In some embodiments, in step 118, computer 26 combines the aligned image portions 45 to produce a combined image 46 of eye region 14. The combined image 46 can be a two-dimensional (2D) image for tracking in two dimensions or a three-dimensional (3D) image for tracking in three dimensions, and the three-dimensional image can allow 6D tracking.

[0057] In step 120, computer 26 tracks one or both eyes of eye region 14 based on the image portions and / or combined image 46. The eyes can be tracked in any suitable manner. For example, computer 26 can identify target eye features in the image portions and / or combined image 46 and track the movement of the features to track the eyes. As another example, computer 26 can use the image portion 45 from the camera that produces a better quality image that is more likely to produce a particular feature, e.g., an image generated at a higher speed, higher resolution, infrared light, or visible light. Then the method ends.

[0058] Components of the systems and devices disclosed herein, such as a control computer, can include an interface, logic, and / or memory, any of which can include computer hardware and / or software. The interface can receive inputs to the component and / or send outputs from the component and is generally used to exchange information between, e.g., software, hardware, peripherals, users, and combinations thereof. A user interface is a type of interface by which a user can communicate with a computer (e.g., send inputs to and / or receive outputs from the computer). Examples of user interfaces include a display, a graphical user interface (GUI), a touch screen, a keyboard, a mouse, a gesture sensor, a microphone, and a speaker.

[0059] Logic can perform the operations of the component. Logic can include one or more electronic devices that process data (e.g., execute instructions for generating an output from an input). Examples of such electronic devices include a computer, a processor, a microprocessor (e.g., a central processing unit (CPU)), and a computer chip. Logic can include computer software that encodes instructions executable by an electronic device to perform the operations. Examples of computer software include a computer program, an application, and an operating system.

[0060] A memory can store information and can include a tangible, computer-readable, and / or computer-executable storage medium. Examples of a memory include a computer memory (e.g., random access memory (RAM) or read-only memory (ROM)), a mass storage medium (e.g., a hard disk), a removable storage medium (e.g., a compact disc (CD) or a digital video or versatile disc (DVD)), a database, a network storage device (e.g., a server), and / or other computer-readable media. Certain embodiments may relate to a memory encoded with computer software.

[0061] Although this disclosure is described in terms of certain embodiments, modifications to the embodiments, such as changes, substitutions, additions, omissions, and / or other modifications, will be apparent to those skilled in the art. Accordingly, the embodiments may be modified without departing from the scope of the invention. For example, the systems and devices disclosed herein may be modified. It will be apparent to those skilled in the art that the components of the systems and devices may be integrated or separated, or that the operations of the systems and devices may be performed by more, fewer, or other components. As another example, the methods disclosed herein may be modified. The methods may include more, fewer, or other steps, and the steps may be performed in any suitable order, which will be apparent to those skilled in the art.

[0062] To assist the Patent Office and the reader in understanding the claims, the applicant does not intend for any claim or claim element to invoke 35 U.S.C. § 112(f), unless the phrase "means for" or "step for" is expressly used in a particular claim. The applicant understands that the use of any other term (e.g., "mechanism," "module," "device," "unit," "component," "element," "member," "apparatus," "machine," "system," "processor," or "controller") within a claim refers to a structure known to those of ordinary skill in the relevant art and is not intended to invoke 35 U.S.C. § 112(f).

Claims

1. An ophthalmic system for tracking movement of an eye region, the ophthalmic system comprising: a camera system comprising a plurality of cameras configured to generate a plurality of image portions of the eye region, each camera configured to image at least a portion of the eye region to generate an image portion of the plurality of image portions, the camera system having a system axis and a system field of view, the eye region comprising one or two eyes, each eye of the eye region having an eye center and an eye axis; as well as A computer, the computer being configured to: receiving the plurality of image portions from the camera system; and Movement of at least one eye of the eye region is tracked based on the plurality of image portions.

2. The ophthalmic system of claim 1, wherein the computer is configured to track movement of at least one eye in two dimensions.

3. The ophthalmic system of claim 1, the computer configured to track movement of at least one eye in three dimensions to allow 6D tracking.

4. The ophthalmic system of claim 1, wherein the plurality of cameras comprises: A set of stereo cameras is arranged symmetrically about the system axis.

5. The ophthalmic system of claim 1, wherein the plurality of cameras comprises: A coaxial camera is aligned with the system axis.

6. The ophthalmic system of claim 1, wherein the plurality of cameras comprises: An asymmetrically arranged camera lacking a corresponding camera that is symmetrical about the system axis.

7. The ophthalmic system of claim 1, wherein the plurality of cameras comprises: A high-speed camera is configured to generate images at greater than 400 frames per second.

8. The ophthalmic system of claim 1, wherein the plurality of cameras comprises: A high-resolution camera is configured to generate images having greater than 4 megapixels.

9. The ophthalmic system of claim 1, at least one camera configured to detect a visible light range from the eye region to generate an image portion.

10. The ophthalmic system of claim 1, at least one camera configured to detect an infrared range of light from the eye region to generate an image portion.

11. The ophthalmic system of claim 1, at least one camera configured to detect ultraviolet light range from the eye region to generate an image portion.

12. The ophthalmic system of claim 1: further comprising a light projector configured to direct a light pattern toward at least one eye of the eye region; and At least one camera configured to detect light patterns reflected by the at least one eye.

13. The ophthalmic system of claim 1, wherein the computer is configured to track movement of at least one eye of the eye region based on the plurality of image portions by: aligning the plurality of image portions to produce a combined image of the eye region; and Movement of at least one eye of the eye region is tracked based on the combined image of the eye region.

14. A method for tracking movement of an eye region, the method comprising: providing a plurality of image portions of the eye region by a camera system including a plurality of cameras, each camera being configured to image at least a portion of the eye region to produce an image portion of the plurality of image portions, the camera system having a system axis and a system field of view, the eye region including one or two eyes, each eye of the eye region having an eye center and an eye axis; receiving, by a computer, the plurality of image portions from the camera system; as well as Movement of at least one eye of the eye region is tracked by the computer based on the plurality of image portions.

15. The method of claim 14, further comprising: Track the movement of at least one eye in two dimensions.

16. The method of claim 14, further comprising: The movement of at least one eye is tracked in three dimensions to allow 6D tracking.

17. The method of claim 14, further comprising: Images are generated at greater than 400 frames per second.

18. The method of claim 14, further comprising: Generates images with greater than 4 megapixels.

19. The method of claim 14, further comprising: directing, by a light projector, a light pattern toward at least one eye of the eye region; as well as The light pattern reflected by the at least one eye is detected by at least one camera.

Images