Eye tracker equipped with multiple cameras

Abstract

In certain embodiments, the ophthalmic system tracks the movement of an eye region and includes a camera system and a computer. The camera system has multiple cameras that generate partial images of the eye region, each camera capturing at least a portion of the eye region. The camera system has a system axis and a system field of view. The eye region includes one or both eyes, each eye having an ocular center and an ocular axis. The computer receives partial images from the camera system and tracks the movement of at least one eye in accordance with the partial images.

Description

Technical Field

[0001] The present disclosure generally relates to an ophthalmic system, and more particularly to an eye tracker equipped with a plurality of cameras.

Background Art

[0002] Certain ophthalmic systems utilize an eye tracker to monitor the movement of the eye. For example, in laser in situ keratomileusis (LASIK), laser pulses are directed at the eye in a specific pattern, and tissue is cut and the cornea is reformed. In order to effectively treat the eye, the laser beam should be accurately directed at a specific point on the eye even when the eye is moving. Therefore, an eye tracker is used to monitor the movement of the eye.

Summary of the Invention

[0003] In certain embodiments, an ophthalmic system tracks the movement of an eye region and includes a camera system and a computer. The camera system has a plurality of cameras that generate partial images of the eye region, and each camera captures at least a portion of the eye region. The camera system has a system axis and a field of view. The eye region includes one or both eyes, and each eye has a center and an axis of the eye. The computer receives the partial images from the camera system and tracks the movement of at least one eye based on the partial images.

[0004] Multiple embodiments may not include any of the following features, or may include one, some, or all of the following features.

[0005] * The computer tracks the movement of at least one eye in two dimensions.

[0006] * The computer tracks the movement of at least one eye in three dimensions, enabling 6D tracking.

[0007] * The cameras include a set of stereo cameras arranged symmetrically about the system axis.

[0008] *The camera system includes coaxial cameras aligned with the system axes.

[0009] *The cameras include asymmetrically positioned cameras that do not have corresponding cameras that are symmetrical with respect to the system axis.

[0010] *The cameras include higher-speed cameras that generate images at over 400 frames per second.

[0011] *The camera includes a higher-resolution camera that produces images exceeding 4 megapixels.

[0012] *At least one camera detects the visible light range from the eye region to generate a partial image.

[0013] *At least one camera detects the infrared light range from the eye region to generate a partial image.

[0014] *At least one camera detects the ultraviolet light range from the eye region to generate a partial image.

[0015] * A floodlight directs a light pattern towards at least one eye in the eye region. At least one camera detects the light pattern reflected by at least one eye.

[0016] *The computer aligns partial images to generate a composite image of the eye region and tracks the movement of at least one eye based on the composite image.

[0017] In certain embodiments, a method for tracking the movement of an eye region includes providing partial images of the eye region by a camera system of cameras. The camera system has multiple cameras that generate partial images of the eye region, each camera capturing at least a portion of the eye region. The camera system has a system axis jump system field of view. The eye region includes one or both eyes, each eye having an ocular center and an ocular axis. A computer receives partial images from the camera system and tracks the movement of at least one eye in the eye region according to the partial images.

[0018] The plurality of embodiments may not include any of the following features, or may include one, some, or all of the following features.

[0019] *The method further includes tracking the movement of at least one eye in two dimensions.

[0020] *The method further includes tracking the movement of at least one eye in three dimensions to enable 6D tracking.

[0021] *The method further includes generating images at a rate of more than 400 frames per second.

[0022] *The method further includes generating images of more than 4 megapixels.

[0023] *The method further includes directing a pattern of light at least at one eye by a projector. At least one camera detects the pattern of light reflected by the eye.

Brief Description of the Drawings

[0024] [Figure 1] An example of an ophthalmic system equipped with an eye tracker according to a specific embodiment is shown. [Figure 2A] An example of the field of view (FOV) of the camera system of FIG. 1 according to a specific embodiment is shown. [Figure 2B] An example of the field of view (FOV) of the camera system of FIG. 1 according to a specific embodiment is shown. [Figure 3A-3B] An example of the camera system of FIG. 1 that tracks an exemplary eye region according to a specific embodiment is shown. [Figure 4] An example of the stereo camera arrangement of the camera of the camera system of FIG. 1 according to a specific embodiment is shown. [Figure 5] An example of the stereoscopic and coaxial arrangement of the camera system of FIG. 1 according to a specific embodiment is shown. [Figure 6] An example of the asymmetric arrangement of the cameras of the camera system of FIG. 1 according to a specific embodiment is shown. [Figure 7] An example of a method that can be performed by the ophthalmic system of FIG. 1 according to a particular embodiment is shown. **DETAILED DESCRIPTION**

[0025] Here, referring to the description and the drawings, exemplary embodiments of the disclosed devices, systems, and methods are shown in detail. The description and the drawings are not intended to be exhaustive or to limit the claims to the specific plurality of embodiments illustrated and disclosed herein. The drawings represent possible embodiments, but the drawings may not necessarily be to an exact scale and may simplify, exaggerate, exclude, or partially divide certain features to more clearly illustrate the embodiments.

[0026] In certain eye trackers, a projector directs light at the eye at a known angle and a camera generates an image showing the light reflection at the eye. Assumptions based on a standard eye model are used to identify the eye movement from the camera image. However, this assumption may not accurately describe the eye of a particular patient, resulting in a decrease in tracking accuracy.

[0027] The eye tracker described herein can provide more accurate tracking because it does not require assumptions of an eye model. The eye tracker includes a camera system having cameras that image the eye from different directions, such as coaxially and obliquely. From the known locations of the cameras, the eye movement can be identified from the resulting images. The tracker can track, for example, translational and / or rotational movements in the x, y, and / or z directions. In certain embodiments, the cameras can record infrared light (IR), visible light, and / or other light and can record images at higher speed and / or higher resolution. The eye tracker can be used in an ophthalmic diagnostic and / or treatment system (e.g., in refractive or cataract surgery).

[0028] Figure 1 shows an example of an ophthalmic system 10 equipped with an eye tracker 12 that monitors an ocular region 14 including one or both eyes of a patient, according to a specific embodiment. Generally, the eye tracker 12 tracks eye movement by monitoring the movement of one or more features of the eye in the image (e.g., pupil, iris, blood vessels, limbus, sclera, eyelashes, and / or eyelids).

[0029] To facilitate explanation, specific eye features are used to define an exemplary coordinate system 16 (x, y, z) for the eye. For example, an eye has a center (e.g., pupil center, corneal apex, vertex) and an axial length 15 (e.g., optical axis or pupillary axis) that can define the z-axis of the eye coordinate system 16, which in turn defines the xy-plane of the coordinate system 16. An eye region 14 has a region axis 17. If the eye region 14 includes one eye, the region axis 17 may substantially coincide with the axial length 15. If the eye region 14 includes both eyes, the region axis 17 may pass through the midpoint between the two eyes.

[0030] As an example of a system overview, the ophthalmic system 10 includes an eye tracker 12, an ophthalmic device 22, a display 24, and a computer 26 (including logic 27 and memory 28), which are connected as shown in the figure. The eye tracker 12 includes a camera system 20 and the computer 26, which are connected as shown in the figure. In a particular embodiment, the eye tracker 12 includes a projector 30 to enable tracking in the z direction. As an example of an overview of operation, the camera system 20 of the eye tracker 12 has multiple cameras that generate partial images of the eye region 14. Each camera is positioned at a known location (e.g., a known position and / or orientation relative to each other and / or relative to the eye region 14) and records at least a portion of the eye region 14 to generate a partial image. The known locations allow for the calculation of eye movements, as will be described in more detail below. The computer 26 receives the partial images from the camera system 20 and tracks the movement of at least one eye according to the partial images.

[0031] Looking at the components of this example, the eye tracker 12 can track eye movements in six "dimensions" (6D), which is known as "6D tracking." The six dimensions include x-translation, y-translation, z-translation, rotation, x-rolling, and / or y-rolling motion with respect to the eye coordinate system 16. In a particular embodiment, x, y, and z-translation motions may be translational motions in the x, y, and z directions, respectively. Rotational motions may be motions around the eye axis 15. x and y-rolling motions may be rotational motions around the x-axis and y-axis, respectively. In a particular embodiment, 6D tracking may track some or all of the 6D motions.

[0032] In certain embodiments, the eye tracker 12 includes a camera system 20 that generates images of the eye region 14. The camera system 20 has a field of view (FOV) (more described in detail with respect to Figures 2A and 2B) that covers the eye region 14. The FOV has a known relationship with the coordinate system of the camera system 20, which in turn has a known relationship with the coordinate system used by the ophthalmic device 22 for treating and / or diagnosing the eye. The eye tracker 12 tracks eye movements by tracking eye movements relative to the FOV. The eye tracking information can be used by the ophthalmic device 22 to treat and / or diagnose the eye.

[0033] In the embodiment, the camera system 20 includes a camera. For ease of explanation, the “location” of the camera with respect to the eye region 14 can refer to the distance between the camera and the eye region 14, and the direction of the camera axis with respect to the region axis 17. The camera detects light from an object and generates a signal in response to the light. The signal carries image data that can be used to generate an image of the eye. The image data is provided to a computer 26 for eye tracking (and optionally other analysis) and may also be provided to a display 24 to present the image of the eye. Examples of cameras include charge-coupled devices (CCDs), video, complementary metal-oxide-semiconductor (CMOS) sensors (e.g., active pixel sensors (APS)), line sensors, and optical coherence tomography (OCT) cameras.

[0034] The camera detects light within any suitable spectral range, which is, for example, the range of infrared (IR), ultraviolet (UV), and / or visible (VIS) wavelengths, and the range may include some or all of the wavelengths. For example, the camera may detect visible light, infrared light, or other visible and infrared light from the eye region 14 to produce a partial image. Certain cameras may capture eye features (e.g., pupil, iris structure, blood vessels, limbus, etc.) better than other cameras. For example, infrared cameras generally provide more stable pupil tracking and higher contrast for the iris structure. Therefore, an IR camera may be used to monitor lateral motion by tracking the pupil and / or iris structure. As another example, a visible range camera may be used to monitor translational and / or rotational motion by tracking blood vessels, as it produces better images of blood vessels.

[0035] A camera can record images at any suitable frequency or resolution. A higher-speed camera may record images at, for example, over 400-1500 frames per second, e.g., over 500, 750, or 1000 frames per second. A higher-resolution camera may produce images at, for example, over 4-24 megapixels, e.g., over 5, 10, 15, or 20 megapixels. Generally, acquiring higher-resolution and higher-speed images can lead to more accurate tracking, but both features may require longer computation times, so a trade-off may exist between resolution and speed. Therefore, the camera's speed and / or resolution may be selected for a particular purpose. In certain embodiments, a higher-speed camera may track eye features that are moving faster and / or can be identified at a lower resolution, while a higher-resolution camera may be used to track eye features that require a higher resolution for identification and / or are moving slower. For example, a lower-resolution, higher-speed camera could track the pupil (which doesn't require high resolution) to detect x and y motion. Another example is a higher-resolution, lower-speed camera that could track blood vessels / iris structures to detect rotation and z motion.

[0036] The ophthalmic device 22 may be a system used to diagnose and / or treat the eye. Examples include refractive surgery systems, cataract systems, topographers, OCT measurement devices, and wavefront measurement devices. The display 24 provides images, such as partial images and / or composite images, to the user of system 10. Examples of the display 24 include computer monitors, 3D displays, projectors / beamers, TV monitors, binocular displays, eyeglasses with monitors, virtual reality displays, augmented reality displays, and mixed reality displays.

[0037] The projector 30 directs a pattern of light onto the eye region 14, and the reflection of the light is used to track the eye. The projector 30 may include one or more light sources that generate the pattern of light. The projection of light can be used in any suitable way. For example, the light can be directed at a known angle and used to align a partial image. As another example, the line projection is distorted by the curvature of the eye, so the distortion of the line can help identify the boundary between the cornea and sclera, where the curvature changes. As yet another example, symmetrical projection can be used to identify the vertex of the eye or the apex of the cornea. As yet another example, a stripe projector can project lines obliquely onto the eye, so the lines appear curved on the cornea, and the curvature changes as the eye moves. Any suitable pattern can be used, for example, lines (such as stripes), crosses, and / or arrays of lines and / or dots.

[0038] Computer 26 controls components of system 10 (e.g., camera system 20, ophthalmic device 22, display 24, and / or illuminator 30) to track the eye. In this example, computer 16 receives partial images from camera system 20 and tracks the movement of at least one eye based on the partial images. In a particular embodiment, computer 26 aligns the partial images to generate a composite image of the eye region 14 and tracks the movement of at least one eye based on the composite image.

[0039] Figures 2A and 2B show an example of the field of view (FOV) 40 of the camera system 20 of Figure 1 according to a particular embodiment. The cameras of the camera system 20 have a field of view (FOV) that detects light from the eye region 14 and generates a partial image 45 of part or all of the eye region 14. Different cameras may have different FOVs that detect light from different parts of the eye region in different directions, and these different FOVs may overlap. The combination of FOVs from the cameras yields the system FOV 40. In general, more cameras in different locations (positions and orientations) can improve the accuracy of detecting and tracking eye features.

[0040] In this example, the camera system 20 has a system FOV 40, a system axis 42, and a system coordinate system 44 (x', y', z'). The system axis 42 may be at any suitable position; for example, the axis 42 may be substantially orthogonal to the system FOV 40 and pass through the center of the system FOV 40. The system axis 42 and the system coordinate system 44 (x', y', z') may be related in any suitable way. In this example, the system axis 42 defines the z' axis of the system coordinate system 44. In this example, the system FOV 40 is generally planar and images the numbers 1 through 9. The camera system 20 includes camera A having FOV A and camera B having FOV B. FOV A covers the system FOV 40 (i.e., image numbers 1 through 9), and FOV B covers only a portion of the system FOV 40 (i.e., image numbers 4 through 9). Camera A produces partial image A, and camera B produces partial image B.

[0041] In certain embodiments, the computer 26 aligns and combines the partial images 45 to generate a composite image 46. The partial images 45 can be aligned by any suitable method. For example, each camera has known locations, such as position (e.g., distance from the system FOV 40 and / or eye region 14), orientation (e.g., camera optical axis with respect to the system axis 42 and / or eye axis 15, or field of view), dimensions, and imaging characteristics. From this information, the computer 26 can identify the locations of the partial images 45 and align them in the composite image 46. As another example, each camera generates an image of a calibration diagram (e.g., a checkerboard), from which the camera's location is identified. As yet another example, the user calibrates the partial images 45 by manually aligning them as viewed through the camera. The computer 26 records the locations of the aligned partial images.

[0042] The partial images 45 can be combined in any suitable way. For example, the partial images 45 can be combined to generate a two-dimensional (2D) image to enable 2D tracking, and / or the partial images 45 (e.g., from a stereo camera) can be combined to generate a three-dimensional (3D) image to enable 3D tracking.

[0043] The eye tracker 12 tracks one or both eyes of the eye region 14 according to the partial and / or composite images 46. For example, the computer 26 tracks the eye by identifying target eye features (e.g., pupil, iris structure, or blood vessels) in the uncomposite or composite partial images and tracking the movement of these features relative to the system FOV 40. The computer 26 may identify a feature using a partial image 45 from a camera that is more likely to produce a higher quality image of that feature. For example, a camera may have a higher FOV, wavelength, resolution, and / or speed that is more likely to capture a feature. Examples of cameras having such characteristics for imaging specific features are presented throughout this specification.

[0044] Figures 3A and 3B show an example of the camera system 20 of Figure 1 tracking an eye region 14 according to a particular embodiment. In Figure 3A, the eye region 14 includes one eye. In this example, the eye axis 15 can first be substantially aligned with the system axis 42 of the camera system 20. As the eye moves relative to the camera system 20, the eye axis 15 moves relative to the system axis 42.

[0045] In Figure 3B, the eye region 14 includes both eyes. The system axis 42 of the camera system 20 is substantially aligned with the midpoint between the two eyes. The camera system 20 includes multiple cameras that capture one or both eyes to produce partial and / or composite images of both eyes simultaneously, so that the camera system 20 can track both eyes simultaneously and independently of each other. In certain embodiments, the camera system 20 includes a pair of stereo cameras, each capable of capturing both eyes separately to provide three-dimensional image information including z-depth information for both eyes.

[0046] Figure 4 shows an example of the stereoscopic arrangement of cameras in a camera system 20a according to a specific embodiment. Cameras A-L and A-R are arranged mirror-symmetrically around the system axis 14, i.e., spatially separated on both sides of the system axis 14 with equal field of view angles. The images can be reconstructed stereoscopically to track the position and orientation of the eyes in three dimensions. The greater the angle and / or distance between the cameras, the higher the accuracy in the z-direction. This can facilitate the positioning of the patient's head.

[0047] Figure 5 shows an example of stereoscopic and coaxial camera arrangement in a camera system 20b according to a specific embodiment. Cameras AL and AR are arranged stereoscopically, and cameras BL and BR are also arranged stereoscopically. Camera C is arranged coaxially, i.e., aligned with the system axis 14.

[0048] Figure 6 shows an example of the asymmetric arrangement of cameras in a camera system 20c according to a particular embodiment. Cameras A and B are arranged asymmetrically with different field of view angles; that is, the cameras are not mirror-symmetric with respect to the system axis 14. Asymmetrically arranged cameras do not have corresponding cameras that are symmetric with respect to the system axis 14. In this example, since neither camera A nor camera B has corresponding cameras that are symmetric with respect to the system axis 14, they are asymmetric cameras.

[0049] Figure 7 shows an example of a method that can be performed by the ophthalmic system 10 of Figure 1 according to a specific embodiment. The method begins with 110, where the camera system 20 records a partial image 45 of the eye region 14. The partial image may show features of the eye and, in some embodiments, may show a light pattern projected onto the eye.

[0050] In step 114, the computer 26 receives a partial image 45 from the camera system 20. In step 116, the computer 26 aligns the partial image 45. For example, the computer 26 may determine the relative location of the partial image from the camera location, from a calibration diagram image, or from user calibration. In certain embodiments, in step 118, the computer 26 combines the aligned partial images 45 to generate a composite image 46 of the eye region 14. The composite image 46 may be a two-dimensional (2D) image for two-dimensional tracking or a three-dimensional (3D) image for three-dimensional tracking, thereby enabling 6D tracking.

[0051] In step 120, the computer 26 tracks one or both eyes of the eye region 14 according to the partial and / or composite image 46. The eyes can be tracked by any suitable method. For example, the computer 26 may track the eyes by identifying a target eye feature in the partial and / or composite image 46 and tracking the movement of that feature. As another example, the computer 26 may track a particular feature using a partial image 45 from a camera that is likely to produce a higher quality image of that feature, for example, an image produced at higher speed, higher resolution, infrared, or visible light. The method is now complete.

[0052] The components of the systems and apparatus disclosed herein (such as control computers) may include interfaces, logic, and / or memory, any of which may include computer hardware and / or software. Interfaces can receive inputs to and / or transmit outputs from components and are typically used to exchange information between, for example, software, hardware, peripherals, users, and combinations thereof. User interfaces are a type of interface that a user can use to communicate with a computer (for example, to send inputs to and / or receive outputs from a computer). Examples of user interfaces include displays, graphical user interfaces (GUIs), touchscreens, keyboards, mice, gesture sensors, microphones, and speakers.

[0053] Logic can perform the actions of components. Logic may include one or more electronic devices that process data, for example, by executing instructions to produce an output from an input. Examples of such electronic devices include computers, processors, microprocessors (e.g., central processing units (CPUs)), and computer chips. Logic may also include computer software that encodes instructions that can be executed by the electronic devices to perform actions. Examples of computer software include computer programs, applications, and operating systems.

[0054] Memory may include tangible, computer-readable and / or computer-executable storage media capable of storing information. Examples of memory include computer memory (e.g., random access memory (RAM) or read-only memory (ROM)), mass storage media (e.g., hard disks), removable storage media (e.g., compact discs (CDs) or digital video or versatile discs (DVDs)), databases, network storage (e.g., servers), and / or other computer-readable media. Specific embodiments may relate to memory encoded in computer software.

[0055] While this disclosure has described certain embodiments, modifications to these embodiments (e.g., changes, substitutions, additions, omissions, and / or other modifications) will be obvious to those skilled in the art. Therefore, modifications to embodiments can be made without departing from the scope of the invention. For example, modifications can be made to the systems and apparatus disclosed herein. As will be obvious to those skilled in the art, the components of the systems and apparatus may be integrated or separated, or the operation of the systems and apparatus may be performed by more components, fewer components, or other components. Another example is the modification of the methods disclosed herein. As will be obvious to those skilled in the art, the methods may include more steps, fewer steps, or other steps, and the steps may be performed in any suitable order.

[0056] To assist the Patent Office and readers in interpreting the claims, the applicants note that, unless the terms “means for” or “steps for” are expressly used in a particular claim, no claim or claim element is intended to be subject to Section 112(f) of the U.S. Patent Act. The applicants understand that any other terms used in the claims (e.g., “mechanism,” “module,” “device,” “unit,” “component,” “element,” “member,” “apparatus,” “machine,” “system,” “processor,” or “controller”) refer to structures known to those skilled in the art and are not intended to be subject to Section 112(f) of the U.S. Patent Act.

Claims

1. An ophthalmic system that tracks eye movements, A camera system comprising a plurality of cameras configured to generate a plurality of partial images of the eye region, each camera configured to capture at least a portion of the eye region and generate a partial image from the plurality of partial images, the camera system having a system axis and a system field of view, the eye region including one or both eyes, and each eye in the eye region having an eye center and an eye axis, It is a computer, The camera system receives the plurality of partial images, The movement of at least one eye in the eye region is tracked according to the plurality of partial images. A computer configured as follows, An ophthalmic system including

2. The ophthalmic system according to claim 1, wherein the computer is configured to track the movement of at least one eye in two dimensions.

3. The ophthalmic system according to claim 1, wherein the computer is configured to track the movement of at least one eye in three dimensions to enable 6D tracking.

4. The ophthalmic system according to claim 1, wherein the plurality of cameras include a set of stereo cameras arranged symmetrically around the system axis.

5. The ophthalmic system according to claim 1, wherein the plurality of cameras include a coaxial camera aligned with the system axis.

6. The ophthalmic system according to claim 1, wherein the plurality of cameras include cameras arranged asymmetrically, and the asymmetrically arranged cameras do not have corresponding cameras that are symmetrical with respect to the system axis.

7. The ophthalmic system according to claim 1, wherein the plurality of cameras include higher-speed cameras configured to generate images at more than 400 frames per second.

8. The ophthalmic system according to claim 1, wherein the plurality of cameras include higher resolution cameras configured to produce images of more than 4 megapixels.

9. The ophthalmic system according to claim 1, wherein at least one camera is configured to detect the visible light range from the eye region and generate a partial image.

10. The ophthalmic system according to claim 1, wherein at least one camera is configured to detect an infrared light range from the eye region and generate a partial image.

11. The ophthalmic system according to claim 1, wherein at least one camera is configured to detect the ultraviolet light range from the eye region and generate a partial image.

12. A projector configured to direct a pattern of light towards at least one eye in the aforementioned eye region, At least one camera configured to detect the light pattern reflected by at least one eye, The ophthalmic system according to claim 1, further comprising:

13. The computer, in accordance with the plurality of partial images, determines the movement of at least one eye in the eye region. The process involves aligning the aforementioned multiple partial images to generate a composite image of the eye region, Tracking the movement of at least one eye in the eye region in accordance with the composite image of the eye region. The ophthalmic system according to claim 1, configured to track by

14. A method for tracking eye movement, A camera system comprising multiple cameras provides multiple partial images of the eye region, each camera configured to capture at least a portion of the eye region and generate a partial image from among the multiple partial images, the camera system has a system axis and a system field of view, the eye region includes one or both eyes, and each eye in the eye region has an ocular center and an ocular axis. The computer receives the multiple partial images from the camera, The computer tracks the movement of at least one eye in the eye region according to the plurality of partial images. A method that includes this.

15. Tracking the aforementioned movement of at least one eye in two dimensions. The method according to claim 14, further comprising:

16. To enable 6D tracking by tracking the aforementioned movement of at least one eye in three dimensions. The method according to claim 14, further comprising:

17. Generating images at over 400 frames per second The method according to claim 14, further comprising:

18. To generate images with more than 4 megapixels The method according to claim 14, further comprising:

19. By directing a light pattern towards at least one eye in the aforementioned eye area using a projector, The detection of the light pattern reflected by at least one eye is performed using at least one camera. The method according to claim 14, further comprising:

Images