Eye tracking device and eye tracking method

Abstract

An eye tracking device and an eye tracking method are disclosed. The eye tracking device includes an optical element, an emitting element, and an image capturing element. The optical element corresponds to an eye. The optical element includes a feature pattern. The emitting element is disposed adjacent to the optical element. The emitting element provides a light signal to the optical element to cause the feature pattern to be mapped to the eye. The image capturing element is disposed adjacent to the eye. The image capturing element captures an image of the eye.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to an eye tracking device and an eye tracking method. BACKGROUND

[0002] With the development of technology, the eye tracking device and the eye tracking method are applied more widely. For example, an advertiser can determine the advertising content to be put according to the length of time that a consumer gazes at an advertisement. For example, a user can control a device through the movement of the eye. For example, in recent years, head-mounted display (HMD) devices have become popular, especially HMD devices that apply virtual reality (VR) technology, augmented reality (AR) technology, etc. If the HMD device can have an eye tracking function, the user experience can be further improved.

[0003] The conventional eye tracking device and the eye tracking method directly irradiate the eye with a plurality of emitting elements (e.g., light sources) to generate a feature pattern used to identify the position of the eye. However, the plurality of emitting elements need to be disposed on the circuit element, so a relatively large space is required. Moreover, a reflecting element (e.g., a hot mirror) with a relatively large volume can be required to reflect the feature pattern on the eye. Therefore, the conventional eye tracking device and the eye tracking method are not conducive to reducing costs and miniaturizing the HMD device. SUMMARY

[0004] Some embodiments of the present disclosure provide an eye tracking device. The eye tracking device includes an optical element, an emitting element, and an image capturing element. The optical element corresponds to an eye. The optical element includes a feature pattern. The emitting element is disposed adjacent to the optical element. The emitting element can provide a light signal to the optical element, so that the feature pattern is mapped to the eye. The image capturing element is disposed adjacent to the eye. The image capturing element can capture an image of the eye.

[0005] In some embodiments, the eye tracking device further comprises a front housing and a rear housing coupled to the front housing. In some embodiments, the optical element comprises a viewing area and a peripheral area, the viewing area is closer to the eye than the peripheral area, and the feature pattern is formed on the peripheral area. In some embodiments, the light signal provided by the light emitting element is an invisible light. In some embodiments, the light signal provided by the light emitting element is an infrared light. In some embodiments, the optical element further comprises a coating layer coated on a surface of the optical element. The coating layer has a transmittance of a visible light between 90% and 100%. In some embodiments, the coating layer has a transmittance of the light signal provided by the light emitting element between 0% and about 10%. In some embodiments, the feature pattern comprises a plurality of patterns. In some embodiments, the feature pattern is formed on the optical element by superfinishing, such as laser ablation.

[0006] Some embodiments of the present disclosure provide an eye tracking method. The method comprises generating a light signal by a light emitting element. The light signal can enter an optical element comprising a feature pattern, such that the feature pattern is mapped to an eye. The method further comprises capturing an image of the eye, and identifying a position of the eye based on the feature pattern on the eye. BRIEF DESCRIPTION OF DRAWINGS

[0007] In order that the features or advantages of the present disclosure can be more fully understood, some embodiments will now be described in detail with the accompanying drawings. It should be noted that the various features will not necessarily be drawn to scale in the drawings. Indeed, the dimensions of the various features can be arbitrarily expanded or reduced for the sake of clarity. Further, it should be noted that some of the features of the various embodiments can be used in combination with one another, while some of the features can be used exclusively from one another.

[0008] Figure 1 is a schematic diagram of an eye and an eye tracking device;

[0009] Figure 2 is an exploded view of an eye tracking device;

[0010] Figures 3 to 6 is a schematic diagram of optical elements with different feature patterns;

[0011] Figure 7 is a flowchart of an eye tracking method;

[0012] Figures 8 to 10 is a schematic diagram of a head-mounted display device with eye tracking function.

[0013] LIST OF ABBREVIATIONS

[0014] 10: eye

[0015] 100: eye tracking device

[0016] 110: front housing

[0017] 120: rear housing

[0018] 130: optical element

[0019] 131: coating

[0020] 132: viewable area

[0021] 133: peripheral area

[0022] 140: emitting element

[0023] 141: optical signal

[0024] 150: circuit element

[0025] 160: image capturing element

[0026] 170, 170A, 170B, 170B, 170D: feature pattern

[0027] 200: eye tracking method

[0028] S201, S202, S203, S204: step

[0029] 300, 400, 500: head-mounted display device

[0030] 301: body

[0031] 302: arm

[0032] 401: main body

[0033] 402: headband

[0034] 501: housing DETAILED DESCRIPTION

[0035] In this specification, numerous different embodiments or examples are provided, and relative terms are used to describe particular examples or implementations of the different features of the application. For example, if a first feature is said to be formed "on" a second feature, this can include embodiments where the first feature is formed directly on the second feature, as well as embodiments where additional features are formed between the first feature and the second feature such that the first feature is not formed directly on the second feature. Relative terms are used to describe the relationship between elements or features in the drawings. In addition to the orientation of the elements in the drawings, relative terms are intended to encompass different orientations of the device in use or operation. The device can be otherwise oriented (rotated 90 degrees or at other orientations) and the relative terms used herein are to be construed accordingly. In addition, like or similar symbols or letters are used to denote like or similar elements or features.

[0036] In this specification, the words "comprise", "contain", and "have" and the like are to be interpreted inclusively rather than exclusively. Thus, when a feature is said to be "comprised in" or "have" or "include" one or more features, this specification also contemplates that the feature can further include one or more other features.

[0037] In this specification, the terms "about" and "substantially" generally mean within 10% or within 5%, or within 3%, or within 2%, or within 1%, or within 0.5% of a given value or range. Also, the term "between a first value and a second value" means that the range includes the first value, the second value, and other values therebetween.

[0038] Reference is first made to Figure 1 and Figure 2 . Figure 1 is a schematic view of an eye 10 and an eye tracking device 100. Figure 2 is an exploded view of the eye tracking device 100. For simplicity, only a single eye 10 is depicted in Figure 1 It is understood that the eye tracking device 100 can be used with both eyes of a user.

[0039] In this embodiment, the eye tracking device 100 includes a front housing 110, a rear housing 120, an optical element 130, an emitting element 140, a circuit element 150, and an image capturing element 160. However, elements can be added or removed as desired.

[0040] The rear housing 120 can be connected to the front housing 110. In some embodiments, the front housing 110 and the rear housing 120 can be secured by locking elements such as screws or glue. The optical element 130, the emitting element 140, the circuit element 150, and the image capturing element 160 can be disposed between the front housing 110 and the rear housing 120. The front housing 110 and the rear housing 120 can house and protect the optical element 130, the emitting element 140, the circuit element 150, and the image capturing element 160. In addition, the connected front housing 110 and rear housing 120 can be sealed to prevent dust from entering the front housing 110 and the rear housing 120. Dust can be magnified due to the optical properties of the optical element 130, thereby interfering with the user experience. Thus, the front housing 110 and the rear housing 120 are typically made of materials that do not attract dust. For example, the front housing 110 and the rear housing 120 can include plastic materials, but are not limited thereto.

[0041] The optical element 130 corresponds to the eyeball 10. Specifically, the eyeball 10 views the display screen through the optical element 130. The optical element 130 can be transparent. The optical element 130 can be a lens, such as a general lens, a Fresnel lens, or the like. The optical element 130 can be made of plastic or glass. The optical element 130 is lighter and less expensive when made of plastic, and the optical properties are better when made of glass. The optical element 130 can have different shapes, such as a circular shape, an elliptical shape, a polygonal shape, or the like. In some embodiments, the shape of the optical element 130 depends on the shape of the optical element in the head-mounted display device to be matched.

[0042] The emitting element 140 is disposed on the circuit element 150 and adjacent to the optical element 130. The emitting element 140 can provide a light signal 141 (or electromagnetic radiation) to the optical element 130. In some embodiments, the light signal 141 is invisible light, so as to reduce the possibility of interfering with the user. In some embodiments, the light signal 141 is infrared light. For example, the emitting element 140 can be an infrared light light emitting diode (IR-LED). The infrared light light emitting diode can convert electrical energy into infrared light signals in the infrared wavelength range (about 700 nanometers (nm) to about 1000 nm), and has the characteristics of low heat generation, low power consumption, etc. The infrared light light emitting diode can include gallium arsenide (GaAs), gallium aluminum arsenide (GaAlAs), etc., but is not limited thereto. Figure 2 In some embodiments, the light signal 141 provided by the emitting element 140 is invisible light, so as to reduce the possibility of interfering with the user. In some embodiments, the light signal 141 provided by the emitting element 140 is infrared light. For example, the emitting element 140 can be an infrared light light emitting diode (IR-LED). The infrared light light emitting diode can convert electrical energy into infrared light signals in the infrared wavelength range (about 700 nanometers (nm) to about 1000 nm), and has the characteristics of low heat generation, low power consumption, etc. The infrared light light emitting diode can include gallium arsenide (GaAs), gallium aluminum arsenide (GaAlAs), etc., but is not limited thereto.

[0043] The circuit element 150 is disposed below the optical element 130. The circuit element 150 can be a circuit board. For example, the circuit element 150 can be a hard board, a flex board, a rigid-flex board, etc., but is not limited thereto.

[0044] The image capturing element 160 is disposed adjacent to the eyeball 10. The image capturing element 160 can capture an image of the eyeball 10. The image capturing element 160 can be a Charge-coupled Device (CCD) or a CMOS image sensor, but is not limited thereto. Notably, in the present embodiment, the image capturing element 160 is disposed on the bottom side of the front housing 110 and the rear housing 120. When a user views a screen, because the image capturing element 160 is located below relative to the eyeball 10, the possibility of disturbing the user can be reduced. However, the image capturing element 160 can be disposed at other positions. Furthermore, in the present embodiment, the emitting element 140, the circuit element 150, and the image capturing element 160 are disposed on the same side of the front housing 110 and the rear housing 120, so that space can be effectively utilized and miniaturization can be achieved.

[0045] The optical element 130 includes a feature pattern 170. In some embodiments, the feature pattern 170 can include a plurality of patterns. In some embodiments, the feature pattern 170 can be a plurality of holes. The light signal 141 provided by the emitting element 140 can exit from the feature pattern 170 after entering the optical element 130, so that the feature pattern 170 can be mapped to the eyeball 10. In a specific embodiment, each pattern of the feature pattern 170 is not completely identical in distance from the emitting element 140, so that the patterns of the feature pattern 170 mapped to the eyeball 10 have energy (e.g., brightness) differences, to facilitate identification of the position of the eyeball 10.

[0046] In some embodiments, the optical element 130 can include a coating 131. The coating 131 can be coated on the surface of the optical element 130, including but not limited to the front surface and the rear surface of the optical element 130. In the present embodiment, the feature pattern 170 is the portion of the front surface (the side close to the eyeball 10) of the optical element 130 that is not coated with the coating 131. In some embodiments, after the coating 131 is coated on the optical element 130, the feature pattern 170 can be formed on the front surface of the optical element 130 by superfinishing methods, such as micro-cutting, high-precision grinding, high-precision polishing, or laser ablation. The aforementioned superfinishing methods can facilitate control of the number, shape, area, arrangement, etc. of the patterns of the feature pattern 170. However, the feature pattern 170 can be formed on the optical element 130 by any suitable method.

[0047] The transmittance of the coating 131 to visible light can be between about 90% and about 100%. For example, the transmittance of the coating 131 to visible light can be about 92%, about 95%, about 98%, but is not limited thereto. In this way, after visible light enters the optical element 130, most of the visible light can pass through the optical element 130 smoothly, so that the possibility of insufficient brightness of the visible light can be reduced, and the possibility of disturbing the user can be reduced.

[0048] In addition, the transmittance of the coating 131 to the light signal 141 provided by the emission element 140 can be between about 0% and about 10%. For example, the transmittance of the coating 131 to the light signal 141 can be about 1%, about 2%, about 5%, about 8%, but is not limited thereto. In this way, after the light signal 141 enters the optical element 130, most of the light signal 141 can be reflected one or more times inside the optical element 130 before being emitted from the feature pattern 170, so as to ensure that the brightness of the feature pattern 170 is sufficient to be mapped to the eyeball 10. In addition, in order to make the light signal 141 smoothly enter the optical element 130, the coating 131 is usually not applied to the area of the optical element 130 corresponding to the emission element 140.

[0049] After the feature pattern 170 is mapped to the eyeball 10, the image capturing element 160 can capture the image of the eyeball 10. The image of the eyeball 10 can be transmitted to an operation unit, such as an image processing unit including a visual processing chip. Through operation, the position of the eyeball 10 can be identified and determined based on the feature pattern 170 on the eyeball 10.

[0050] In addition, the optical element 130 includes a visible area 132 and a peripheral area 133. When a user watches a display screen, the eyeball 10 mainly corresponds to the visible area 132 of the optical element 130. That is, the visible area 132 is closer to the eyeball 10 than the peripheral area 133. In some embodiments, in order to reduce the possibility of the user seeing the feature pattern 170, the feature pattern 170 is formed in the peripheral area 133. Figure 1 In addition, in Figure 2 , the range of the visible area 132 is schematically shown by a dashed line. In some embodiments, in order to reduce the possibility of the user seeing the feature pattern 170, the feature pattern 170 is formed in the peripheral area 133.

[0051] The number, shape, area, arrangement, etc. of the patterns of the feature pattern 170 are not limited to the embodiments shown in Figure 1 and Figure 2 . Next, please refer to Figure 3 to 6 . Figures 3 to 6 is a schematic diagram of the optical element 130 with different feature patterns 170A, 170B, 170C, 170D. As shown in Figure 3 , the feature pattern 170A includes six patterns, and each pattern has a circular shape. As shown in Figure 4 , the feature pattern 170B includes ten patterns, and each pattern has a circular shape. As shown in Figure 5 , the feature pattern 170C includes six patterns, and each pattern has a square shape. As shown in Figure 6 , the feature pattern 170D includes six patterns, and each pattern has a square shape and an area that is not completely the same.

[0052] When the number of patterns included in the feature pattern 170 is reduced, the cost can be reduced. When the number of patterns included in the feature pattern 170 is increased, the shapes are different, or the areas are different, the recognition accuracy can be improved. In other words, the feature pattern 170 can be determined according to actual requirements.

[0053] It is noted that the conventional eye tracking method directly irradiates the eye with a plurality of emitting elements to generate the feature pattern. For example, when the feature pattern includes ten patterns, ten emitting elements are needed to generate the feature pattern. Moreover, the conventional eye tracking method can need a reflecting element with a relatively large volume to reflect the feature pattern on the eye. The eye tracking device 100 of the present application can greatly reduce the number of emitting elements 140, so the cost can be reduced. In some embodiments, for a single eye 10, only a single emitting element 140 can be needed. Also, the eye tracking device 100 of the present application can not need to be provided with a reflecting element to reflect the feature pattern on the eye 10, so the volume of the eye tracking device 100 can be reduced to achieve miniaturization.

[0054] Furthermore, in the test development stage of the eye tracking device 100, when different feature patterns are tested, the emitting element 140 and the circuit element 150 do not need to be adjusted, only the optical element 130 with different feature patterns (for example, the feature patterns 170, 170A, 170B, 170C, 170D) need to be replaced, which can reduce the cost and simplify the process. However, for the conventional eye tracking device, when different feature patterns are tested, a plurality of emitting elements must be replaced together with the circuit element, so a higher cost and a longer time are needed.

[0055] Next, please refer to Figure 7 . Figure 7 is a flowchart of the eye tracking method 200. It will be combined with Figure 7The eye tracking device 100 is capable of eye tracking. The eye tracking method 200 includes steps S201, S202, S203, and S204. In step S201, a light signal is generated by a light emitting element. For example, the light emitting element 140 can generate the light signal 141, such as an infrared light signal. In step S202, the light signal enters an optical element including a feature pattern, so that the feature pattern is mapped to an eye. For example, the light signal 141 can enter the optical element 130 including the feature pattern 170, and the energy intensity of the light signal 141 exiting the feature pattern 170 is sufficient to map the feature pattern 170 to the eye 10. In step S203, an image of the eye is captured. For example, the image capturing element 160 can capture the image of the eye 10. In step S204, the position of the eye is identified based on the feature pattern on the eye. For example, the image of the eye 10 can be transmitted to a computing unit, and the position of the eye 10 is determined by computation, i.e., based on the relative position of the eye and the feature pattern in the image, the position and gaze direction of the eye can be tracked or determined.

[0056] In a specific embodiment, if the horizontal position of the eye in the captured image is located at the upper half of the feature pattern, it can represent that the user is looking upward; if the horizontal position of the eye in the captured image is located at the lower half of the feature pattern, it can represent that the user is looking downward; if the eye in the captured image is located between all the patterns of the feature pattern, it can represent that the user is looking forward; if the position of the eye in the captured image is close to one side of the feature pattern, it can represent that the user is looking at that side.

[0057] The eye tracking device 100 and the eye tracking method 200 can be applied in different fields. In some embodiments, the eye tracking device 100 and the eye tracking method 200 can be combined with a head-mounted display device. The head-mounted display device with eye tracking function can interact with the display screen according to the movement of the eye 10, so as to further improve the user experience. For example, the entire eye tracking device 100 can be arranged on the side of the head-mounted display device close to the eye 10.

[0058] Next, please refer to Figures 8 to 10 . Figures 8 to 10 are schematic diagrams of head-mounted display devices 300, 400, 500 with eye tracking function, and they apply the eye tracking method 200. Figure 8 The head-mounted display device 300 of FIG. 9 is a glasses type, including a body 301 and an arm portion 302 connected to the body 301. The head-mounted display device 400 of FIG. 9 is a helmet type, including a main body 401 and a headband 402 connected to the main body 401. Figure 10 The head-mounted display device 500 of FIG. 9 is a mask type, including a housing 501.

[0059] The feature pattern 170 can be directly formed on the optical elements in the head-mounted display devices 300, 400, and 500. The emitting element 140 can be positioned adjacent to the optical elements in the head-mounted display devices 300, 400, and 500. The image capturing element 160 can be positioned adjacent to the user's eyeballs. Taking the head-mounted display device 300 as an example, the emitting element 140 can be positioned on the body 301 of the head-mounted display device 300, and the image capturing element 160 can be positioned on the arm 302 of the head-mounted display device 300. However, the positions of the emitting element 140 and the image capturing element 160 are not limited to... Figures 8 to 10 The embodiment shown. As long as the light signal provided by the emitting element 140 can enter the optical element including the feature pattern 170, such that the feature pattern 170 can be mapped onto the eyeball, and the image capturing element 160 can capture the image of the eyeball, it falls within the scope of the present invention.

[0060] As mentioned above, based on this invention, it is not necessary to generate feature patterns by directly irradiating the eyeball with multiple emitting elements. Feature patterns comprising multiple patterns can be generated with relatively few emitting elements, significantly reducing the number of emitting elements required in the eye-tracking device and eye-tracking method, thus lowering costs. Furthermore, it eliminates the need for reflective elements to reflect the feature patterns on the eyeball, enabling miniaturization. Moreover, the number, shape, area, and arrangement of the feature patterns can be determined according to actual needs. Additionally, different feature patterns can be tested at low cost and in a short time during the test and development phase. Furthermore, the eye-tracking device and eye-tracking method of this invention can be applied to various fields, including but not limited to head-mounted display devices.

[0061] The foregoing overview of several embodiments enables those skilled in the art to better understand various aspects of the invention. It should be understood by those skilled in the art that the invention can be readily used as the basis for designing or modifying other manufacturing processes and structures to achieve the same objectives or advantages as the embodiments described herein. It should be understood by those skilled in the art that such equivalent configurations do not depart from the spirit and scope of the invention, and that various changes, substitutions, and modifications can be made to the invention without departing from its spirit and scope. Furthermore, features from the various embodiments can be freely combined and used as long as they do not violate or conflict with the spirit of the invention.

Claims

1. An eye tracking device, comprising: an optical element corresponding to an eye, the optical element comprising a feature pattern, the feature pattern being a hole provided on a front surface of the optical element; an emitting element disposed adjacent to the optical element, the emitting element providing a light signal to the optical element so that the feature pattern is mapped to the eye; and an image capturing element disposed adjacent to the eye, the image capturing element capturing an image of the eye, wherein the optical element further comprises a coating layer coated on a surface of the optical element, wherein the coating layer has a transmittance of visible light between 90% and 100%, and the coating layer has a transmittance of the light signal provided by the emitting element between 0% and 10%, wherein the eye tracking device does not need to generate a feature pattern by a plurality of emitting elements directly illuminating the eye, and does not need to dispose a reflecting element to reflect the feature pattern on the eye.

2. The eye tracking device of claim 1, further comprising a front housing and a rear housing connected to the front housing.

3. The eye tracking device of claim 1, wherein the optical element comprises a visual zone and a peripheral zone, the visual zone being closer to the eye than the peripheral zone, and the feature pattern is formed in the peripheral zone.

4. The eye tracking device of claim 1, wherein the light signal provided by the emitting element is invisible light.

5. The eye tracking device of claim 1, wherein the light signal provided by the emitting element is infrared light.

6. The eye tracking device of claim 1, wherein the feature pattern comprises a plurality of patterns.

7. The eye tracking device of claim 1, wherein the feature pattern is formed in the optical element by superfinishing.

8. An eye tracking method using the eye tracking device of any one of claims 1-7, comprising: generating a light signal by an emitting element, the light signal entering an optical element comprising a feature pattern so that the feature pattern is mapped to an eye; capturing an image of the eye; and identifying a position of the eye based on the feature pattern on the eye. ​

Images