Eye tracking using light directing mechanism

Abstract

In some aspects, an eye tracking device can use an infrared illumination device external to a lens module to produce infrared light used to illuminate a user's eye for eye tracking. The eye tracking device can use a substrate with a light directing mechanism to direct infrared light reflected from the user's eye toward an eye tracking camera within the lens module. The eye tracking device can perform eye tracking on the user's eye using the eye tracking camera. Numerous other aspects are provided.

Description

[0001] Cross-reference to related applications

[0002] This patent application claims priority to U.S. nonprovisional patent application No. 17 / 109,963, filed December 2, 2020, entitled “EYE TRACKING USING A LIGHTDIRECTING MECHANISM,” which is expressly incorporated herein by reference. Technical Field

[0003] All aspects of this disclosure relate to eye tracking in general, and to eye tracking using, for example, a light-guiding mechanism. Background Technology

[0004] A head-mounted display can provide information to a user wearing the display. The head-mounted display may include cameras for tracking the user's eyes. Eye tracking may include measuring the gaze direction of the eyes (e.g., the direction the eyes are looking) or the movement of the eyes relative to the user's head. Eye tracking may include measuring the position and movement of the eyes. The cameras in the head-mounted display may track the user's eyes based at least in part on images captured by the cameras. In other words, when performing eye tracking, eye position and / or eye movement may be detected at least in part based on images captured by the cameras. Summary of the Invention

[0005] In some aspects, an eye-tracking device includes: a lens module including one or more lens elements; an infrared illumination device external to the lens module and configured to generate infrared light capable of illuminating a user's eye for eye tracking; an eye-tracking camera within the lens module; and a substrate having a light guiding mechanism configured to guide infrared light reflected from the user's eye toward the eye-tracking camera.

[0006] In some aspects, an eye-tracking device includes: a lens module comprising one or more lens elements; an infrared illumination device external to the lens module and configured to generate infrared light for illuminating a user's eye for eye tracking; an eye-tracking camera internal to the lens module; and a light guiding mechanism located on the surface of one of the lens elements and configured to guide infrared light reflected from the user's eye toward the eye-tracking camera.

[0007] In some aspects, one method includes: using an infrared illumination device external to the lens module to generate infrared light to illuminate a user's eye for eye tracking; using a substrate with a light guiding mechanism to guide the infrared light reflected from the user's eye toward an eye-tracking camera within the lens module; and using the eye-tracking camera to perform eye tracking on the user's eye.

[0008] In some aspects, an apparatus includes: components for generating infrared light to illuminate a user's eye for eye tracking using an infrared illumination device external to a lens module; components for guiding the infrared light reflected from the user's eye toward an eye-tracking camera within the lens module using a substrate having a light guiding mechanism; and components for performing eye tracking on the user's eye using the eye-tracking camera.

[0009] The entirety of the categories includes a method, apparatus, system, computer program product, non-transitory computer-readable medium, user device, user equipment, wireless communication equipment and / or processing system, as described generally with reference to the accompanying drawings and description and as shown in the accompanying drawings and description.

[0010] The foregoing has provided a fairly broad overview of the features and technical advantages of the examples according to this disclosure in order to better understand the following detailed description. Additional features and advantages will be described below. The disclosed concepts and specific examples can be readily used as the basis for modifying or designing other structures for achieving the same purpose as this disclosure. Such equivalent constructions do not depart from the scope of the appended claims. The features, organization, and operation of the concepts disclosed herein, as well as their associated advantages, can be better understood from the following description when considered in conjunction with the accompanying drawings. Each drawing is provided for illustrative and descriptive purposes and not as a definition of limitation of the claims. Attached Figure Description

[0011] To gain a more detailed understanding of the foregoing features of this disclosure, reference can be made to various aspects, some of which are illustrated in the accompanying drawings. However, it should be noted that the drawings illustrate only certain typical aspects of this disclosure and should therefore not be considered as limiting its scope, as the description may allow for other equivalent aspects. The same reference numerals in different drawings may identify the same or similar elements.

[0012] Figure 1 This is a diagram illustrating an exemplary environment according to various aspects of this disclosure, in which eye tracking using the light-guiding mechanism described herein can be implemented.

[0013] Figure 2 This illustrates various aspects according to this disclosure. Figure 1 A diagram of exemplary components of one or more devices (such as eye-tracking devices) shown.

[0014] Figures 3 to 4 This is a diagram illustrating examples of eye tracking associated with various aspects of this disclosure.

[0015] Figures 5 to 12 This is a diagram illustrating an example of eye tracking using a light-guided mechanism according to various aspects of this disclosure.

[0016] Figure 13 This is a flowchart of an exemplary process associated with eye tracking using a light-guiding mechanism, according to various aspects of this disclosure. Detailed Implementation

[0017] Various aspects of this disclosure are described more fully below with reference to the accompanying drawings. However, this disclosure may be embodied in many different forms and should not be construed as limited to any particular structure or function presented throughout this disclosure. Rather, these aspects are provided so that this disclosure will be thorough and complete and will fully convey the scope of this disclosure to those skilled in the art. Based on the teachings herein, those skilled in the art will understand that the scope of this disclosure is intended to cover any aspect of this disclosure disclosed herein, whether implemented independently of or in combination with any other aspect of this disclosure. For example, any number of aspects set forth herein may be used to implement an apparatus or practice. Furthermore, the scope of this disclosure is intended to cover an apparatus or method that is practiced using a structure, function, or structure and function other than or different from the aspects of this disclosure set forth herein. It should be understood that any aspect of this disclosure disclosed herein may be implemented by one or more elements of the claims.

[0018] Eye-tracking devices may include infrared illumination devices to illuminate a user's eyes with infrared light for eye tracking. Eye-tracking devices may include infrared reflectors to reflect the infrared light from the user back to the eye-tracking camera of the eye-tracking device. Infrared reflectors allow the eye-tracking camera to be placed outside the user's field of view. While infrared reflectors consume relatively much space, they can be accommodated because the lens module including the infrared reflector can also be relatively large. However, in head-mounted displays such as augmented reality (AR) or virtual reality (VR) glasses, relatively large lens modules can be bulky. While smaller lens modules (e.g., pancake lens modules) are available, such modules do not have sufficient space to accommodate infrared reflectors, and therefore, the eye-tracking camera may be undesirably placed within the user's field of view. Alternatively, the eye-tracking camera may be placed at a relatively large angle relative to the user, which can cause problems (e.g., the user's eyelashes may obstruct the camera's view of the eyes), potentially reducing eye-tracking performance.

[0019] In various aspects of the techniques and apparatus described herein, an eye-tracking device may include a lens module having one or more lens elements. The lens module may be a flat lens module. The lens module may include an eye-tracking camera positioned outside the user's field of view. The lens module may include a substrate and a light-directing mechanism associated with the substrate. The light-directing mechanism may direct infrared light reflected from the user's eye toward the eye-tracking camera. The light-directing mechanism may be a mirror coupled to the substrate, a film coupled to the substrate, an angled slit extending at least partially into the substrate, a facet associated with the substrate, a facet patch associated with the substrate, a trough extending into the substrate, or one or more grooves extending into the substrate. The eye-tracking camera may generate an image of the user's eye based at least in part on the infrared light directed toward the eye-tracking camera by the light-directing mechanism. By employing a substrate with a light-directing mechanism, the eye-tracking camera can be positioned within the lens module and outside the user's field of view. Therefore, certain situations (e.g., the user's eyelashes obscuring the camera view of the eye) can be avoided, thereby improving eye-tracking performance.

[0020] Figure 1 This is a diagram of an exemplary environment 100 in which a system and / or method for eye tracking using the light guiding mechanism described herein can be implemented. Figure 1 As shown, environment 100 may include user equipment 110, network 120, and server equipment 130. The devices in environment 100 may be interconnected via wired connections, wireless connections, or a combination of wired and wireless connections.

[0021] User equipment 110 may include one or more devices capable of receiving, generating, storing, processing, and / or providing information (such as the information described herein). For example, user equipment 110 may include a head-mounted display, a computing device (e.g., a desktop computer, laptop computer, tablet computer, handheld computer, server, etc.), a mobile phone (e.g., a smartphone, cordless phone, etc.), or a similar device employing the eye-tracking technology described herein. In some embodiments, user equipment 110 may receive information from and / or send information to server device 130.

[0022] Network 120 may include one or more wired and / or wireless networks. For example, network 120 may include cellular networks, public land mobile networks (PLMNs), local area networks (LANs), wide area networks (WANs), metropolitan area networks (MANs), telephone networks (e.g., public switched telephone networks (PSTNs)), ad hoc networks, intranets, the Internet, fiber-optic networks, and / or combinations of these or other types of networks.

[0023] Server device 130 may include one or more devices capable of receiving, generating, storing, processing, and / or providing information (such as the information described herein). For example, server device 130 may include computing devices such as servers, desktop computers, laptop computers, tablet computers, handheld computers, or similar devices.

[0024] Figure 1 The number of devices and networks shown is provided as an example. In reality, with... Figure 1 Compared to those shown, there may be additional equipment and / or networks, fewer equipment and / or networks, different equipment and / or networks, or equipment and / or networks with different arrangements. Furthermore, Figure 1 The two or more devices shown can be implemented within a single device, or Figure 1 The single device shown can be implemented as multiple distributed devices. Furthermore, one or more devices in environment 100 can perform one or more functions described as being performed by another one or more devices in environment 100.

[0025] Figure 2 This is a diagram illustrating exemplary components of device 200 according to various aspects of this disclosure. Device 200 may correspond to user equipment 110, which may be a head-mounted display. In some aspects, user equipment 110 may include one or more devices 200 and / or one or more components of device 200. Figure 2As shown, device 200 may include bus 205, processor 210, memory 215, storage component 220, input component 225, output component 230, communication interface 235, or eye-tracking device 240. Eye-tracking device 240 may include lens module 245, infrared illumination device 250, eye-tracking camera 255, substrate 260, and light guiding mechanism 265.

[0026] Bus 205 includes components that allow communication between components of device 200. Processor 210 is implemented in hardware, firmware, or a combination of hardware and software. Processor 210 is a central processing unit (CPU), graphics processing unit (GPU), accelerated processing unit (APU), microprocessor, microcontroller, digital signal processor (DSP), field-programmable gate array (FPGA), application-specific integrated circuit (ASIC), or another type of processing component. In some aspects, processor 210 includes one or more processors that can be programmed to perform functions. Memory 215 includes random access memory (RAM), read-only memory (ROM), and / or another type of dynamic or static storage device (e.g., flash memory, magnetic storage, and / or optical storage) that stores information and / or instructions for use by processor 210.

[0027] Storage component 220 stores information and / or software related to the operation and use of device 200. For example, storage component 220 may include hard disk (e.g., magnetic disk, optical disk, magneto-optical disk and / or solid-state disk), optical disk (CD), digital versatile disk (DVD), floppy disk, cartridge, magnetic tape and / or another type of non-transitory computer-readable medium, and corresponding drives.

[0028] Input component 225 includes components that allow device 200 to receive information, such as via user input (e.g., a touchscreen display, keyboard, keypad, mouse, button, switch, and / or microphone). Additionally or alternatively, input component 225 may include components for determining the location or orientation of device 200 (e.g., a Global Positioning System (GPS) component, a Global Navigation Satellite System (GNSS) component, etc.) and sensors for sensing information (e.g., an accelerometer, gyroscope, actuator, another type of position or environmental sensor, etc.). Output component 230 includes components that provide output information from device 200 (e.g., a display, speaker, haptic feedback component, audio or visual indicator, etc.).

[0029] Communication interface 235 includes transceiver-like components (e.g., a transceiver and / or separate receiver and transmitter) that enable device 200 to communicate with other devices, such as via wired connections, wireless connections, or a combination of wired and wireless connections. Communication interface 235 can allow device 200 to receive information from and / or provide information to another device. For example, communication interface 235 may include an Ethernet interface, an optical interface, a coaxial interface, an infrared interface, a radio frequency interface, a universal serial bus (USB) interface, a wireless LAN interface (e.g., a Wi-Fi interface), a cellular network interface, etc.

[0030] Lens module 245 may include one or more lens elements. Lens module 245 may be a flat lens module. Infrared illumination device 250 may be external to lens module 245 and may be configured to generate infrared light capable of illuminating a user's eye for eye tracking. Eye tracking camera 255 may be internal to lens module 245.

[0031] The substrate 260 may be associated with a light guiding mechanism 265, which may be configured to guide infrared light reflected from the user's eye toward an eye-tracking camera 255. The substrate 260 may be transparent or translucent to both infrared and visible light. The substrate 260 may be a glass substrate or a plastic substrate.

[0032] The light guiding mechanism 265 may be a mirror coupled to the substrate 260, a film coupled to the substrate 260, or an angled slit that extends at least partially into the substrate 260. The film coupled to the substrate 260 may be a diffraction film or a holographic film.

[0033] The light guiding mechanism 265 may be a facet associated with the substrate 260, a facet patch associated with the substrate 260, a groove extending into the substrate 260, or one or more recesses extending into the substrate 260. One or more recesses extending into the substrate 260 may correspond to a series of parallel recesses along an axis. The facet patch associated with the substrate 260 may include a first facet patch corresponding to a first axis and a second facet patch corresponding to a second axis.

[0034] The eye-tracking camera 255 can generate a first image of at least a portion of the user's eye, based at least in part on the detection of infrared light at a first angle. The eye-tracking camera 255 can generate a second image of at least a portion of the user's eye, based at least in part on the detection of infrared light at a second angle, wherein the first and second images are along an axis. The center between the first and second images can be related to the user's gaze angle along the axis. This axis can be a horizontal or vertical axis.

[0035] The eye-tracking camera 255 can be a first eye-tracking camera, and the light-guiding mechanism 265 can be a first light-guiding mechanism. A second eye-tracking camera and a second light-guiding mechanism may be included, wherein the first light-guiding mechanism may be perpendicular to the second light-guiding mechanism. The first eye-tracking camera and the first light-guiding mechanism can perform eye tracking along a first axis, and the second eye-tracking camera and the second light-guiding mechanism can perform eye tracking along a second axis.

[0036] Device 200 can perform one or more of the processes described herein. Device 200 can perform these processes based on software instructions stored on a non-transitory computer-readable medium such as memory 215 and / or storage component 220, executed by processor 210. Computer-readable medium is defined herein as a non-transitory memory device. A memory device includes memory space within a single physical storage device or memory space distributed across multiple physical storage devices.

[0037] Software instructions may be read from another computer-readable medium or from another device via communication interface 235 into memory 215 and / or storage component 220. When executed, the software instructions stored in memory 215 and / or storage component 220 may cause processor 210 to perform one or more of the processes described herein. Additionally or alternatively, hard-wired circuitry may be used in place of or in conjunction with software instructions to perform one or more of the processes described herein. Therefore, the aspects described herein are not limited to any particular combination of hardware circuitry and software.

[0038] In some aspects, device 200 includes components for performing one or more of the processes described herein and / or components for performing one or more operations of the processes described herein. For example, device 200 may include: components for generating infrared light to illuminate a user's eye for eye tracking using an infrared illumination device external to the lens module; components for guiding the infrared light reflected from the user's eye toward an eye-tracking camera within the lens module using a substrate having a light-guiding mechanism; and / or components for performing eye tracking on the user's eye using the eye-tracking camera. In some aspects, such components may include combinations of... Figure 2 The described device 200 includes one or more components such as bus 205, processor 210, memory 215, storage component 220, input component 225, output component 230, communication interface 235, and / or eye-tracking device 240.

[0039] Figure 2 The number and arrangement of components shown are provided as an example. In practice, device 200 may include components with... Figure 2The components shown are compared to additional components, fewer components, different components, or components with different arrangements. Additionally or alternatively, a collection of components of device 200 (e.g., one or more components) may perform one or more functions described as being performed by a collection of other components of device 200.

[0040] Eye-tracking solutions can be used to track the position or movement of a user's eyes. Eye tracking can be employed in head-mounted displays such as AR or VR glasses. For example, a head-mounted display may include a camera attached to the frame of the display, and the camera may be pointed inwards towards the user's eyes. Infrared light-emitting diodes (LEDs) may be mounted in the frame of the head-mounted display to illuminate the user's eyes for eye tracking.

[0041] Figure 3 This is a diagram illustrating an example 300 of eye tracking. An eye-tracking device may include one or more lens elements associated with a lens, an infrared LED, an infrared mirror, an eye-tracking camera, and a display. The infrared LED can illuminate a user's eyes with infrared light. The infrared mirror can be used to enable the eye-tracking camera to be placed outside the user's field of view. The infrared mirror can guide infrared light reflected from the user's eyes to the eye-tracking camera while allowing visible light emitted from the display to pass through. In other words, visible light can be unaffected by the infrared mirror. The eye-tracking camera can capture an image of the user's eyes based at least in part on the infrared light reflected from the user's eyes, and this image can be used to perform eye tracking on the user's eyes.

[0042] In an eye-tracking device, the eye-tracking camera can be located within a lens and therefore outside the user's field of view. The eye-tracking camera can be positioned within the lens and outside the user's field of view, at least partially based on an infrared reflector within the lens. The infrared reflector can guide or deflect light approximately ninety degrees away from the user's field of view and toward the eye-tracking camera.

[0043] As pointed out above, Figure 3 This is provided as an example. Other examples may be related to... Figure 3 The descriptions are different.

[0044] Figure 4 This is a diagram illustrating an example 400 of eye tracking. An eye tracking device may include a flat lens having one or more flat lens elements, an eye tracking camera, and a display. The eye tracking camera may be placed outside the flat lens. The eye tracking camera can capture an image of a user's eyes, and this image can be used to perform eye tracking on the user's eyes.

[0045] like Figure 4As shown, a flat lens may include one or more flat lens elements with partially reflective surfaces, which can cause light to be reflected several times before leaving the flat lens. For example, the light path in a flat lens may include: light being reflected from the surface of a first flat lens element, reflected from the surface of a second flat lens unit, and then propagating to the outside of the flat lens.

[0046] Flat lenses offer various advantages over non-flat lenses. For example, they can be smaller. While they may be less efficient, achieving approximately 10% light transmittance, which is lower than that associated with non-flat lenses, they do occupy less space. This reduction in space can be advantageous for head-mounted displays, while allowing sufficient space for infrared reflectors.

[0047] like Figure 4 As shown, in an eye-tracking device, the eye-tracking camera can be placed outside the flat lens. The eye-tracking camera can be positioned outside the user's field of view to avoid obstructing it. Therefore, the eye-tracking camera may be pointed at the user's eye at a large angle (α), such as greater than 40 degrees. At this large angle, certain user features may obstruct the eye-tracking camera's view of the eye. For example, the user's eyelashes may obstruct the eye-tracking camera's view of the eye, resulting in the inability to perform eye tracking and / or reduced accuracy when performing eye tracking.

[0048] As pointed out above, Figure 4 This is provided as an example. Other examples may be related to... Figure 4 The descriptions are different.

[0049] Figure 5 This is a diagram illustrating example 500 of eye tracking using a light-guiding mechanism according to various aspects of this disclosure.

[0050] like Figure 5 As shown, an eye-tracking device (e.g., eye-tracking device 240) may include a lens module (e.g., lens module 245), an infrared illumination device (e.g., infrared illumination device 250), an eye-tracking camera (e.g., eye-tracking camera 255), a substrate (e.g., substrate 260), a light guiding mechanism (e.g., light guiding mechanism 265), and a display. The eye-tracking device may be included in a user device such as a head-mounted display (e.g., user device 110) for tracking a user's eyes using the user device.

[0051] In some aspects, a lens module may include one or more lenses or one or more lens elements. In some aspects, a lens module may be a flat lens module having one or more flat lens elements. As described herein, "flat lens" may refer to a relatively flat, thin lens or a series of lenses used in eye-tracking devices.

[0052] In some aspects, the infrared illumination device may be external to the lens module. The infrared illumination device may be an infrared LED. The infrared illumination device may generate infrared light capable of illuminating the user's eyes for eye tracking. The eye-tracking camera may be within the lens module. The substrate may include or be associated with a light guiding mechanism. The light guiding mechanism may guide or direct the infrared light reflected from the user's eyes toward the eye-tracking camera. The eye-tracking camera may be located above the substrate such that it can capture the infrared light from the light guiding mechanism. The eye-tracking camera may capture an image based at least in part on the infrared light reflected from the user's eyes, and this image may be used to perform eye tracking on the user's eyes.

[0053] In some aspects, the substrate can be transparent or translucent to both infrared and visible light. The substrate can be a glass substrate or a plastic substrate. The substrate can be positioned below the eye-tracking camera.

[0054] For example, the substrate may have a width of approximately three millimeters. The distance between the user's eye and the front surface of the substrate may be approximately ten millimeters. The distance between the rear surface of the substrate and the display may be approximately 18 millimeters. The distance between the rear surface of the substrate and the display may include one or more lens elements. The height of the substrate and one or more lens elements within the lens module may be approximately 40 millimeters.

[0055] In some aspects, the light guiding mechanism may be a mirror coupled to a substrate, which guides infrared light reflected from the user's eye toward an eye-tracking camera. In some aspects, the light guiding mechanism may be a film coupled to a substrate, which guides infrared light reflected from the user's eye toward an eye-tracking camera. This film may be a diffractive film or a holographic film. In some aspects, the light guiding mechanism may be an angled slit that extends at least partially into the substrate, and the angled slit guides infrared light reflected from the user's eye toward an eye-tracking camera.

[0056] In some aspects, the light guiding mechanism may be a facet associated with a substrate, and the facet may guide infrared light reflected from a user's eye toward an eye-tracking camera. In some aspects, the facet may be formed as a film laminated to the substrate. In some aspects, the light guiding mechanism may be a facet patch associated with a substrate, and the facet patch may guide infrared light reflected from a user's eye toward an eye-tracking camera. In some aspects, the light guiding mechanism may be a groove extending into the substrate, and the groove may guide infrared light reflected from a user's eye toward an eye-tracking camera.

[0057] In some aspects, the light guiding mechanism may be one or more grooves extending into the substrate, and the one or more grooves may guide infrared light reflected from the user's eye toward an eye-tracking camera. The one or more grooves may correspond to a series of parallel grooves along an axis (e.g., a horizontal axis). In some aspects, the one or more grooves may be machined or printed into the surface of the substrate, or the one or more grooves may be embossed into a film laminated to the surface of the substrate.

[0058] like Figure 5 As further shown, the light guiding mechanism can be a series of grooves, each approximately 30 micrometers (μm) deep. Infrared light reflected from the user's eye can strike the grooves, and the grooves can guide or direct the infrared light toward the eye-tracking camera.

[0059] In some aspects, the groove depth can vary between 10 micrometers and 500 micrometers. The angle of the groove apex can be approximately 45 degrees. The groove can extend through the surface of the substrate. The groove can be centered or not centered on the substrate. The groove can be offset in a direction perpendicular to its length.

[0060] As pointed out above, Figure 5 This is provided as an example. Other examples may be related to... Figure 5 The descriptions are different.

[0061] Figure 6 This is a diagram illustrating an example 600 of eye tracking using a light-guiding mechanism according to various aspects of this disclosure.

[0062] In some aspects, infrared illumination devices in user equipment (e.g., head-mounted displays) can generate infrared light capable of illuminating the user's eyes. The infrared light emitted from the infrared illumination device can have a wavelength of approximately 850 nanometers (nm). The infrared illumination device can be external to a lens module. A light guiding mechanism associated with a substrate can guide or direct the infrared light reflected from the user's eyes toward an eye-tracking camera. The eye-tracking camera can capture an image based at least in part on the infrared light reflected from the user's eyes, and this image can be used to perform eye tracking on the user's eyes.

[0063] In some aspects, the eye-tracking camera can generate a first image of at least a portion of the user's eye, based at least in part on the detection of infrared light at a first angle. The eye-tracking camera can generate a second image of at least a portion of the user's eye, based at least in part on the detection of infrared light at a second angle. The first and second images can be aligned along an axis such as a horizontal axis. The center between the first and second images can be related to the user's gaze angle along the axis.

[0064] like Figure 6 As shown, the user's eye can be captured at three different angles. At the first angle, infrared light can be guided from a light-guiding mechanism (e.g., a series of horizontal slots) and captured by an eye-tracking camera. Due to total internal reflection of the infrared light, it may strike the eye-tracking camera's sensor at two angles, potentially resulting in two images at the eye-tracking camera. These two images can correspond to the user's eye and can correspond to the first angle. These two images can correspond to an axis such as the horizontal axis. Similarly, the eye-tracking camera can capture two images of the user's eye at each of the second and third angles. In this example, the first angle could correspond to the user's eye looking to the left, the second angle could correspond to the user's eye looking straight ahead, and the third angle could correspond to the user's eye looking to the right.

[0065] In some respects, infrared light can be reflected or scattered from the eye and strike a light guiding mechanism (e.g., an angled groove or facet on a substrate). The reflected or scattered infrared light can undergo total internal reflection at the light guiding mechanism (or air interface). The infrared light can be trapped in the substrate and propagated toward an eye-tracking camera, which may be located at the edge of the substrate. The infrared light can strike the light guiding mechanism (e.g., an angled groove or facet on the substrate) at a range of incident angles that can be maintained as the infrared light propagates toward the eye-tracking camera in the light guide.

[0066] In some respects, infrared light can be reflected from the top and bottom surfaces of the light guide. Therefore, when the eye-tracking camera performs imaging at least partially based on infrared light, it can capture two separate images along the horizontal axis. The centers of the two separate images can be associated with the same x-position on the eye-tracking camera's sensor, where the x-position can be directly related to the eye's gaze angle along the horizontal axis.

[0067] In some respects, depending on the light-guiding mechanism (e.g., a series of vertical slots), the eye-tracking camera can capture two separate images along a vertical axis. The centers of the two separate images can be associated with the same y-position on the eye-tracking camera's sensor, where the y-position can be directly related to the eye's gaze angle along the vertical axis.

[0068] As pointed out above, Figure 6 This is provided as an example. Other examples may be related to... Figure 6 The descriptions are different.

[0069] Figure 7 This is a diagram illustrating an example 700 of eye tracking using a light-guiding mechanism according to various aspects of this disclosure.

[0070] As shown by reference numerals 702, 704, and 706 in the accompanying drawings, infrared light can be reflected or scattered from the user's eye. The infrared light can strike a light guiding mechanism, such as a series of horizontal grooves on a substrate. The infrared light can propagate towards an eye-tracking camera located at the edge of the substrate. Depending on the angle associated with the user's eye, the infrared light can strike the light guiding mechanism at a small angle of incidence, which can be maintained as the infrared light propagates towards the eye-tracking camera in the light guide.

[0071] As pointed out above, Figure 7 This is provided as an example. Other examples may be related to... Figure 7 The descriptions are different.

[0072] Figure 8 This is a diagram illustrating an example 800 of eye tracking using a light-guiding mechanism according to various aspects of this disclosure.

[0073] As shown by reference numeral 802 in the attached figure, infrared light can be reflected or scattered from the user's eye. The infrared light can strike a first light guiding mechanism, such as a series of horizontal grooves on a substrate. The infrared light can propagate towards a first eye-tracking camera located at a first edge of the substrate. Furthermore, the infrared light can strike a second light guiding mechanism, such as a series of vertical grooves on the substrate. The second light guiding mechanism can be perpendicular to the first light guiding mechanism. The infrared light can propagate towards a second eye-tracking camera located at a second edge of the substrate, wherein the second edge of the substrate can be adjacent to the first edge of the substrate.

[0074] In some aspects, the first light guide mechanism and the first eye-tracking camera can be used to perform eye tracking along a first axis (such as a horizontal axis). The second light guide mechanism and the second eye-tracking camera can be used to perform eye tracking along a second axis (such as a vertical axis). In other words, the first light guide mechanism and the first eye-tracking camera can be used to determine the user's eye gaze angle along the horizontal axis, and the second light guide mechanism and the second eye-tracking camera can be used to determine the user's eye gaze angle along the vertical axis.

[0075] As shown by reference numeral 804 in the attached figure, the first light guiding mechanism and / or the second light guiding mechanism may be offset from the center of the substrate. In other words, the first light guiding mechanism may not be located at the center of the substrate, and / or the second light guiding mechanism may not be located at the center of the substrate.

[0076] In some aspects, two eye-tracking cameras can be used to determine the horizontal and vertical positions of a user's eyes, respectively, where the first eye-tracking camera may be orthogonal to the second eye-tracking camera. In other aspects, a single eye-tracking camera can be used to determine the horizontal and vertical positions of a user's eyes. As mentioned above, a single eye-tracking camera can determine the horizontal position of the user's eyes. A single eye-tracking camera can determine the vertical position of the user's eyes based at least in part on the vertical position of the eyes on the sensor of the single eye-tracking camera.

[0077] As pointed out above, Figure 8 This is provided as an example. Other examples may be related to... Figure 8 The descriptions are different.

[0078] Figure 9 This is a diagram illustrating an example 900 of eye tracking using a light-guiding mechanism according to various aspects of this disclosure.

[0079] like Figure 9 As shown, when a lens module (e.g., a flat lens module) is used and the eye-tracking device is located within the lens module, the angle (Φ) between the infrared light reflected from the user's eye and the infrared light captured at the eye-tracking device can be determined. Furthermore, the eye-tracking device can detect the infrared light via a light-guiding mechanism associated with a substrate within the lens module. The angular resolution can be determined as a function of the angle Φ. The angular resolution can be relatively constant across the eye's field of view, such as over the entire horizontal offset of the eye. In some cases, the angular resolution can be less than 0.35 degrees, which can be less than that of eye-tracking solutions using infrared reflectors, which can be associated with an angular resolution ranging from 0.5 degrees to 1.5 degrees.

[0080] As pointed out above, Figure 9 This is provided as an example. Other examples may be related to... Figure 9 The descriptions are different.

[0081] Figure 10 This is a diagram illustrating an example 1000 of eye tracking using a light-guiding mechanism according to various aspects of this disclosure.

[0082] As shown by reference numeral 1002 in the attached figure, infrared light can be reflected or scattered from the user's eyes. The infrared light can strike a first light guiding mechanism, such as a first facet patch on a substrate corresponding to a horizontal axis. The infrared light can propagate towards a first eye-tracking camera located at a first edge of the substrate. Furthermore, the infrared light can strike a second light guiding mechanism, such as a second facet patch on the substrate corresponding to a vertical axis. The second light guiding mechanism can be perpendicular to the first light guiding mechanism. The infrared light can propagate towards a second eye-tracking camera located at a second edge of the substrate, wherein the second edge of the substrate can be adjacent to the first edge of the substrate. In this example, the first facet patch can be used to determine the user's eye gaze angle along the horizontal axis, and the second facet patch can be used to determine the user's eye gaze angle along the vertical axis. In other words, the first facet patch can be associated with a first eye gaze direction, and the second facet patch can be associated with a second eye gaze direction, which is orthogonal to the first eye gaze direction.

[0083] As shown by reference numeral 1004 in the attached figure, infrared light can be reflected or scattered from the user's eye. The infrared light can strike a light guiding mechanism, such as a facet patch on a substrate. The facet patch can include a plurality of orthogonal facets or grooves. Infrared light striking a facet or groove in a facet patch associated with a horizontal axis can propagate toward a first eye-tracking camera located at a first edge of the substrate. Infrared light striking a facet or groove in a facet patch associated with a vertical axis can propagate toward a second eye-tracking camera located at a second edge of the substrate.

[0084] As pointed out above, Figure 10 This is provided as an example. Other examples may be related to... Figure 10 The descriptions are different.

[0085] Figure 11 This is a diagram illustrating an example 1100 of eye tracking using a light-guiding mechanism according to various aspects of this disclosure.

[0086] As shown by reference numeral 1102 in the attached figure, infrared light can be reflected or scattered from the user's eye. The infrared light can strike a light guiding mechanism, such as an infrared mirror coupled to a substrate. For example, the infrared mirror can be positioned at an angle (e.g., 45 degrees), and can be embedded in or coupled to the substrate. The infrared mirror can be coated with an infrared reflector, which allows visible light to pass through, preventing artifacts since visible light can be transmitted without refraction. The infrared light striking the infrared mirror can propagate towards an eye-tracking camera located at the edge of the substrate.

[0087] As shown by reference numeral 1104 in the attached figure, infrared light can be reflected or scattered from the user's eye. The infrared light can strike a light guiding mechanism, such as an angled slit that extends at least partially into the substrate. The angled slit may not extend across the entire width of the substrate to maintain its mechanical strength. The angled slit may not extend through the entire thickness of the substrate to maintain its mechanical strength. The infrared light can be totally internally reflected from the glass / air interface formed by the angled slit and can propagate towards an eye-tracking camera located at the edge of the substrate.

[0088] As pointed out above, Figure 11 This is provided as an example. Other examples may be related to... Figure 11 The descriptions are different.

[0089] Figure 12 This is a diagram illustrating an example 1200 of eye tracking using a light-guiding mechanism according to various aspects of this disclosure.

[0090] In some aspects, an eye-tracking device may include a lens module, an infrared illumination device, an eye-tracking camera, and a light guiding mechanism. The eye-tracking device may not include a substrate. Instead, the light guiding mechanism may be a film (e.g., a light guiding film) that can be coupled to the lens elements of the lens module. In this example, infrared light reflected from the user's eye can be guided by the film to the eye-tracking camera.

[0091] In some aspects, as indicated by reference numeral 1202, the film may be coupled to the front surface of the first lens element in the lens module. In some aspects, as indicated by reference numeral 1204, the film may be coupled to the rear surface of the first lens element in the lens module.

[0092] In some cases, coupling a film to the surface of a lens element may interfere with the function of the dielectric coating on the lens element surface. However, this interference may be minimal when the film covers a relatively small area. In some cases, the film can be compatible with the dielectric coating, where no additional substrate needs to be added.

[0093] In some cases, the surface of the lens element can be curved. In such cases, a film patch can be used on the entire surface of the lens element instead of a film strip, because the film patch can be more easily attached to the curved surface.

[0094] As pointed out above, Figure 12 This is provided as an example. Other examples may be related to... Figure 12 The descriptions are different.

[0095] Figure 13 This is a flowchart of an exemplary process 1300 associated with eye tracking using a light-guiding mechanism. In some embodiments, Figure 13One or more processing frames can be performed by an eye-tracking device (e.g., eye-tracking device 240). In some embodiments, Figure 13 One or more processing blocks may be performed by another device or a group of devices, such as user equipment 110, that is separate from or includes the eye-tracking device. Additionally or alternatively, Figure 13 One or more processing blocks may be executed by one or more components of device 200, such as processor 210, memory 215, storage component 220, input component 225, output component 230 and / or communication interface 235.

[0096] like Figure 13 As shown, process 1300 may include using an infrared illumination device external to the lens module to generate infrared light to illuminate the user's eyes for eye tracking (box 1310). For example, as described above, the eye tracking device may use an infrared illumination device external to the lens module to generate infrared light to illuminate the user's eyes for eye tracking.

[0097] like Figure 13 As further shown, process 1300 may include guiding infrared light reflected from the user's eye toward an eye-tracking camera within a lens module using a substrate with a light guiding mechanism (box 1320). For example, as described above, an eye-tracking device may use a substrate with a light guiding mechanism to guide infrared light reflected from the user's eye toward an eye-tracking camera within a lens module.

[0098] like Figure 13 As further shown, processing 1300 may include performing eye tracking on a user's eyes using an eye-tracking camera (box 1330). For example, as described above, an eye-tracking device may use an eye-tracking camera to perform eye tracking on a user's eyes.

[0099] Process 1300 may include additional embodiments, such as any single embodiment or any combination of embodiments of one or more other processes described below and / or in conjunction with those described elsewhere herein.

[0100] In the first embodiment, the substrate is transparent or translucent to infrared and visible light, and the substrate is either a glass substrate or a plastic substrate.

[0101] In the second embodiment, either alone or in combination with the first embodiment, the light guiding mechanism is a mirror, a film, a facet, a facet patch, a groove, or one or more recesses.

[0102] In a third embodiment, either alone or in combination with one or more of the first and second embodiments, processing 1300 includes generating a first image of the user's eye using an eye-tracking camera based at least in part on the detection of infrared light at a first angle, and generating a second image of the user's eye based at least in part on the detection of infrared light at a second angle, wherein the first image and the second image are along an axis, and wherein the center between the first image and the second image corresponds to the gaze angle of the user's eye along the axis.

[0103] although Figure 13 An example box for processing 1300 is shown, but in some embodiments, with Figure 13 Compared to the boxes depicted, process 1300 may include additional boxes, fewer boxes, different boxes, or boxes with different arrangements. Additionally or alternatively, two or more boxes in process 1300 may be executed in parallel.

[0104] The foregoing disclosure provides illustrations and descriptions, but is not intended to be exhaustive or to limit these aspects to the precise forms disclosed. Modifications and variations can be made based on the foregoing disclosure, or from various practices.

[0105] As used herein, the term "component" is intended to be interpreted broadly as hardware, firmware, and / or a combination of hardware and software. As used herein, a processor is implemented as a combination of hardware, firmware, and / or hardware and software. It will be apparent that the systems and / or methods described herein can be implemented in various forms of hardware, firmware, and / or combinations of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods is not limiting in these respects. Therefore, while the operation and behavior of systems and / or methods are described herein without reference to specific software code, it should be understood that software and hardware can be designed to implement systems and / or methods, at least in part, based on the descriptions herein.

[0106] As used in this article, depending on the context, a threshold can refer to a value that is greater than the threshold, greater than or equal to the threshold, less than the threshold, less than or equal to the threshold, equal to the threshold, not equal to the threshold, etc.

[0107] Although specific combinations of features are recited in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of aspects. In fact, many of these features can be combined in ways not specifically recited in the claims and / or disclosed in the specification. While each dependent claim listed below may directly depend on only one claim, the disclosure of aspects includes each dependent claim in combination with each other claim in the claim set. As used herein, the phrase “at least one” in the list of items refers to any combination of those items, including a single member. As an example, “at least one of a, b, or c” is intended to cover a, b, c, ab, ac, bc, and abc, as well as any combination having multiple identical elements (e.g., aa, aaa, aab, aac, abb, acc, bb, bbb, bbc, cc, and ccc, or any other ordering of a, b, and c).

[0108] Unless explicitly stated otherwise, no element, action, or instruction used herein should be construed as critical or necessary. Additionally, as used herein, the articles “a” and “an” are intended to include one or more items and may be used interchangeably with “one or more.” Furthermore, as used herein, the article “the” is intended to include one or more items referenced in connection with the article “the” and may be used interchangeably with “one or more.” Furthermore, as used herein, the terms “set” and “group” are intended to include one or more items (e.g., related items, unrelated items, a combination of related and unrelated items) and may be used interchangeably with “one or more.” Where only one item is intended, the phrase “only one” or similar language is used. Additionally, as used herein, the terms “has,” “have,” “having,” etc., are intended to be open-ended terms. Furthermore, unless explicitly stated otherwise, the phrase “based on” is intended to mean “at least partially based on.” Furthermore, as used herein, the term “or” is intended to be inclusive when used in a series of forms and may be used interchangeably with “and / or” unless otherwise expressly stated (e.g., if used in conjunction with “any” or “only one”).

Claims

1. An eye-tracking device, comprising: Lens module, the lens module comprising one or more lens elements; An infrared illumination device, external to the lens module, is configured to generate infrared light capable of illuminating a user's eye for eye tracking. An eye-tracking camera, the eye-tracking camera being located within the lens module; as well as A substrate having a light guiding mechanism configured to guide infrared light reflected or scattered from the user's eye toward the eye-tracking camera via total internal reflection. The eye-tracking camera is configured to: A first image of at least a portion of the user's eye is generated, based at least in part on the detection of the infrared light at a first angle. as well as A second image of at least a portion of the user's eye is generated, based at least in part on the detection of the infrared light at the second angle.

2. The eye-tracking device according to claim 1, wherein the substrate is transparent or translucent to infrared and visible light, and wherein the substrate is a glass substrate or a plastic substrate.

3. The eye-tracking device of claim 1, wherein the light-guiding mechanism configured to guide light is: a mirror coupled to the substrate, a film coupled to the substrate, or an angled slit extending at least partially into the substrate.

4. The eye-tracking device of claim 3, wherein the film coupled to the substrate is a diffractive film or a holographic film.

5. The eye-tracking device of claim 1, wherein the light-guiding mechanism configured to guide light is: a facet associated with the substrate, a facet patch associated with the substrate, a groove extending into the substrate, or one or more grooves extending through the surface of the substrate, wherein the facet patch comprises a collection of orthogonal facets or orthogonal grooves.

6. The eye-tracking device of claim 5, wherein the one or more grooves extending through the surface of the substrate correspond to a series of parallel grooves along an axis.

7. The eye-tracking device of claim 5, wherein the facet patch associated with the substrate comprises a first facet patch corresponding to a first axis and a second facet patch corresponding to a second axis.

8. The eye-tracking device of claim 1, wherein the first image and the second image are along an axis.

9. The eye-tracking device of claim 8, wherein the center between the first image and the second image is related to the gaze angle of the user's eye along the axis.

10. The eye-tracking device of claim 8, wherein the axis is a horizontal axis.

11. The eye-tracking device of claim 8, wherein the axis is a vertical axis.

12. The eye-tracking device according to claim 1, wherein: The eye-tracking camera is a first eye-tracking camera, and the light-guiding mechanism is a first light-guiding mechanism. It also includes a second eye-tracking camera and a second light-guiding mechanism, wherein the first light-guiding mechanism is perpendicular to the second light-guiding mechanism.

13. The eye-tracking device of claim 12, wherein the first eye-tracking camera and the first light-guiding mechanism are configured to perform eye tracking along a first axis, and the second eye-tracking camera and the second light-guiding mechanism are configured to perform eye tracking along a second axis.

14. The eye-tracking device of claim 1, wherein the lens module comprising the one or more lens elements is a flat lens module.

15. The eye-tracking device of claim 1, wherein the eye-tracking device is included in a head-mounted display.

16. An eye-tracking device, comprising: Lens module, the lens module comprising one or more lens elements; An infrared illumination device, external to the lens module, is configured to generate infrared light to illuminate a user's eyes for eye tracking. An eye-tracking camera, the eye-tracking camera being located within the lens module; as well as A light guiding mechanism, located on the surface of one of the one or more lens elements, is configured to guide infrared light reflected or scattered from the user's eye toward the eye-tracking camera via total internal reflection. The eye-tracking camera is configured to: A first image of at least a portion of the user's eye is generated, based at least in part on the detection of the infrared light at a first angle. as well as A second image of at least a portion of the user's eye is generated, based at least in part on the detection of the infrared light at the second angle.

17. The eye-tracking device of claim 16, wherein the light-guiding mechanism configured to guide light is: a mirror, a membrane, a facet, a facet patch, a slot, or one or more recesses, wherein the facet patch comprises a collection of orthogonal facets or orthogonal recesses.

18. The eye-tracking device of claim 17, wherein the one or more grooves comprise a series of parallel grooves along an axis or different axes.

19. The eye-tracking device of claim 17, wherein the facet patch comprises a first facet patch corresponding to a first axis and a second facet patch corresponding to a second axis, wherein the first axis corresponds to a first gaze angle of the user's eye along the first axis, wherein the second axis corresponds to a second gaze angle of the user's eye along the second axis, and wherein the axis is either the first axis or the second axis.

20. The eye-tracking device of claim 16, wherein the first image and the second image are along an axis, and wherein the center between the first image and the second image corresponds to the gaze angle of the user's eye along the axis.

21. The eye-tracking device of claim 20, wherein the axis is a horizontal axis or a vertical axis.

22. The eye-tracking device according to claim 16, wherein: The eye-tracking camera is a first eye-tracking camera, and the light-guiding mechanism is a first light-guiding mechanism. It also includes a second eye-tracking camera and a second light-guiding mechanism, wherein the first light-guiding mechanism is perpendicular to the second light-guiding mechanism.

23. The eye-tracking device of claim 22, wherein the first eye-tracking camera and the first light-guiding mechanism are configured to perform eye tracking along a first axis, and the second eye-tracking camera and the second light-guiding mechanism are configured to perform eye tracking along a second axis.

24. A method performed by an eye-tracking device, comprising: Infrared illumination devices are used outside the lens module to generate infrared light to illuminate the user's eyes for eye tracking. Using a substrate with a light guiding mechanism, infrared light reflected or scattered from the user's eyes is guided toward an eye-tracking camera within the lens module by total internal reflection; The eye-tracking camera is used to generate a first image of at least a portion of the user's eye, based at least in part on the detection of infrared light at a first angle. as well as The eye-tracking camera is used to generate a second image of at least a portion of the user's eye, based at least in part on the detection of infrared light at a second angle.

25. The method of claim 24, wherein the substrate is transparent or translucent to infrared and visible light, and wherein the substrate is a glass substrate or a plastic substrate.

26. The method of claim 24, wherein the light guiding mechanism is: a mirror, a film, a facet, a facet patch, a groove or one or more recesses, wherein the facet patch comprises a collection of orthogonal facets or orthogonal recesses.

27. The method of claim 24, wherein the first image and the second image are along an axis, and wherein the center between the first image and the second image corresponds to the gaze angle of the user's eye along the axis.

28. A device for eye tracking, comprising: A component used to generate infrared light for illuminating a user's eyes for eye tracking, using an infrared illumination device located outside the lens module; A component for using a substrate with a light guiding mechanism to guide infrared light reflected or scattered from the user's eyes toward an eye-tracking camera within the lens module via total internal reflection; as well as Components for generating a first image of at least a portion of the user's eye using an eye-tracking camera, based at least in part on the detection of infrared light at a first angle; as well as A component for generating a second image of at least a portion of the user's eye using an eye-tracking camera, based at least in part on the detection of infrared light at a second angle.

29. The apparatus of claim 28, wherein the light guiding mechanism is: a mirror, a film, a facet, a facet patch, a groove or one or more recesses, wherein the facet patch comprises a collection of orthogonal facets or orthogonal recesses.

30. The apparatus of claim 28, wherein the first image and the second image are along an axis, wherein the center between the first image and the second image corresponds to the gaze angle of the user's eye along the axis.

Images