An optical system and method for eye tracking based on redirecting light from the eye using an optical arrangement associated with an optical guide element.

Abstract

To provide various apparatus and corresponding methods for tracking a gaze direction of a human eye based on imaging of the eye via an optical arrangement associated with a light-guide optical element.SOLUTION: A light-transmitting substrate has at least two major surfaces and is deployed with a first of the at least two major surfaces in facing relation to an eye of a viewer. A light redirecting arrangement is associated with the light-transmitting substrate and deflects light from the eye toward an optical sensor that senses light, such that the light deflection occurs at the light-transmitting substrate and the deflected light that reaches the optical sensor is unguided by the light-transmitting substrate. A processor derives a current gaze direction of the eye by processing signals from the optical sensor.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] Cross - reference to Related Applications This application claims priority from U.S. Provisional Patent Application No. 62 / 953,557, filed on December 25, 2019; U.S. Provisional Patent Application No. 62 / 958,755, filed on January 9, 2020; and U.S. Provisional Patent Application No. 63 / 023,891, filed on May 13, 2020, the disclosures of which are hereby incorporated by reference in their entireties.

[0002] The present invention relates to eye tracking.

Background Art

[0003] In the optical arrangements of near - eye displays (NEDs), head - mounted displays (HMDs), and head - up displays (HUDs), large apertures are required to cover the area where the observer's eyes are located (commonly referred to as the eye movement box or EMB). To implement a compact device, the images projected onto the observer's eyes are generated by a small optical image generator (projector) having a small aperture that is magnified to produce a large aperture.

[0004] An approach to one-dimensional aperture multiplication has been developed based on a slab of parallel surfaces of a transparent material in which an image propagates by internal reflection within it. A portion of the image wavefront is coupled out of the slab by using a partially reflected reflector with an oblique angle, or by using a diffractive optical element on one side of the slab. Such a slab is referred to herein as an optical guide optics (LOE), optical transmission substrate, or optical waveguide. The principle of such aperture multiplication is schematically shown in Figure 1, which shows an optical guide optics 10 having a pair of parallel main external surfaces (planes) 12, 14 for guiding light by internal reflection (preferably total internal reflection, but not required). An image projector 16 ( schematically represented as a rectangle) produces a projected image 18, schematically represented here by an illumination beam 18 containing sample rays 18A and 18B spanning the beam. The projected image 18 is coupled into the optical guide optical element 10 by an optical coupling-in configuration 20, as schematically shown here by a prism 20 (replaced interchangeably referred to as a "wedge"), to generate reflected rays 22 that are trapped by internal reflections within the substrate, and also to generate rays 24. Here, the coupling wedge 20 includes three main surfaces, one of which is located next to (or common to) the inclined edge 26 of the LOE 10 (the edge 26 is at an oblique angle to surfaces 12, 14).

[0005] The coupling-in image 18 propagates along the substrate 10 by repeated internal reflections from surfaces 12 and 14, at an oblique angle (α) relative to the parallel surfaces 12 and 14. surAs schematically shown by the sequence of partially reflective surfaces 28, the light strikes the optical coupling-out configuration 28, and a portion of the image intensity is reflected by the parallel surfaces 12, 14 and coupled out from the substrate as rays 30A and 30B toward the pupil 32 of the observer's eye 34 located at an eye relief (ER) distance 38 from the surface 12 to the EMB 36. To minimize undesirable reflections that may produce ghost images, the partially reflective surfaces 28 are preferably coated to have a low reflectivity for a first range of incident angles, while having a desired partial reflectivity for a second range of incident angles. Here, rays with a small inclination (represented here as angle βref) with respect to the normal of the partially reflective surface 28 are split to produce reflected rays for coupling out, while rays with a high inclination (with respect to the normal) are transmitted with negligible reflection.

[0006] The projected image 18 is a collimated image, meaning that each pixel is represented by a beam of parallel light rays at a corresponding angle, corresponding to light from a scene far away from the observer (a collimated image is said to be "infinitely collimated"). Here, the image is simply represented by a ray corresponding to a single point in the image, typically the centroid of the image, but in reality, it includes a range of angles to each side of this central beam, and those rays are coupled in to the substrate at a corresponding range of angles and likewise coupled out at corresponding angles, thereby creating a field of view corresponding to parts of the image that reach the observer's eye 34 in different directions.

[0007] One optical feature that can be useful in the design of NEDs, HMDs, or HUDs is eye tracking, or sensing the direction in which the observer's eyes are looking relative to the direction of the head (commonly referred to as gaze direction). Various solutions for eye tracking have been proposed. In one set of solutions, the eye is imaged in the EMB via the LOE by coupling in the light reflected from the eye, and as a result, the reflected light propagates back to the image projector through the LOE via internal reflection (i.e., in the opposite direction to the image light from the image projector). These solutions attempt to overcome the fundamental problem that the EMB is located relatively close to the LOE, rather than at infinity like a collimated image from an image projector. In another set of solutions, the EMB is imaged using one or more cameras positioned in front of the eye at an off-axis position on the peripheral part of the machine on which the LOE is mounted, e.g., on an eyeglass frame. However, the proximity between the peripheral part of the machine and the eye makes imaging the eye in the EMB difficult due to the relatively large keystone angle. Clearly, positioning the camera directly in front of the eyes allows for high-quality EMB image capture and processing, but it also obscures the viewer's natural field of vision. [Overview of the project]

[0008] Aspects of the present invention provide an eye tracker and corresponding method for tracking the gaze direction of a human eye based on imaging the eye via an optical guide element, which are particularly suitable for integration as part of a NED, HMD, or HUD.

[0009] Aspects of the present invention provide an eye tracker and corresponding method for tracking the gaze direction of a human eye based on imaging the eye, via an optical redirection optical arrangement associated with an optical guide optical element that redirects light reflected from the eye as uninductive light toward an optical sensor in response to eye illumination, and are particularly suitable for integration as part of a NED, HMD, or HUD.

[0010] An optical system is provided according to the teaching of one embodiment of the present invention. The optical system comprises: a light-transmitting substrate having at least two main surfaces, the first of which is positioned facing the eye of a viewer; an optical sensor positioned to sense light; an optical reversal arrangement associated with the light-transmitting substrate, configured to deflect light from the eye toward the optical sensor so that the deflected light reaching the optical sensor is not induced by the light-transmitting substrate, and the deflection of light by the optical reversal arrangement occurs on the light-transmitting substrate; and at least one processor electrically coupled to the optical sensor and configured to process signals from the optical sensor to derive the current gaze direction of the eye.

[0011] Optionally, the optical system further comprises an illumination arrangement positioned to illuminate the eye with light such that the eye reflects a portion of the light from the illumination arrangement as reflected light, the reflected light corresponding to the light from the eye which is deflected by the light deflection arrangement.

[0012] Optionally, the lighting arrangement includes at least a first light source and a second light source, wherein the first light source is configured to produce light having wavelengths in a given first wavelength range, and the second light source is configured to produce light having wavelengths in a given second wavelength range, and the given first wavelength range and the given second wavelength range are non-overlapping ranges.

[0013] Selectively, light from the eye, deflected by a light direction reversal arrangement, primarily includes light with wavelengths outside the visible light region of the electromagnetic spectrum.

[0014] The light from the eye, which is selectively deflected by the optical direction reversal arrangement, mainly consists of light having wavelengths within the visible light region of the electromagnetic spectrum.

[0015] Optionally, the optical direction reversal configuration transmits light with wavelengths within the visible light region of the electromagnetic spectrum and reflects light with wavelengths outside the visible light region of the electromagnetic spectrum.

[0016] Optionally, the optical direction reversal arrangement includes at least one partially reflective surface located within the light-transmitting substrate.

[0017] Optionally, the two main surfaces of the light-transmitting substrate are parallel to each other, and at least one partially reflective surface is a flat surface at an oblique angle to the two main surfaces.

[0018] Optionally, the light-transmitting substrate is configured to induce light corresponding to an infinitely collimated image by internal reflection between two main surfaces of the light-transmitting substrate, and the optical system further comprises at least one second partially reflective surface located within the light-transmitting substrate for coupling out the light induced by internal reflection between the two main surfaces from the light-transmitting substrate to the viewer's eye.

[0019] Optionally, the second, at least one, partially reflective surface is a flat surface that is at an oblique angle to the two primary surfaces.

[0020] Optionally, at least one partially reflective surface and at least one second partially reflective surface are parallel to each other.

[0021] Optionally, at least one partially reflective surface and at least one second partially reflective surface are nonparallel to each other.

[0022] Optionally, at least one partially reflective surface is arranged in a non-overlapping relationship with respect to a second at least one partially reflective surface.

[0023] Optionally, at least one partially reflective surface is arranged in an overlapping relationship with a second at least one partially reflective surface.

[0024] Optionally, the optical direction reversal configuration includes a diffraction element associated with at least one portion of the main surface of the light-transmitting substrate.

[0025] Optionally, the light direction conversion arrangement includes a selectively reflective surface associated with at least a portion of one of the major surfaces of the light transmissive substrate.

[0026] Optionally, the selectively reflective surface is formed from at least one of a dielectric coating or a dichroic coating applied to at least a portion of the major surface.

[0027] Optionally, the light direction conversion arrangement deflects a first set of light rays from the eye, through an imaging lens, towards an optical sensor so as to form a first image of at least a portion of the eye, and the optical system further includes a second light direction conversion arrangement configured to deflect a second set of light rays from the eye, through the imaging lens, towards the optical sensor so as to form a second image of at least a portion of the eye.

[0028] Optionally, at least one processor is further configured to process signals from the optical sensors corresponding to the first and second images so as to determine the distance between the eye and the first major surface.

[0029] Optionally, the light direction conversion arrangement includes one of at least one partially reflective surface located within a first portion of the light transmissive surface, a diffraction element associated with at least a first portion of one of the major surfaces of the light transmissive substrate, or a selectively reflective surface associated with at least a first portion of one of the major surfaces of the light transmissive substrate, and the second light direction conversion arrangement includes one of at least one partially reflective surface located within a second portion of the light transmissive surface, a diffraction element associated with at least a second portion of one of the major surfaces of the light transmissive substrate, or a selectively reflective surface associated with at least a second portion of one of the major surfaces of the light transmissive substrate.

[0030] Optionally, at least one of the major surfaces of the light transmissive substrate is a curved surface.

[0031] Optionally, the light-transmitting substrate is configured to induce light corresponding to an infinitely collimated image through internal reflection between two main surfaces of the light-transmitting substrate, and the optical system further comprises an optical coupling-out configuration for coupling out the light induced by internal reflection between the two main surfaces from the light-transmitting substrate.

[0032] Optionally, the optical coupling-out configuration includes a diffracting element.

[0033] Optionally, the optical coupling out configuration includes at least one partially reflective surface located within the light-transmitting substrate.

[0034] Optionally, the two main surfaces of the light-transmitting substrate are parallel to each other, and at least one partially reflective surface is a flat surface at an oblique angle to the two main surfaces.

[0035] The optical coupling out configuration is optionally deployed in a non-overlapping relationship with respect to the optical direction reversal configuration.

[0036] An optional optical coupling-out configuration is deployed in an overlapping relationship with the optical direction reversal configuration.

[0037] The light-transmitting substrate is optionally configured to guide light in one dimension.

[0038] The light-transmitting substrate is optionally configured to guide light in two dimensions.

[0039] Optionally, the optical direction reversal configuration is arranged in a first set of parallel planes, and the optical coupling out configuration is arranged in a second set of parallel planes.

[0040] Optionally, the first and second sets of planes are parallel to each other.

[0041] The first and second sets of planes are, by any choice, orthogonal to each other.

[0042] The first and second sets of planes are, by any choice, oblique to each other.

[0043] Optionally, a light-transmitting substrate is integrated as part of the nearsighted display.

[0044] Optionally, a light-transmitting substrate is integrated as part of the head-up display.

[0045] Optionally, an optical sensor is placed between the viewer's eye and the first main surface.

[0046] Selectively, light from the eye, which is deflected by the optical direction reversal arrangement, undergoes at most one reflection within the light-transmitting substrate before reaching the optical sensor.

[0047] Optionally, the optical system further comprises at least one imaging optical element positioned in an optical path from an optical reversal arrangement to an optical sensor to form at least one image of at least a portion of the eye on the optical sensor.

[0048] An optical system according to one embodiment of the teachings of the present invention is also provided. The optical system comprises: a light-transmitting substrate having two mutually parallel main external surfaces, one of which is positioned facing the eye of a viewer; an optical coupling-in configuration for coupling in light corresponding to a collimated image into the light-transmitting substrate so as to propagate through the light-transmitting substrate by internal reflection between the main external surfaces; an optical coupling-out configuration for coupling out light propagating through the light-transmitting substrate by internal reflection from the light-transmitting substrate; an optical sensor positioned to sense light; an optical reversal configuration associated with the light-transmitting substrate, configured such that the deflected light from the eye toward the optical sensor is not induced by the light-transmitting substrate, wherein the deflection of light by the optical reversal configuration occurs in the light-transmitting substrate; and at least one processor electrically coupled to the optical sensor and configured to process signals from the optical sensor to derive the current gaze direction of the eye.

[0049] Optionally, the optical direction reversal configuration includes at least one partially reflective surface located within the light-transmitting surface, wherein the at least one partially reflective surface is a flat surface at an oblique angle to the two main external surfaces.

[0050] Optionally, the optical coupling out configuration includes a second, at least one partially reflective surface, wherein the second, at least one partially reflective surface is a flat surface at an oblique angle to the two main external surfaces.

[0051] Optionally, at least one partially reflective surface and at least one second partially reflective surface are parallel to each other.

[0052] Optionally, at least one partially reflective surface is nonparallel to a second at least one partially reflective surface.

[0053] The optical coupling out configuration is optionally deployed in a non-overlapping relationship with respect to the optical direction reversal configuration.

[0054] An optional optical coupling-out configuration is deployed in an overlapping relationship with the optical direction reversal configuration.

[0055] Selectively, the light corresponding to the collimated image mainly consists of light with wavelengths in the visible light region of the electromagnetic spectrum, while the light from the eye, deflected by the optical refraction arrangement, mainly consists of light with wavelengths outside the visible light region of the electromagnetic spectrum.

[0056] Optionally, the optical system further comprises at least one imaging optical element positioned in an optical path from an optical reversal arrangement to an optical sensor to form at least one image of at least a portion of the eye on the optical sensor.

[0057] An optical system according to one embodiment of the teachings of the present invention is also provided. The optical system is a light-transmitting substrate having two mutually parallel main external surfaces, one of which is positioned facing the viewer's eye; an optical coupling-in configuration for coupling in light corresponding to a collimated image into the light-transmitting substrate so that it propagates within the light-transmitting substrate by internal reflection between the main external surfaces; and at least one partially reflective surface located within the light-transmitting substrate for coupling out light propagating within the light-transmitting substrate by internal reflection, the at least one partially reflective surface being relatively parallel to the two main surfaces. The optical reversal arrangement comprises: at least one partially reflective surface which is a flat surface at an oblique angle to a surface; an optical sensor positioned to sense light; and a second partially reflective surface located within a light-transmitting substrate configured to deflect light from the eye toward the optical sensor so that the deflected light reaching the optical sensor is not induced by the light-transmitting substrate, wherein the deflection of light rays by the optical reversal arrangement occurs in the light-transmitting substrate; and at least one processor electrically coupled to the optical sensor and configured to process signals from the optical sensor to derive the current gaze direction of the eye.

[0058] Optionally, at least one second partially reflective surface transmits light having wavelengths within the visible light region of the electromagnetic spectrum and reflects light having wavelengths outside the visible light region of the electromagnetic spectrum.

[0059] Selectively, the light corresponding to the collimated image mainly consists of light with wavelengths in the visible light region of the electromagnetic spectrum, while the light from the eye, deflected by the optical refraction arrangement, mainly consists of light with wavelengths outside the visible light region of the electromagnetic spectrum.

[0060] Optionally, at least one first partially reflective surface and at least one second partially reflective surface are parallel to each other.

[0061] Optionally, at least one first partially reflective surface is nonparallel to at least one second partially reflective surface.

[0062] Optionally, at least one first partially reflective surface is arranged in a non-overlapping relationship with respect to at least one second partially reflective surface.

[0063] Optionally, at least one first partially reflective surface is arranged in an overlapping relationship with at least one second partially reflective surface.

[0064] Optionally, the optical system further comprises at least one imaging optical element positioned in an optical path from an optical reversal arrangement to an optical sensor to form at least one image of at least a portion of the eye on the optical sensor.

[0065] An optical system according to one embodiment of the teachings of the present invention is also provided. The optical system comprises an optical sensor arranged to sense light, and an optical refraction arrangement associated with a light-transmitting substrate having at least two main surfaces, the first of which is arranged facing the eye of a viewer, and the optical refraction arrangement is configured to deflect light from the eye toward the optical sensor, so that the deflected light reaching the optical sensor is not induced by the light-transmitting substrate, and the deflection of light by the optical refraction arrangement occurs on the light-transmitting substrate, and at least one processor electrically coupled to the optical sensor and configured to process signals from the optical sensor to derive the current gaze direction of the eye.

[0066] An optical system according to one embodiment of the teachings of the present invention is also provided. The optical system comprises: a light-transmitting substrate having at least two main surfaces, the first of which is positioned opposite the viewer's eye, and the position of the eye relative to the light-transmitting substrate defines an eye movement box; an illumination arrangement positioned to illuminate the eye movement box with eye-tracking light, such that a certain ratio of the intensity of the eye-tracking light is reflected by the eye as reflected light; an optical sensor positioned to sense light; an optical refraction arrangement associated with the light-transmitting substrate configured to deflect reflected light toward the optical sensor, wherein the deflected light reaching the optical sensor is not induced by the light-transmitting substrate, and the deflection of light by the optical refraction arrangement occurs on the light-transmitting substrate; and at least one processor electrically coupled to the optical sensor and configured to process signals from the optical sensor to derive the current gaze direction of the eye.

[0067] An optical system according to one embodiment of the teachings of the present invention is also provided. The optical system comprises: an illumination arrangement positioned to illuminate an eye movement box with eye-tracking light, wherein the eye movement box defines the position of a viewer's eye relative to a light-transmitting substrate having at least two main surfaces, the first of which is positioned facing the eye, and the intensity of a certain ratio of the eye-tracking light is reflected by the eye as reflected light; an optical sensor positioned to sense light; an optical refraction arrangement associated with the light-transmitting substrate and configured to deflect reflected light toward the optical sensor, wherein the deflected light reaching the optical sensor is not induced by the light-transmitting substrate, and the deflection of light by the optical refraction arrangement occurs in the light-transmitting substrate; and at least one processor electrically coupled to the optical sensor and configured to process signals from the optical sensor to derive the current gaze direction of the eye.

[0068] In the context of this document, the term “induced” generally refers to light trapped within a light-transmitting material (e.g., a substrate) by internal reflection at the main outer surface of the light-transmitting material, so that the trapped light propagates through the light-transmitting material in the direction of propagation. When propagating light is incident on the main outer surface of the light-transmitting material at an angle of incidence within a given angular range, the light propagating through the light-transmitting substrate is trapped by internal reflection. The internal reflection of trapped light can take the form of total internal reflection, thereby causing propagating light incident on the main outer surface of the light-transmitting material at an angle greater than a critical angle (partially defined by the refractive index of the light-transmitting material and the medium in which the transmitted light is deployed, e.g., air) to undergo internal reflection at the main outer surface. Alternatively, the internal reflection of trapped light can be achieved by a coating, such as an angle-selective reflective coating, applied to the main outer surface of the light-transmitting material to achieve reflection of light incident on the main outer surface within a given angular range. Light induced through a light-transmitting material undergoes at least two reflections from the main outer surface of the light-transmitting material.

[0069] Where used in the context of this disclosure, the term “uninducible” generally refers to light that is not induced. Uninducible light traverses a light-transmitting material (e.g., a substrate) without being trapped within the light-transmitting material, i.e., without being trapped between the outer main surfaces of the light-transmitting material by internal reflection. The optical redirection arrangements of this disclosure deflect light from the EMB / viewer’s eye to an imaging / detection subsystem, particularly an optical sensor, so that the light is deflected in the light-transmitting material without being trapped within it, so that it propagates to the optical sensor under free-space propagation. In other words, light from the EMB / viewer’s eye is deflected by the optical redirection arrangement so that the light reaching the optical sensor is not induced by the light-transmitting material.

[0070] Eye trackers according to various embodiments of the present invention rely on the deflection of light reflected from an optical sensor by an optical direction reversal arrangement. The light reflected from the eye is also referred to herein as eye-tracking light and is referred herein as being within the “eye-tracking spectrum”. Eye trackers according to various embodiments of the present invention are particularly effective when the eye-tracking light is in the near-infrared (NIR) region of the electromagnetic spectrum (i.e., when the eye-tracking spectrum is within the NIR region). However, when the eye-tracking light is in the visible region of the electromagnetic spectrum (i.e., when the eye-tracking spectrum is within the visible light region), the eye tracker may also be effective. In addition, embodiments of the present invention are also contemplated in which the eye-tracking spectrum is outside both the visible light and NIR regions, as will be considered in more detail.

[0071] In the context of this document, light in the NIR region of the electromagnetic spectrum generally refers to light with wavelengths in the range of 700–1400 nanometers (nm), and in specific examples, 680–1400 nm. Wavelengths near 700 nm, i.e., in the range of 680–750 nm, may erode darker red visible light, but can be particularly advantageous when used to illuminate the eye for eye-tracking purposes. In the context of this document, light described as having wavelengths primarily in the NIR region generally refers to light with wavelengths in the range of 700–1400 nm or 680–1400 nm, unless explicitly stated otherwise. In the context of this document, light described as having wavelengths outside the NIR region generally refers to light with wavelengths less than 700 nm (or less than 680 nm) or greater than 1400 nm, unless explicitly stated otherwise.

[0072] In the context of this document, light in the visible region of the electromagnetic spectrum generally refers to light having wavelengths in the range of 380–750 nm. Therefore, there may be some overlap between the NIR region and the visible light region. In the context of this document, light described as having wavelengths mainly in the visible light region generally refers to light having wavelengths in the range of 380–700 nm or 380–680 nm, unless otherwise explicitly stated. In the context of this document, light described as having wavelengths mainly outside the visible light region generally refers to light having wavelengths less than 380 nm or greater than 700 nm (or greater than 680 nm), unless otherwise explicitly stated. The visible region is interchangeably referred to herein as the “visible light region,” the “light region,” and the “light spectrum.”

[0073] Unless otherwise defined herein, all technical and / or scientific terms used herein have the same meaning as commonly understood by those skilled in the art to whom the present invention relates. Similar or equivalent methods and materials may be used in carrying out or testing embodiments of the present invention, but exemplary methods and / or materials are described below. In case of any conflict, the patent specification, including definitions, shall prevail. Furthermore, materials, methods, and examples are illustrative and not necessarily intended to be limiting. [Brief explanation of the drawing]

[0074] Some embodiments of the present invention are described herein only as examples, with reference to the accompanying drawings. It should be emphasized that the details given, with regard to specific references to the drawings, are for illustrative purposes only and for the purpose of illustrative consideration of embodiments of the present invention. In this regard, the description made in conjunction with the drawings will make it clear to those skilled in the art how embodiments of the present invention may be carried out. Now, turning our attention to the drawings, we see that similar reference numbers or letters indicate corresponding or similar components. The drawings are as follows:

[0075] [Figure 1]The above is a schematic side view of a conventional optical guide element employing a partially reflective surface for use in near-eye displays. [Figure 2] This is a schematic side view of an optical system constructed and operable according to an embodiment of the present invention for displaying an image via an optical guide optical element, including a device for tracking the gaze direction of the human eye using an optical reversal arrangement implemented as a set of partially reflective surfaces associated with an optical guide optical element that deflects light toward an optical sensor. [Figure 3] This is a schematic side view of an alternative deployment configuration for the optical direction reversal arrangement shown in Figure 2. [Figure 4] Figure 2 is a schematic side view of yet another alternative deployment configuration of the optical direction reversal arrangement, where the partially reflective surface is in an overlapping relationship with the optical coupling-out configuration, which is implemented as another set of partially reflective surfaces that couple the image from the optical guide optics to the eye. [Figure 5] This is a schematic side view of an optical system constructed and operable according to another embodiment of the present invention for displaying an image via an optical guide optical element, which includes a device for tracking the gaze direction of the human eye using an optical direction reversal arrangement implemented as a diffractive optical element associated with a portion of one of the main surfaces of the optical guide optical element that deflects light toward an optical sensor. [Figure 6] This is a schematic side view of an alternative deployment configuration for the optical direction reversal arrangement in Figure 5, in which the diffractive optical element is associated with another portion of the main surface of the optical guide optical element. [Figure 7] This is a schematic side view of an optical system constructed and operable according to another embodiment of the present invention for displaying an image via an optical guide optical element, which includes a device for tracking the gaze direction of the human eye using an optical reversal arrangement implemented as a set of selectively reflective surfaces associated with a portion of one of the main surfaces of the optical guide optical element, which deflects light toward an optical sensor. [Figure 8] This is a partial schematic isometric view of the optical system shown in Figure 2, implemented in the form factor of eyeglasses. [Figure 9] This is a schematic side view of an optical system constructed and operable according to an embodiment of the present invention, which combines two optical direction deflection arrangements that independently deflect light toward an optical sensor. [Figure 10] This represents two independent images captured by a single optical sensor using two optical direction reversal configurations. [Figure 11] This is a block diagram of a processing subsystem configured to process signals from an optical sensor in order to derive the current gaze direction of the eye. [Figure 12] This is a schematic side view of an optical system constructed and operable according to an embodiment of the present invention, having a device for tracking the gaze direction of the human eye using an optical refraction arrangement implemented as a set of partially reflective surfaces associated with a light-transmitting substrate that deflects light toward an optical sensor. [Modes for carrying out the invention]

[0076] Embodiments of the present invention provide various devices and corresponding methods for tracking the gaze direction of the human eye, based on imaging the eye through an optical arrangement associated with an optical guide element.

[0077] The principles and operation of the various eye-tracking devices according to the present invention can be better understood by referring to the drawings accompanying this description.

[0078] Before describing in detail at least one embodiment of the present invention, it should be understood that the present invention is not necessarily limited to its application to the construction details and arrangement of the components and / or methods described in the following description and / or shown in the drawings and / or examples. Other embodiments of the present invention are possible or can be practiced or implemented in various ways.

[0079] Referring here to the drawings, Figure 2 shows various aspects of the structure and operation of an optical system generally shown in 100, which is constructed and operable according to a non-limiting embodiment of the present invention for deriving the gaze direction of a human eye 152. Generally speaking, the optical system 100 includes a substrate 102 formed from a transparent material (such as glass) and having a pair of faces (main surfaces) 104, 106, the pair of faces such that one of the faces 104 faces the viewer's eye 152, and a device associated with the substrate 102 for deriving the gaze direction of the eye 152. The device includes an optical sensor 140 for sensing light, an optical refraction configuration (hereinafter referred to as the "optical refraction configuration") 124 associated with the substrate 102 for deflecting light reflected from the eye 152 toward the optical sensor 140, and a processing subsystem 146 for processing signals from the optical sensor 140 to derive the current gaze direction of the eye 152.

[0080] Light reflected from eye 152 propagates toward the optical deflection configuration 124 in the general reflection direction (roughly represented in the diagram by a thick arrow labeled "RD"), and light deflected by the optical deflection configuration 124 propagates toward the optical sensor 140 in the general deflection direction (roughly represented in the diagram by a thick arrow labeled "DD").

[0081] The illustrated embodiments are particularly suitable for augmented reality (AR) and virtual reality (VR) applications in which a projected image is coupled in to a substrate 102, guided through the substrate 102 by internal reflection, and coupled out from the substrate for viewing by an eye 152. However, it should be noted that embodiments of the present invention may also be suitable for non-AR / VR applications in which the substrate is not configured to guide light by internal reflection. Such embodiments are discussed in subsequent sections of this disclosure.

[0082] In the non-limiting embodiments shown in Figure 2, the substrate 102 is a light-transmitting substrate (i.e., an optical guide optics, or "LOE") having planar and mutually parallel surfaces 104, 106, as described with reference to Figure 1, and configured to guide light by internal reflection. In certain embodiments, propagation by internal reflection is in the form of total internal reflection (i.e., internal reflection is governed by a critical angle, as considered above), and in other embodiments, propagation by internal reflection is achieved by a coating applied to surfaces 104, 106 (e.g., an angle-selectively reflective coating).

[0083] In the illustrated embodiment, LOE 102 is part of a device for deriving the gaze direction, and the device is further configured to display an image to the eye 152 (when positioned within EMB 154). Here, the optical system 100 further includes an image projection device 108 (hereinafter referred to as the "image projector") that produces a projected image 110 schematically represented by an illumination beam 110 (within the visible light region of the electromagnetic spectrum) including sample rays 110A and 110B spanning the beam. Although not shown in the drawings, the image projector 108 includes a microdisplay such as liquid crystal on silicon (LCoS) or organic light-emitting diode (OLED) for generating image light, and a corresponding collimating optical system for collimating the image to infinity. When the microdisplay is mounted on a reflective or transmissive display, illumination components (such as one or more LEDs) and illumination optical systems (such as a beam splitter) are also included in the image projector 108 so as to guide light from the illumination components to the microdisplay and to the collimating optical system.

[0084] The image light 110 is coupled into the LOE 102 by an optical coupling-in configuration 112, schematically shown here as a wedge positioned on or near the inclined edge 118 of the LOE 102, so as to generate a reflected ray 114 (downward ray) that is trapped within the substrate 102 by internal reflection, and also generates a ray 116 (upward ray). However, it should be noted that other optical coupling-in configurations are possible, for example, including reflective surfaces positioned obliquely on surfaces 104 and 106.

[0085] The coupling-in image 110 propagates along the substrate 102 by repeated internal reflections from surfaces 104 and 106 (i.e., guided by the substrate 102) and collides with the optical coupling-out configuration 120 by a sequence of mutually parallel, partially reflective surfaces 120 arranged at oblique angles to the parallel surfaces 104 and 106 within the substrate 102, as schematically shown here, and is reflected by the optical coupling-out configuration 120 so that a portion of the image intensity is coupled out from the substrate 102 toward the eye 152. The coupled-out image light is schematically represented as rays 122A and 122B. The set of partially reflective surfaces 102 is merely an example of one non-limiting implementation of the optical coupling-out configuration, and other optical coupling configurations can be used to couple out the image light from the LOE 102. The optical coupling out configuration can be any optical coupling configuration that deflects a portion of the image incident light already propagating within LOE 102 by internal reflection at an angle such that the deflected portion of the image incident light exits LOE 102. Other examples of such preferred optical coupling arrangements include, but are not limited to, one or more diffractive optical elements and beam splitter arrangements located on either plane 104 or 106.

[0086] The illumination arrangement 126 is positioned to illuminate the EMB 154 with light (referred to as “eye-tracking light”) 130 so that the eye 152 is illuminated with eye-tracking light 130 when the eye 152 is positioned within the EMB 154. In certain preferred but non-limiting embodiments, the illumination arrangement 126 is configured to illuminate the EMB 154 with light having wavelengths outside the light-adapted region of the electromagnetic spectrum. In other words, the illumination arrangement 126 is preferably configured to illuminate the eye 152 with light invisible to the human eye so as not to interfere with the human field of vision. In particularly preferred but non-limiting embodiments, the illumination arrangement 126 is configured to illuminate the eye 152 with light having wavelengths in the near-infrared (NIR) region of the electromagnetic spectrum, preferably in the range of 700 to 1000 nanometers (nm), and in certain examples, in the range of 680 to 1000 nm. However, other non-limiting embodiments are considered in which the illumination arrangement 126 is configured to illuminate the eye 152 with light having wavelengths in the light-adapted spectrum (i.e., the visible light region of the electromagnetic spectrum) or in the region of the electromagnetic spectrum outside the visible, IR, and NIR regions.

[0087] The illumination arrangement 126 includes at least one light source, preferably more than one, each configured to illuminate the EMB 154 with eye-tracking light. The light sources (or more) of the illumination arrangement 126 can be implemented as LEDs(or more) or any other light sources configured to emit (produce) light in the eye-tracking spectrum. In certain non-limiting embodiments, the light sources of the illumination arrangement 126 are isotropic (or nearly isotropic) light sources that emit light in all directions. The illumination of the EMB 154 by the illumination arrangement 126 with eye-tracking light 130 is schematically represented in Figure 2 by sample rays 130A, 130B, and 130C. It should be understood that these rays 130A, 130B, and 130C merely represent the eye-tracking light 130 reaching the EMB 154 (and the eye 152), and other rays from other illumination directions may also reach the EMB 154.

[0088] The intensity of eye-tracking light 130 illuminating a certain ratio of eyes 152 is reflected by the eyes 152 as light reflected in the general reflection direction toward the optical refraction configuration 124. The intensity of the reflected light, schematically represented as reflected rays 132A, 132B, and 132C in a certain ratio, reaches the optical refraction configuration 124 associated with the substrate 102 and is deflected by the optical refraction configuration 124 as deflected light, schematically represented as reflected rays 138A, 138B, and 138C, in order to redirect the reflected light toward the optical sensor 140 in the general deflection direction. The deflection of the eye-tracking light by the optical refraction configuration 124 occurs on the substrate 102, meaning that the point of deflection of the eye-tracking light is within the substrate 102.

[0089] Explicitly, for the sake of brevity of presentation, this specification refers to a single “reflection direction” and a single “deflection direction” corresponding to the light wave emitted from the eye as it propagates toward an optical sensor via deflection by an optical direction reversal arrangement. These “directions” are used to represent the general propagation direction of the aforementioned light wave and, strictly speaking, indicate the general path through free space in which the light wave moves from one point to another, rather than the vector direction. Each of the individual reflected rays (e.g., rays 132A, 132B, 132C) has its own vector propagation direction, which spans together with the general reflection direction considered above. Similarly, each of the individual deflected rays (e.g., rays 138A, 138B, 138C) has its own vector propagation direction, which spans together with the general deflection direction considered above.

[0090] In the non-limiting embodiment shown in Figure 2, the optical reversal arrangement 124 is implemented as a sequence (set) of partially reflective surfaces 124 arranged in the substrate 102 at an oblique angle to parallel planes 104, 106. The partially reflective surfaces 124 are preferably parallel to each other, but this is not required. Furthermore, the partially reflective surfaces 124 are planar (i.e., flat) surfaces. Although three partially reflective surfaces 124 are shown herein, the optical reversal arrangement 124 may include fewer than three such surfaces or four or more such surfaces. Generally, the optical reversal arrangement 124 includes at least one partially reflective surface.

[0091] The implementation of the optical reversal arrangement as a set of partially reflective surfaces 124 is merely one exemplary example of the optical reversal arrangement. Other exemplary implementations of the optical reversal arrangement, including implementations based on diffraction and dielectric and / or dichroism coating techniques, are described in detail in subsequent sections of this disclosure.

[0092] The partially reflective surface 124 is configured to reflect light having wavelengths in the eye-tracking spectrum (i.e., to reflect light reflected by the eye 152 in response to illumination by the illumination arrangement 126). In certain non-limiting embodiments, the partially reflective surface 124 is further configured to transmit light having wavelengths within the light-adaptation region (e.g., to transmit visible light).

[0093] In certain non-limiting embodiments, the optical redirection configuration 124 is configured such that eye-tracking light reaching the optical redirection configuration 124 is deflected by the optical redirection configuration 124 at an angle of inclination with respect to the angle of incidence, meaning that the angle of the incident light (rays 132A, 132B, 132C) to the optical redirection configuration 124 measured relative to surface 104 is different from the angle of the deflected light (rays 138A, 138B, 138C). In a particular example, the angle of the incident light measured relative to surface 106 is, for example, the configuration angle (α) of the partially reflective surface 124. sur124) As a result, the angle is steeper than that of the deflected light (for example, as shown in Figure 2).

[0094] In the non-limiting example configuration illustrated in Figure 2, the light incident on the optical redirection configuration 124, and the light reflected by the optical redirection configuration 124, are refracted by the surface 106. In particular, the light reflected from the eye 152 (e.g., rays 132A, 132B, 132C) incident on the main surface 104 of LOE 102 is refracted by the surface 104 (circularly shown as refracted rays 134A, 134B, 134C). The refracted rays 134A, 134B, 134C are deflected by the optical redirection configuration 124, i.e., reflected by the partially reflective surface 124, so as to strike the main surface 104 of LOE 102 again (circularly shown as reflected rays 136A, 136B, 136C). The incident light rays 136A, 136B, and 136C are refracted again by surface 104 (figuratively shown as refracted rays 138A, 138B, and 138C) and propagate toward optical sensor 140 (in free space).

[0095] As should be clear, reflected light from eye 152 can strike different parts of the partially reflective surface 124, not just at the location shown in Figure 2. It should be made clear that beyond the illustrated rays 132A, 132B, and 132C, additional rays may be incident on surface 104 and produce refracted rays that strike parts or regions of the partially reflective surface 124 that substantially cover the entirety of each partially reflective surface 124.

[0096] In contrast to the image light 110 from the image projector 108 which is guided by the substrate 102, it is emphasized that the light deflected by the optical direction reversal arrangement 124 is not guided by the substrate 102. In particular, as defined above, the light deflected by the optical direction reversal arrangement 124 reaches the optical sensor 140 without being trapped by internal reflections within the substrate 102 (i.e., free-space propagation). When implemented as a set of partially reflective surfaces 124, the partially reflective surfaces 124 are positioned at an oblique angle (α) relative to the main surfaces 104, 106 so that eye-tracking light (rays 134A, 134B, 134C) incident on the partially reflective surfaces 124 is not trapped by internal reflections, but is reflected by the partially reflective surfaces 124 and propagates towards the optical sensor 140 under free-space propagation. sur 124) is deployed. In the illustrated embodiment, the reflected light from the eye 152 undergoes at most one reflection within the substrate 102 before reaching the optical sensor 140 (the single reflection is a reflection by a partially reflective surface 124).

[0097] A focusing optical system 142 (which may include a set of lenses), schematically represented as a lens, is deployed in the optical path between the optical reversal arrangement and the optical sensor 140. The focusing optical system 142 receives the deflected light 138A, 138B, and 138C by the optical reversal arrangement 124 and converts the received light 138A, 138B, and 138C into a focused beam of light ( schematically represented as rays 144A, 144B, and 144C) that strikes the optical sensor 140. In certain preferred embodiments, the focusing optical system 142 is an imaging optical system that forms an image of the eye 152 on the optical sensor 140. The focusing optical system 142 is preferably deployed to define a field of view corresponding to a region or portion associated with the substrate 102 to which the deflected eye-tracking light reaches, so as to enable the focusing optical system 142 to capture the deflected eye-tracking light. The region or portion associated with the substrate 102 is generally the region or portion to which the optical reversal arrangement is deployed. The focusing optical system 142 is preferably integrated with an optical sensor 140 as part of a camera system deployed between the eye 152 and the surface 104 to image the eye 152 by capturing deflected light 138A, 138B, and 138C by an optical direction reversal arrangement 124.

[0098] The processing subsystem 146 is electrically associated with the optical sensor 140 and configured to process signals from the optical sensor 140 to derive the current gaze direction of the eye 152. The processing subsystem 146 is also preferably electrically associated with the lighting arrangement 126 to control the lighting timing of the EMB by the lighting arrangement 126. The processing system 146 can be implemented using any suitable type of processing hardware and / or software known in the art, including but not limited to any combination of various dedicated computerized processors that operate under any suitable operating system and implement suitable software or firmware modules. The processing system 146 may further include various communication components to enable wired or wireless communication with LAN and / or WAN devices for bidirectional transfer of information and graphic content. A simplified block diagram of the processing subsystem 146 in a non-limiting exemplary implementation is shown in Figure 11. Here, the processing subsystem 146 includes at least one computerized processor 148 coupled to a storage medium 150. The storage medium 150 can be one or more computerized memory devices, such as volatile data storage. The processor 148 can be implemented as any number of computerized processors, including but not limited to microprocessors, microcontrollers, graphics processors, display drivers, application-specific integrated circuits (ASICs), digital signal processors (DSPs), image processors, field-programmable gate arrays (FPGAs), and field-programmable logic arrays (FPLAs). Such a computerized processor, when executed by the computerized processor, may include or be able to electronically communicate with computer-readable media that stores program code or instruction sets that cause the computerized processor to perform actions.Types of computer-readable media include, but are not limited to, electronic, optical, magnetic, or other storage or transmission devices that can provide computer-readable instructions to a computerized processor.

[0099] The following paragraphs describe various deployment options for the optical direction reversal arrangement 124 according to non-limiting embodiments of the present invention. In non-limiting implementations in which the optical direction reversal arrangement 124 is implemented as a set of partially reflective surfaces 124, the deployment angle (α) of the partially reflective surfaces 124 is in the range of 10 to 35 degrees. sur It has been found that 124) is particularly suitable for deflecting light in a manner that does not guide it from the EMB 154 (eye 152) to the optical sensor 140. In particular, such an arrangement angle is suitable for deflecting incident light (rays 132A, 132B, 132C) incident on the main surface 104 at an angle of incidence (AOI) in the range of 18 to 40 degrees to a deflection angle in the range of 60 to 90 degrees (where the deflection angle is measured between rays 138A, 138B, 138C and the normal to the main surface 104). The angle range of 18 to 40 degrees corresponds to the angular distribution of eye-tracking light reflected from the eye toward the optical deflection arrangement 124.

[0100] As mentioned above, each of the partially reflective surfaces preferably (but not required) has the same deployment angle (α sur 124) has. Furthermore, the deployment angle (α) of the partially reflective surface 124. sur 124) is the deployment angle (α) of the partially reflective surface 120. sur120) may be identical to or different from 120). Generally, the trade-off between the manufacturing complexity of the optical system 100 (in particular, the substrate 102 having a set of embedded partially reflective surfaces 120 and 124) and the position of the optical sensor 140 (i.e., camera) may be considered when determining the deployment angle of the partially reflective surface 124. For example, deploying the partially reflective surfaces 120, 124 at a common oblique angle to the main surfaces 104, 106 (i.e., so that the surfaces 120, 124 are parallel to each other) may offer certain advantages during the manufacturing of the substrate 102 by reducing the number of manufacturing steps. Typically, the substrate 102 having embedded partially reflective surfaces is manufactured by deploying a stack of transparent plates coated with a partially reflective coating and then slicing the stack at an oblique angle (e.g., as described in U.S. Patent No. 8,432,614) to form a slab (substrate) with embedded oblique-angled partial reflectors. Since the lamination and slicing of the coated plates can be carried out in the same manner as described in U.S. Patent No. 8,432,614, the manufacturing process of the substrate 102 of the present invention is simplified when the partially reflective surfaces 120, 124 are arranged at a common oblique angle.

[0101] However, if the partially reflective surfaces 120 and 124 are arranged at different oblique angles (i.e., such that the partially reflective surfaces 124 and 120 are not parallel to each other), the complexity of the manufacturing process may increase by requiring additional steps. Here, the manufacturing process may generally include steps such as producing a first portion of the substrate having partially reflective surfaces 120 parallel to each other at a first oblique angle, producing a second portion of the substrate having partially reflective surfaces 124 parallel to each other at a second oblique angle, and joining the two substrate portions together (e.g., via optical cement) to form a single substrate having two sets of partially reflective surfaces at two different arrangement angles. While having manufacturing disadvantages, the two different arrangement angles can offer certain advantages in the arrangement of the components of the optical system 100. In particular, the different arrangement angles provide flexibility in the arrangement (i.e., spatial positioning) of the optical sensor 140 relative to the substrate 102. For example, the ability to deploy the partially reflective surface 124 at a steeper or shallower angle than the deployment angle of the partially reflective surface 120 may allow the optical sensor 140 to be positioned further outside the viewer's peripheral field of view than would be possible when using a common deployment angle for the partially reflective surfaces 120, 124.

[0102] In addition to having the same or different orientation angles, the partially reflective surfaces 124 and 120 can be deployed in the same or different planar orientations. Preferably, the partially reflective surfaces 124 are mutually parallel planes lying on a first set of mutually parallel planes. Similarly, the partially reflective surfaces 124 are mutually parallel planes lying on a second set of mutually parallel planes. In one non-limiting deployment configuration, for example, as shown in Figure 2, the first and second sets of planes are parallel (i.e., all of the reflective surfaces 124 and 120 are parallel to each other). In another non-limiting deployment configuration, the first and second sets of planes are oriented orthogonally to each other (i.e., each plane of the partially reflective surface 124 is orthogonal to each plane of the partially reflective surface 120). In yet another non-restrictive deployment configuration, the first and second sets of planes are neither parallel nor orthogonal to each other (i.e., they are oriented obliquely, i.e., each plane of the partially reflective surface 124 is oblique to each plane of the partially reflective surface 120). As with the above consideration of deployment angles, when considering the planar orientation of the partially reflective surfaces 120, 124, there may be a trade-off between manufacturing complexity and flexibility in the layout of the components. Using parallel planes may provide the simplest manufacturing process for constructing the substrate 102 with embedded partially reflective surfaces 120, 124. However, sets of orthogonal or oblique planes may provide flexibility with respect to the placement of the optical sensor 140.

[0103] The partially reflective surface 124 is preferably dimensional and preferably positioned at an angle and orientation such that both horizontal and vertical eye movements across the desired vertical and horizontal fields are captured. It should be noted that the human gaze direction is wider across the horizontal field of view than across the vertical field of view. Therefore, in certain preferred embodiments, the partially reflective surface 124 is dimensional and positioned at a certain orientation (plane and positioning angle) to deflect light over a wider angular range coming from the horizontal field than from the vertical field.

[0104] In the non-limiting embodiment illustrated in Figure 2, the optical direction reversal configuration 124 is located at or near the distal end of the substrate 102 (the proximal end of the substrate 102 is the end where the optical coupling in configuration 112 and the image projector 108 are located), and the optical coupling out configuration 120 is located along the central portion of the substrate 102 between the proximal and distal ends. However, the configuration shown in Figure 2 is only one example of a configuration. Other configurations of the optical direction reversal configuration 124 are intended herein. For example, Figure 3 shows another non-limiting configuration in which the optical direction reversal configuration 124 is located at or near the proximal end of the substrate 102, and the optical coupling out configuration 120 is located along the central portion of the substrate 102 between the proximal and distal ends.

[0105] In the non-limiting deployment configuration described above with reference to Figures 2 and 3, the optical reversal configuration 124 and the optical coupling out configuration 120 are separated into respective regions of the substrate 102 such that the optical reversal configuration 124 does not overlap with the optical coupling out configuration 120. In other words, the partially reflective surface 124 does not overlap with any of the partially reflective surfaces 120. However, in other non-limiting exemplary deployment configurations, the optical reversal configuration 124 is deployed so as to at least partially overlap with the optical coupling out configuration 120. An example of such a configuration is shown in Figure 4, where the partially reflective surface 124 intersects with the partially reflective surface 120.

[0106] Referring here to Figures 5 and 6, there exists a shown optical redirection arrangement 224 associated with the substrate 102, which is implemented as one or more diffractive optical elements (referred to interchangeably as diffraction gratings) for deflecting eye-tracking light toward the optical sensor 140, according to a non-limiting embodiment of the present invention. Similar to the embodiments described with reference to Figures 2-4, the light deflected toward the optical sensor 140 by the optical redirection arrangement 224 is not induced by the substrate 102.

[0107] Referring first to Figure 5, the optical direction reversal arrangement 224 is implemented as a diffractive optical element deployed on one of the main surfaces 104 of the substrate 102. As described above with reference to Figure 2, the intensity of a certain proportion of the reflected eye-tracking light (rays 132A, 132B, 132C) propagating in the general reflection direction reaches the diffraction grating 224 and is deflected by the optical direction reversal arrangement 124 as deflected light (rays 138A, 138B, 138C), redirecting the light (rays 132A, 132B, 132C) toward the optical sensor 140 in the general deflection direction. The deflection of the eye-tracking light by the optical direction reversal arrangement 224 occurs in the substrate 102, meaning that the deflection point of the eye-tracking light is within the substrate 102 (as shown in Figure 6) and / or near one of the main surfaces of the substrate 102 (as shown in both Figures 5 and 6).

[0108] In this non-limiting example configuration, the diffraction grating 224 is deployed on the main surface 104 at or near the distal end of the substrate 102 and is non-overlapping with the optical coupling-out configuration 120 (as described with reference to Figure 2). However, it should be noted that other non-overlapping deployment locations of the diffraction grating 224 on the main surface 104 may include the proximal end of the substrate 102 or near it (as described with reference to Figure 3). Furthermore, the diffraction grating 224 may be deployed on the main surface 104 in a manner that overlaps with the optical coupling-out configuration 120. For example, the diffraction grating 224 may be deployed along a portion of the same part of the main surface 104 over which one or more projections of the partially reflective surface 120 extend.

[0109] Similar to the optical redirection configuration 124 (implemented as at least one partially reflective surface), the optical redirection configuration 224 in a particular deployment configuration is such that eye-tracking light (rays 132A, 132B, 132C) reaching the optical redirection configuration 224 is deflected by the optical redirection configuration 124 at an inclination angle, meaning that the angle of incident light (rays 132A, 132B, 132C) to the optical redirection configuration 224 measured relative to surface 106 is different from the angle of deflected light (rays 138A, 138B, 138C). In a particular example, the angle of incident light measured relative to surface 106 is steeper than the angle of deflected light by the optical redirection configuration 224.

[0110] Unlike the optical direction reversal configuration 124, it should be noted that when the diffractive optical element is deployed on the main surface 104, the eye tracking light is not refracted by the main surface 104. However, in the non-limiting implementation shown in Figure 6, the diffraction grating is deployed on the main surface 106 such that the eye tracking light is refracted by the main surface 104. Here, the light reflected from the eye 152 (e.g., rays 132A, 132B, 132C) propagating in the general reflection direction incident on the main surface 104 of LOE 102 is refracted by surface 104 as refracted light (rays 134A, 134B, 134C). The refracted light 134A, 134B, 134C is deflected by the diffraction grating 224 as deflected light (reflected rays 136A, 136B, 136C) so that it strikes the main surface 104 of LOE 102 again. The incident light (rays 136A, 136B, 136C) is refracted again by surface 104 as refracted light (rays 138A, 138B, 138C), which propagates in free space toward the optical sensor 140 in the general deflection direction. Note that in this embodiment, as in the embodiments described with reference to Figures 2-4, the light from eye 152 undergoes at most one reflection within substrate 102 before reaching optical sensor 140, and that the single reflection is a reflection by diffraction grating 224.

[0111] In the non-limiting exemplary configuration shown in Figure 6, the diffraction grating 124 is located at or near the distal end of the substrate 102 and is non-overlapping with the optical coupling-out configuration 120 (as described with reference to Figures 2 and 5). However, it should be noted that other non-overlapping locations for the diffraction grating 224 on the main surface 106 may include the proximal end of the substrate 102 or near it (as described with reference to Figure 3). Furthermore, the diffraction grating 224 may be located on the main surface 106 in a manner that overlaps with the optical coupling-out configuration 120 (as described with reference to Figure 5).

[0112] While a single continuous diffraction grating deployed along portions of the main surfaces 104 and 106 is shown in Figures 5 and 6, respectively, an array of diffractive optical elements (i.e., two or more diffraction gratings) may be deployed discontinuously along at least one portion of the main surfaces 104 and 106 of the substrate 102.

[0113] Similar to the optical direction reversal configuration 124, the optical direction reversal configuration 224 is preferably configured to deflect incident light (rays 132A, 132B, 132C) incident on the main surface 104 with an AOI in the range of 18 to 40 degrees to a deflection angle in the range of 60 to 90 degrees (where the deflection angle is measured between rays 138A, 138B, 138C and the normal to the main surface 104).

[0114] Referring here to Figure 7, an optical redirection arrangement 324 associated with a substrate 102 for deflecting eye-tracking light toward an optical sensor 140 is shown according to another non-limiting embodiment of the present invention. Similar to the embodiments described with reference to Figures 2-6, the light deflected toward the optical sensor 140 by the optical redirection arrangement 324 is not guided by the substrate 102. In this embodiment, the optical redirection arrangement 324 is implemented as a selectively reflective surface associated with at least one portion of one of the main surfaces 106 of the substrate 102 (i.e., the surface of the substrate 102 facing away from the eye 152). The selectively reflective surface is preferably formed from one or more layers of optical coatings, such as a dielectric and / or dichroic coating, applied to at least one portion of the main surfaces 106 of the substrate 102. In this embodiment, the eye-tracking light reflected from the eye propagates in the general reflection direction and is deflected by the dielectric and / or dichroic coating on the substrate 102 (in this example, on the main surface 106). The dielectric and / or dichroic coating is designed to be highly transmittant to light in the light-adapted spectrum (i.e., light having wavelengths in the visible region of the electromagnetic spectrum) and highly reflectable to light having wavelengths in the eye-tracking spectrum (e.g., in the NIR region) so that a viewer can see a real-world scene through the substrate 102. A “hot mirror” is a special class of dielectric mirror having a dichroic filter that provides the required transmission and reflection properties for the optical reversal arrangement 324. To form the optical reversal arrangement 324, a simple hot mirror can be deployed in the required area or portion of the main surface 106 via optical cement or mechanical mounting.

[0115] In this embodiment, as in the embodiments described with reference to Figures 2-6, it should be noted that the light from the eye 152 undergoes at most one reflection within the substrate 102 before reaching the optical sensor 140, and that this single reflection is due to the optical coating (or coating) used to implement the light direction reversal arrangement 324.

[0116] The optical refraction arrangement 324 is preferably configured to deflect incident light (rays 132A, 132B, 132C) incident on the main surface 104 in an AOI range of 18 to 40 degrees by a deflection angle within the same angular range as the incident light (where the deflection angle is measured between rays 138A, 138B, 138C and the normal to the main surface 104). The incident light angular range of 18 to 40 degrees corresponds to the angular distribution of eye-tracking light reflected from the eye toward the optical refraction arrangement 124.

[0117] In certain embodiments, a color-selective reflective coating may be applied to a region of the substrate 102 where the optical direction reversal arrangement 324 is formed, in order to support the aforementioned angular distribution of incident light and the angular range of deflected light.

[0118] In the non-limiting implementation shown in Figure 7, an optical refraction arrangement 324 (hot mirror) is positioned on the main surface 106 so that the eye-tracking light is refracted by the main surface 104. Here, the light reflected from the eye 152 (e.g., rays 132A, 132B, 132C) propagating in the general reflection direction incident on the main surface 104 of the substrate 102 is refracted by the surface 104 as refracted light (rays 134A, 134B, 134C). The refracted light 134A, 134B, 134C is deflected by the hot mirror 324 as deflected light (refracted rays 136A, 136B, 136C) so that it strikes the main surface 104 of the substrate 102 again. The incident light (rays 136A, 136B, 136C) is refracted again by surface 104 as refracted light (rays 138A, 138B, 138C), which propagates in free space toward the optical sensor 140 in the general deflection direction.

[0119] The deflection of eye-tracking light on the main surface 106 allows the optical sensor 140 (i.e., the camera system) to be positioned closer to the main surface 104, thereby increasing the distance between the optical sensor 140 (i.e., the camera system) and the eye 152 (and EMB), providing the viewer with a more comfortable viewing experience.

[0120] In the non-limiting configuration illustrated in Figure 7, the main surface 104 is preferably coated with an anti-reflective coating that reduces reflections in both the light-adapted spectrum (e.g., the visible light region) and the eye-tracking spectrum (e.g., the NIR region) so as to allow light from the external scene to pass through the surface 104 to the viewer's eye 152 and eye-tracking light deflected by the light-direction-reversing arrangement 324 to pass through the surface 104 (with potential refraction) to the optical sensor 140.

[0121] Figure 7 shows an implementation of the optical reversal arrangement 324 deployed on at least a portion of the surface 106 of the substrate 102, either near or at the proximal edge of the substrate 102, but such an implementation is only a non-limiting example of the optical reversal arrangement 324. Other non-limiting implementations are possible in which the optical reversal arrangement 324 is deployed on at least a portion of the main surface 104, either near or at the proximal edge of the substrate 102. In such implementations, the optical reversal arrangement 324 is preferably implemented as a dielectric coating (referred to as a “dielectric mirror”) applied to at least a portion of the surface 104. In this implementation, the optical sensor 140 should be deployed further away from the surface 104 than in the implementation shown in Figure 7, so that the optical sensor 140 can capture eye-tracking light covering a wide angular distribution. Therefore, the mounting configuration of the optical direction reversal arrangement 324 on surface 104 is less preferable than the mounting configuration shown in Figure 7, because the distance between the optical sensor 140 (i.e., the camera system) and the eye 152 (and EMB) is reduced compared to the mounting configuration shown in Figure 7.

[0122] It should be noted that the sections of the main surfaces 104, 106 in which the optical direction reversal arrangement 324 is deployed may be a single continuous portion of the main surfaces 104, 106, or may be one or more discontinuous portions (i.e., separate segments) of the main surfaces 104, 106.

[0123] The non-limiting implementation forms of the optical reversal configuration 324 described so far relate to the deployment of a reflective surface on one or more portions of the surfaces 104, 106 at or near the proximal end of the substrate 102. These implementation forms of the optical reversal configuration 324 do not overlap with the optical coupling out configuration 120 (similar to the one described with reference to Figure 3). However, it should be noted that other non-overlapping deployment locations of the optical reversal configuration 324 on one of the main surfaces 104, 106 are possible, including at or near the distal end of the substrate 102 (similar to the ones described with reference to Figures 2 and 5). Furthermore, the optical reversal configuration 324 may be deployed on one of the main surfaces 104, 106 in a manner that overlaps with the optical coupling out configuration 120. For example, the reflective surface may be deployed along a portion of the same main surface 104, 106 on which one or more projections of the partially reflective surface 120 extend.

[0124] The optical redirection arrangements 124, 224, and 324 according to various embodiments of the present invention can be positioned on or near a portion of the main surface 104 such that the eye-tracking light transmitted from the eye 152 to the optical redirection arrangements 124, 224, and 324 is first incident on a portion of the main surface 104 at an oblique angle (i.e., a relatively high AOI measured with respect to the normal to the surface 104), and (in embodiments where the surface 104 is planar) at an equivalent shallow angle with respect to the plane of the surface 104. This is most clearly shown in Figures 2, 5, and 6, where the rays 132A, 132B, and 132C (representing the eye-tracking light transmitted from the eye 152 to the optical redirection arrangements) are incident at a relatively high AOI. The high AOI and reflective properties of the optical reversal arrangement (including, for example, the arrangement angle of the partially reflective surface 124) are beneficial for deflected light (rays 138A, 138B, 138C) which have a higher AOI than the incident light 132A, 132B, 132C.

[0125] Such deployment configurations of the optical direction reversal arrangements 124, 224, and 324 allow for the arrangement of the optical sensor 140 (and focusing optical system 142) mounted on a mechanical body or housing that accommodates the components of the image projector 108, or near the image projector 108, and in certain preferred embodiments.

[0126] The following paragraphs describe some of the deployment options for the illumination arrangement 126. Generally speaking, various configurations of the illumination arrangement 126 are intended. In all of the illumination arrangement configurations, the illumination arrangement 126 includes at least one light source configured to illuminate the EMB 154 (and thus the eye 152 when positioned at the EMB 154) with eye-tracking light in the NIR region of the electromagnetic spectrum, in a preferred but non-limiting implementation.

[0127] In one non-restrictive deployment configuration, eye-tracking illumination may be incorporated as part of the image projected by the image projector 108. The illumination may occur during image projection or at a separate time period. Timing control of the eye-tracking illumination and image projection is preferably controlled by the processing subsystem 146. When the eye-tracking illumination is in the IR region, there are various options for providing IR illumination to the eye. When NIR wavelengths close to visible wavelengths are used, the IR illumination may be combined as a fourth "color" in a conventional visible image projector, for example, using an LCoS modulator. If patterned illumination is desired for longer wavelength IR, a digital photoprocessing (DPL) device is typically preferred. For unpatterned illumination, a dedicated illumination source is typically provided independently of the image projector. The eye-tracking illumination generated by the image projector is coupled into the substrate 102, as with the visible image illumination 110, and propagated by internal reflection. The optical coupling out configuration 120 is preferably configured to transmit light in the eye-tracking spectrum, while the optical reversal configurations 124, 224, and 324 are preferably configured to deflect light in the eye-tracking spectrum and transmit it within the optical spectrum. The desired reflection and transmission properties of the optical coupling out configuration 120 can be achieved by applying a suitable coating to the partially reflective surface 120. Similarly, the desired reflection and transmission properties of the optical reversal configuration 124 can be achieved by applying a suitable coating to the partially reflective surface 124.

[0128] In a more preferred configuration, the lighting arrangement 126 includes one or more light sources positioned separate from the image projector, close to the optical sensor 140, and / or around the mechanical body of the optical system 100 to which the substrate 102 is mounted. Figure 8 shows such a preferred non-limiting example in which the optical system 100 is implemented in an eyeglass form factor with a head-mounted mechanical body implemented as an eyeglass frame 156 with side arms 158 for resting over the ears of the observer (viewer). The optical system 100 is powered from a preferred power source, which may be any combination of a battery and / or an externally provided power source, schematically shown here as a power source 160 connected via a cable 162. When using a battery power source, the battery can be integrated as part of a structure mounted on eyeglasses or a helmet. It should be noted that other form factors, such as a helmet-mounted form factor, a vehicle windshield form factor, and other head-up and near-eye display form factors, are also clearly within the scope of the present invention.

[0129] In this non-exclusive implementation, the lighting arrangement 126 includes three separate light sources 128A, 128B, and 128C (implemented, for example, as three LEDs). Two of the sources 128A and 128B are positioned on or near the optical coupling out configuration 120 on the peripheral portion of the eyeglass frame 156. The third light source 128C is positioned near the side of the observer's head, close to the optical sensor 140 (shown in Figure 8 as being mounted on an image projector 108 attached to a side arm 158).

[0130] In general, the illumination arrangement 126 may be configured to illuminate a specific area of ​​the eye 152 or the entire eye 152 with eye-tracking light. As discussed in detail above, the eye-tracking light emitted by the illumination arrangement (rays 130A, 130B, 130C) is reflected by the eye 152 as reflected light 132A, 132B, 132C and deflected towards the optical sensor 140 by the light reversal arrangements 124, 224, 324 as deflected light (rays 138A, 138B, 138C), and preferably focused onto the optical sensor 140 (by the optical system 142). In certain non-limiting embodiments, the deflected light (e.g., rays 138A, 138B, 138C) is focused onto different areas or portions of the optical sensor 140 by the focusing optical system 142. The optical sensor 140 generates signals in response to the detection of focused light (e.g., corresponding to an image of the eye), and these signals are transferred to a processing system 146 configured to process the signals to derive the current gaze direction of the eye 152. In certain non-limiting embodiments, the optical system 100 obtains the gaze direction (angle orientation of the eye, or line of sight of the eye 152) by imaging patterns present in a specific region of the eye. The position and movement of such patterns indicate the current gaze direction and eye movement. The human eye includes a variety of traceable features, including, for example, patterns generated by corneal nerves based on corneal reflexes (i.e., corneal nerve patterns), the center of the pupil of the eye, and patterns generated by the blood vessels of the optic disc. These traceable features can be tracked using appropriate tracking algorithms implemented by preferred image processing instructions performed by the processing system 146. In certain non-limiting embodiments, the processing system 146 calculates the gaze direction based on a vector between the pupil center and the corneal reflex.

[0131] Generally, all background illumination introduces noise that degrades the quality of the eye image. To reduce the impact of external light sources (e.g., ambient light, natural sunlight), the illumination arrangement 126 may be configured to generate short pulses of light (preferably less than 1 ms), and the optical sensor 140 may be synchronized (by the processing subsystem 146) to integrate the light only during this short illumination duration. In this way, continuous background illumination can be significantly suppressed. Additionally or alternatively, a passband spectral filter may be deployed in the optical path from the optical reversal arrangement to the optical sensor 140 to block light of wavelengths outside a given range of wavelengths, within which eye-tracking illumination is generated from reaching the optical sensor 140. The spectral filter may ideally be positioned between the focusing optical system 142 and the optical sensor 140, but alternatively, it may be deployed before the focusing optical system 142.

[0132] In a non-restrictive process for deriving and tracking the gaze direction, the retinal pattern is mapped (optionally in combination with the optic disc pattern and / or pupil), traceable features are determined during the initialization process, and then a continuous tracking process is performed. For example, an image marker may be displayed to the observer so that the observer can see it during initialization. While the observer is facing the marker, the illumination arrangement 126 fully illuminates the cornea, and a complete image of the cornea (and pupil) is acquired (via the optical sensor 140). This image is then processed by the processing system 146 to identify traceable features (e.g., the optic disc and fovea). During the continuous tracking process, a selected region of interest (ROI) 152 of an eye is selectively illuminated by the illumination arrangement 126, and an image of the ROI (acquired by the optical sensor 140) is sampled and processed (by the processing system 146) during the corresponding illumination pulse to determine the current gaze direction (line of sight). Using this derived gaze direction, the position of the ROI for subsequent illumination cycles is updated, and the continuous tracking process is repeated by illuminating the updated ROI. Assuming that the frequency of tracking measurements is high compared to the speed of eye movement, this update process is typically effective in maintaining continuous tracking and can be optionally combined with tracking information from other eyes. As the gaze direction changes, the illuminated area also changes. ROI updates may be performed according to the "current" gaze direction determined from the last sampled image, or, in some cases, predictive extrapolation based on eye movement between the previous two or more measurements may be used. If tracking fails, the size of the illuminated area can be temporarily increased until trackable functionality is restored.

[0133] The light sources of the illumination arrangement 126 may be configured to emit eye-tracking light at approximately the same or different central wavelengths within the eye-tracking spectrum. Typically, in the NIR region, the dispersion of the glass material on which the substrate 102 can be constructed is low enough that it does not suffer from distortion within the spectral width of a single eye-tracking light source (the spectral width is typically in the range of 20-50 nm for LEDs). However, using light sources that emit eye-tracking light at two spectrally separated central wavelengths (while still within the same region of the eye-tracking electromagnetic spectrum) can offer certain advantages when imaging the eye. For example, arranging the illumination arrangement 126 with first and second light sources emitting light centered around approximately 700 nm and 950 nm, respectively, can result in two different images of the eye formed on the optical sensor 140, one shifted relative to the other. By applying appropriate image processing algorithms, such as correlation algorithms, the processing subsystem 146 can achieve higher resolution in calculating the gaze direction.

[0134] It should be noted that the processing subsystem 146 and the camera (optical sensor with a focusing optical system) are preferably positioned close to each other to allow for a simple electrical connection (e.g., a short electrical wire or cable) between the optical sensor and the processing subsystem 146, as shown in Figure 8, for example. Furthermore, it should be noted that the optical refraction arrangement of the present invention is positioned to deflect light in the general deflection direction corresponding to the region or area in which the camera is positioned. As described above, the focusing optical system 142 defines a field of view corresponding to the region to which the deflected eye-tracking light reaches. Therefore, the focusing optical system 142 is positioned to capture light from the deflection direction. In the non-limiting exemplary implementation shown in Figure 8, the focusing optical system 142 is positioned to the right of the spectacle frame 156 to capture deflected eye-tracking light emitted from the region to the left of the focusing optical system 142. Therefore, the optical refraction arrangement in Figure 8 should be positioned to deflect light to the right in the general deflection direction (i.e., towards the focusing optical system 142). However, the optical direction reversal configuration can be configured to deflect light to the left in the general deflection direction, provided that a focusing optical system is deployed accordingly. In Figure 8, this involves positioning the focusing optical system to the left side of the spectacle frame 156, which is (directly or indirectly) attached to the left side arm (not shown). The processing subsystem 146 preferably also controls the image projector 108, and it may be advantageous to keep the processing subsystem 146 close to the image projector 108, at the expense of proximity to the camera, and instead deploy a longer electrical connection between the optical sensor and the processing subsystem 146.

[0135] The general deflection direction associated with an optical reversal configuration can be easily changed from rightward to leftward (and vice versa) by changing the configuration orientation and / or optical parameters of the optical reversal configuration. For example, looking at the general rightward deflection direction of the partially reflective surface 124 in Figure 2, a general leftward deflection direction can be achieved by rotating the partially reflective surface 124 180 degrees around a vertical or horizontal axis to change the orientation of the partially reflective surface 124. In another example, the periodic structure used to form the diffraction grating 224 can be inverted or rotated to change the deflection direction.

[0136] Returning to Figures 2-7, any pair of optical reversal arrangements 124, 224, and 324 can be used in combination to acquire two independent images of the eye 152 using a single optical sensor 140. Preferably, the two optical reversal arrangements are deployed on different parts of the substrate 102. For example, one optical reversal arrangement may be deployed at or near the proximal end of the substrate 102, while the other optical reversal arrangement may be deployed at or near the distal end of the substrate 102. The two optical reversal arrangements may be of the same type (e.g., both may be implemented as partially reflective surfaces) or of different types (e.g., one may be implemented as a set of partially reflective surfaces, and the other as one or more diffraction gratings).

[0137] Figure 9 shows a schematic diagram of a non-limiting example of an optical system having two different optical direction reversal arrangements. In this non-limiting example, the first optical direction reversal arrangement 124 is implemented as described with reference to Figure 2, and the second optical direction reversal arrangement 324 is implemented as described with reference to Figure 7. The crossing of light from the eye to the optical sensor 140 is schematically shown in Figure 9. For brevity, the refraction of eye-tracking light by the main surface 104 is not shown in the figure, but it should be understood that the refraction of eye-tracking light can occur as described with reference to Figures 2 and 7.

[0138] Eye-tracking illumination is reflected from the eye 152, schematically represented by rays 132A, 132B, 132C, 132D, 132E, and 132F. Some of the eye-tracking light reflected by the eye, schematically represented by a first set of rays 132A, 132B, and 132C, propagates in a first general reflection direction ( schematically represented by a thick arrow labeled "RD1") and is deflected by the optical redirection configuration 124 as deflected light ( schematically represented by rays 138A, 138B, and 138C) in a first general deflection direction ( schematically represented by a thick arrow labeled "DD1"). Some of the eye-tracking light reflected by the eye, schematically represented by a second set of rays 132D, 132E, and 132F, propagates in a second general reflection direction ( schematically represented by a thick arrow labeled "RD2") and is deflected by the optical redirection configuration 324 as deflected light ( schematically represented by rays 138D, 138E, and 138F) in a second general deflection direction ( schematically represented by a thick arrow labeled "DD2"). The deflected light reaches the focusing optical system 142 in the form of two sets of deflected rays. The focusing optical system 142 converts the first set of deflected incident light 138A, 138B, and 138C into a focused beam of light ( schematically represented as rays 144A, 144B, and 144C) that strikes a designated first portion or region 141a of the optical sensor 140 so as to form a first image of the eye 152 within the EMB. The focusing optical system 142 also converts a second set of deflected incident light 138D, 138E, and 138F into a focused beam of light ( schematically represented as rays 144D, 144E, and 144F) that collide with a designated second portion or region 141b of the optical sensor 140 to form a second image of the eye 152 within the EMB.

[0139] Figure 10 shows two independent images captured by a single optical sensor 140 using two optical direction reversal arrangements in a laboratory simulation.

[0140] It should be noted that the processing subsystem 146 may process the signals generated by the optical sensor 140 to calculate or estimate the eye relief distance using triangulation techniques when generating two independent images of the eye. It should also be noted that, in principle, it is possible to generate three or more sets of independent images of the eye by using a combination of three or more optical direction change arrangements.

[0141] Certain embodiments of this disclosure may be particularly valuable when deployed as part of a head-up display (HUD) in a vehicle or aircraft, thereby allowing the display of images projected by the image projector 108 in an automotive or aviation environment to depend on, or at least partially control, the direction in which the user of the HUD is looking. In an automotive environment, a HUD employing the main components of an eye-tracking device according to the disclosed embodiments may be installed, for example, in front of the driver of a vehicle, integrated into the vehicle's dashboard or windshield. In an aviation environment, a HUD may be installed in front of an aircraft pilot, for example, as part of a pilot helmet in the forward area of ​​the helmet.

[0142] Embodiments of the optical system have so far been described in the context of a substrate 102 which is an optical guide element (LOE) configured to guide image light (injected from an image projector 108) by internal reflection. Such embodiments are particularly valuable when used in AR and / or VR applications, where the AR / VR image is generated by a compact image projector having a small aperture that is amplified by the LOE to produce a large aperture. As discussed in the background section, approaches to one-dimensional aperture augmentation have been developed based on parallel-plane slabs of transparent material in which the image propagates by internal reflection. It should also be noted that two-dimensional aperture augmentation has been developed using various optical waveguide configurations. In one example configuration, two-dimensional aperture augmentation is achieved by first and second optical waveguides. The first optical waveguide has two pairs of parallel main external surfaces that form a rectangular cross section. A first set of mutually parallel, partially reflective surfaces traverses the first optical waveguide obliquely with respect to the extension direction of the optical waveguide. A second optical waveguide, optically coupled to a first optical waveguide, has a pair of parallel main outer surfaces that form a slab-type waveguide. A second set of mutually parallel, partially reflective surfaces obliquely traverses the second optical waveguide to the main outer surfaces of the second optical waveguide. Furthermore, the plane containing the first set of partially reflective surfaces is preferably oblique to the plane containing the second set of partially reflective surfaces.The optical coupling between the two optical waveguides, and the arrangement and configuration of the two sets of partially reflective surfaces, are such that when the image is coupled to the first optical waveguide in the initial direction of propagation at an oblique coupling angle to both pairs of the parallel main external surfaces of the first optical waveguide, the image propagates along the first optical waveguide (i.e., in two dimensions) by four times internal reflection with a certain intensity of the image reflected by the first set of partially reflective surfaces, so that the image is coupled out of the first optical waveguide and coupled in of the second optical waveguide, and then propagates through two times internal reflection in the second optical waveguide (i.e., in one dimension) so that the image is coupled out of the first optical waveguide and coupled in of the second optical waveguide, so that the image is coupled out of the first optical waveguide and then propagates through two times internal reflection in the second optical waveguide (i.e., in one dimension) so that the image is coupled out of the second optical waveguide and then propagates along the second optical waveguide (i.e., in one dimension) by four times internal reflection, so that the image is coupled out of the first optical waveguide and coupled in of the second optical waveguide, so that the image is reflected by a certain intensity of the image reflected by the second set of partially reflective surfaces, so that the image is coupled out of the second optical waveguide and then propagates along the second optical waveguide (i.e., in one dimension) by four times internal reflection, so that the image is coupled out of the first When the optical reversal arrangement is implemented as a set of partially reflective surfaces 124 in parallel planes, the planes of the partially reflective surfaces 124 can be oriented parallel, perpendicular, or oblique to the planes of a first or second set of mutually parallel partially reflective surfaces. Further details of the two-dimensional aperture multiplier can be found in various patent documents, including, for example, U.S. Patent No. 10,564,417, which is incorporated herein by reference in its entirety.

[0143] In another example configuration, two-dimensional aperture expansion is achieved by first and second slab-type optical waveguides. The first optical waveguide has two pairs of parallel main outer surfaces that form a slab-type waveguide. A first set of mutually parallel internal partially reflective surfaces traverses the first optical waveguide at an oblique angle to the two pairs of parallel main outer surfaces. The second optical waveguide also has two pairs of parallel main outer surfaces. A second set of mutually parallel internal partially reflective surfaces traverses the second optical waveguide at an oblique angle to the two pairs of parallel main outer surfaces of the second optical waveguide. Furthermore, the plane containing the first set of partially reflective surfaces is oblique or perpendicular to the plane containing the second set of partially reflective surfaces. The optical coupling between the two optical waveguides, and the arrangement and configuration of the two partially reflective surfaces, are such that when an image is coupled in to the first optical waveguide, the image propagates through a double internal reflection within the first optical waveguide between one of the pair of outer surfaces in the first optical waveguide with a certain intensity of the image reflected by the first set of partially reflective surfaces, so that when the image is coupled in to the first optical waveguide, the image propagates through a double internal reflection within the first optical waveguide between one of the pair of outer surfaces in the first optical waveguide in the first induced direction, so that when the image is coupled in to the first optical waveguide, the image propagates through a double internal reflection within the second optical waveguide between one of the pair of outer surfaces in the second optical waveguide in the second induced direction (oblique to the first induced direction), so that when the image is coupled in to the first optical waveguide, the image propagates through a certain intensity of the image reflected by the second set of partially reflective surfaces, so that when the image is coupled in to the second optical waveguide, the image propagates through a double internal reflection within the second optical waveguide between one of the pair of outer surfaces in the second optical waveguide in the second induced direction (oblique to the first induced direction), so that when the image is coupled in to the first optical waveguide, the image propagates through a double internal reflection within the second optical waveguide between one of the pair of outer surfaces in the second optical waveguide in the second induced direction, so that when the image is coupled in to the first optical waveguide, the image propagates through a certain intensity of the image reflected by the second set of partially reflective surfaces, so that when the image is coupled in to the second optical waveguide, the image propagates through a double internal reflection within the second optical waveguide between one of the pair of outer surfaces in the second optical waveguide in the second induced direction, so that when the image is coupled in to the first optical waveguide, the image propagates through a certain intensity of the image reflected by the first set of partially reflective surfaces. When the optical reversal arrangement is implemented as a set of partially reflective surfaces 124 in parallel planes, the planes of the partially reflective surfaces 124 can be oriented parallel, perpendicular, or oblique to the planes of a first or second set of mutually parallel partially reflective surfaces. Further details of the two-dimensional aperture multiplier can be found in various patent documents, including, for example, U.S. Patent No. 10,551,544, which is incorporated herein by reference in its entirety.

[0144] Furthermore, it should be noted that devices for guiding the direction of gaze can also be valuable when used in non-AR or non-VR applications. In one additional set of applications, devices for guiding the direction of gaze can be advantageously used in combination with assemblies having any type of substrate formed from a material that can at least partially transmit light and is positioned in front of the viewer's eyes. The “substrate” in such assemblies may include, but are not limited to, sunglasses, eyeglasses, eyeglasses with optical diopters, eyeglasses with diffraction gratings or grids, and eye protection devices having a shield or dome-shaped structure formed from plastic or glass and positioned to protect the eyes from debris or particulate matter. Some or all of the main external surfaces of the aforementioned substrates may be curved surfaces, unlike the main external surface of the LOE, which is a flat plane.

[0145] In another set of additional applications, the gaze direction deriving device can be used in conjunction with computer or mobile device-related applications, and the gaze direction of the user's eyes may be used to navigate display screens, web pages, menus, etc., or to interact with computerized games played on computer devices (e.g., video game systems, mobile devices, laptop computers, tables, etc.). In such applications, the “substrate” may include the display screen of the computer device, and the light direction deflection arrangement may be appropriately associated with a portion of the display screen to deflect the eye-tracking light reflected from the eyes to the display screen.

[0146] Figure 12 schematically illustrates an embodiment of the present invention that may be used in the non-AR / VR applications described above. Here, the substrate 102 is formed from a light-transmitting material that forms part of the assembly of the type described above and is not configured to guide light by internal reflection. Furthermore, since the light is not guided by the substrate 102, there is no optical coupling out configuration associated with the substrate 102. Instead, in this example, which is implemented as a set of partially reflective surfaces 124, only an optical reversal arrangement is associated with the substrate 102. The optical reversal arrangement 124 deflects the incident eye-tracking light (approximately represented by rays 132A, 132B, and 132C) toward the optical sensor 140 as deflected light (approximately represented by rays 138A, 138B, and 138C). For simplicity, please note that the refraction of eye-tracking light 132A, 132B, and 132C by the main surface 104 is not shown in the figures, but please understand that the refraction of eye-tracking light can occur in the same way as described with reference to Figures 2 and 7.

[0147] As should be understood, the embodiment shown in Figure 12 can be implemented either individually or in paired combination using any of the optical refraction arrangements 124, 224, and 324, as described above. In such non-AR / VR applications, the EMB is generally defined as a two-dimensional region positioned at or within a given distance from the surface 104 of the substrate 102, and as a result, when the eye is positioned within the EMB, the optical sensor 140 can sense the light from the eye deflected by the optical refraction arrangements 124, 224, and 324, and preferably image the portion of the eye.

[0148] The embodiments of the present disclosure described herein so far relate to illuminating the eye 152 within the EMB with eye-tracking light, preferably in the non-visible region (preferably the NIR region) of the electromagnetic spectrum; however, other embodiments are possible in which the eye is illuminated with light having wavelengths outside the NIR region, including, for example, visible light and ultraviolet (UV) light. In embodiments in which the illumination arrangement 126 illuminates the eye with visible light, it may be advantageous to deploy the illumination source(s) to concentrate the illumination on areas of the eye that are less sensitive to visible light, such as the sclera, so as to avoid hitting the eye with non-image visible light. In embodiments in which the illumination arrangement 126 irradiates the eye with UV light, precautions should be taken to mitigate or minimize the eye's exposure to harmful UV radiation, for example, by limiting the intensity / power of the UV beam received on a given area of ​​the eye over a given duration (e.g., for UV light having wavelengths in the range of 315 nm to 400 nm, for a period exceeding 1000 seconds, and less than 1 milliwatt per square centimeter).

[0149] It should also be noted that the light sources of the illumination arrangement 126 may be configured to emit light in two or more different regions of the electromagnetic spectrum. For example, a first set of light sources (i.e., one or more) may emit light in the NIR region, while a second set of light sources may emit light in the visible region (preferably focused on the sclera).

[0150] According to certain non-limiting embodiments, various eye-tracking devices of this disclosure may be replicated to track both eyes of a subject simultaneously. In addition, when the eye-tracking device is integrated as part of an optical system that also projects images onto the eyes, the optical system may be replicated to project images onto both eyes. Combining data from two eye trackers may allow for improved tracking stability and continuity. For example, while the eyes are moving, the trackable portion of the eye may be displayed on the tracker for one eye but not for the other. When using a tracking algorithm that employs tracking of trackable features, simultaneous tracking of both eyes allows for continuous tracking throughout periods when only one eye tracker can track blind spots.

[0151] In a binocular optical system, each eye has its own image projection and eye-tracking device, and various processing and power supply components can be optionally shared between the two eye-tracking systems. Eye-tracking information collected by binocular eye-tracking devices can be merged, as described above, to enhance tracking stability and continuity.

[0152] The descriptions of the various embodiments of this disclosure have been presented for illustrative purposes only and are not intended to be exhaustive or limitful to the embodiments disclosed. Many modifications and variations that do not deviate from the scope and spirit of the embodiments described will be apparent to those skilled in the art. The terminology used herein has been chosen to best describe the principles of the embodiments, their practical applications or technical improvements to marketable technologies, or to enable those skilled in the art to understand the embodiments disclosed herein.

[0153] As used herein, the singular forms "a," "an," and "the" include plural references unless the context clearly indicates otherwise.

[0154] The term “exemplary” is used herein to mean “serving as an example, illustration, or illustration.” Any embodiment described as “exemplary” should not necessarily be construed as being preferable or advantageous to other embodiments, and / or preclude the incorporation of features from other embodiments.

[0155] For clarity, it is understood that certain features of the Invention described in the context of separate embodiments may also be provided in combination in a single embodiment. Conversely, for brevity, various features of the Invention described in the context of a single embodiment may also be provided separately, in any preferred partial combination, or as suitable in any other described embodiment of the Invention. Certain features described in the context of different embodiments should not be considered essential features of those embodiments unless the embodiments would not function without those elements.

[0156] The attached claims are drafted without multiple dependencies, solely to comply with the formal requirements of jurisdictions that do not permit such multiple dependencies. It should be noted that all possible combinations of features that would be implied by making the claims multiple dependencies are explicitly assumed and should be considered part of the invention.

[0157] While the present invention has been described in conjunction with its specific embodiments, it is evident that many alternatives, modifications, and variations will be apparent to those skilled in the art. Therefore, it is intended to encompass all such alternatives, modifications, and variations that fall within the spirit and broad scope of the appended claims.

Claims

1. An optical system, A light-transmitting substrate having two mutually parallel main external surfaces, wherein one of the two mutually parallel main external surfaces is positioned facing the viewer's eye, An optical coupling-in configuration for coupling in light corresponding to a collimated image into the light-transmitting substrate so that it propagates within the light-transmitting substrate by internal reflection between the main outer surfaces, An optical coupling out configuration for coupling out light propagating within the light-transmitting substrate by internal reflection, wherein the optical coupling out configuration includes at least two first internal surfaces arranged at an oblique angle to the two parallel main surfaces within the light-transmitting substrate, and the at least two first internal surfaces are flat, parallel to each other, and are partially reflective surfaces. Optical sensors positioned to detect light, An optical direction reversal arrangement comprising at least one second internal surface arranged at an oblique angle to the two parallel main surfaces within the light-transmitting substrate, wherein the at least one second internal surface is a flat partial reflective surface that deflects light from the eye toward the optical sensor, and as a result, the deflected light reaching the optical sensor is not guided by the light-transmitting substrate, and the deflection of light due to the optical direction reversal arrangement occurs within the light-transmitting substrate, An optical system comprising: at least one processor electrically coupled to the optical sensor and configured to process signals from the optical sensor to derive the current gaze direction of the eye.

2. The optical system according to claim 1, wherein the optical coupling out configuration is arranged in a non-overlapping relationship with respect to the optical direction reversal arrangement.

3. The optical system according to claim 1, wherein the optical coupling out configuration is arranged in an overlapping relationship with the optical direction reversal configuration.

4. The optical system according to claim 1, wherein the light corresponding to the collimated image mainly comprises light having wavelengths in the visible light region of the electromagnetic spectrum, and the light from the eye deflected by the light direction reversal arrangement mainly comprises light having wavelengths outside the visible light region of the electromagnetic spectrum.

5. The optical system according to claim 1, further comprising at least one imaging optical element arranged in an optical path from the optical direction reversal arrangement to the optical sensor for forming at least one image of at least a portion of the eye on the optical sensor.

6. An optical system, A light-transmitting substrate having two mutually parallel main external surfaces, wherein one of the two mutually parallel main external surfaces is positioned facing the viewer's eye, An optical coupling-in configuration for coupling in light corresponding to a collimated image into the light-transmitting substrate so that it propagates within the light-transmitting substrate by internal reflection between the main outer surfaces, At least two first internal surfaces located within the light-transmitting substrate for coupling out light propagating within the light-transmitting substrate by internal reflection, wherein the at least two first internal surfaces are flat, parallel to each other, and partially reflective surfaces at an oblique angle to the two main external surfaces, Optical sensors positioned to detect light, An optical direction reversal arrangement comprising at least one second internal surface arranged at an oblique angle to the two parallel main surfaces within the light-transmitting substrate, wherein the at least one second internal surface is a flat partial reflective surface that deflects light from the eye toward the optical sensor, and as a result, the deflected light reaching the optical sensor is not guided by the light-transmitting substrate, and the deflection of the light ray due to the optical direction reversal arrangement occurs within the light-transmitting substrate, An optical system comprising: at least one processor electrically coupled to the optical sensor and configured to process signals from the optical sensor to derive the current gaze direction of the eye.

7. The optical system according to claim 6, wherein the at least one second internal surface transmits light having wavelengths in the visible light region of the electromagnetic spectrum and reflects light having wavelengths outside the visible light region of the electromagnetic spectrum.

8. The optical system according to claim 6, wherein the at least two first internal surfaces and the at least one second internal surface are parallel to each other.

9. The optical system according to claim 6, wherein the at least two first internal surfaces are nonparallel to the at least one second internal surface.

10. The optical system according to claim 6, wherein the at least two first internal surfaces are arranged in a non-overlapping relationship with respect to the at least one second internal surface.

11. The optical system according to claim 6, wherein the at least two first internal surfaces are arranged in an overlapping relationship with respect to the at least one second internal surface.

12. The optical system according to claim 6, further comprising at least one imaging optical element arranged in an optical path from the optical direction reversal arrangement to the optical sensor in order to form at least one image of at least a portion of the eye on the optical sensor.

13. An optical system, Optical sensors positioned to detect light, An optical reversal arrangement provided on or within a light-transmitting substrate having at least two parallel main surfaces, wherein the first of the at least two parallel main surfaces is positioned facing the viewer's eye, the light-transmitting substrate is configured to induce image illumination corresponding to an infinitely collimated image by internal reflection between the two parallel main surfaces of the light-transmitting substrate, and the light-transmitting substrate has at least two first internal surfaces positioned at an oblique angle to the two parallel main surfaces, wherein the at least two first internal surfaces are flat, parallel to each other, and induced forward To couple out the image illumination from the light-transmitting substrate to the viewer's eye, the optical direction reversal arrangement includes a partially reflective surface, the optical direction reversal arrangement includes at least one second internal surface positioned at an oblique angle to the two parallel main surfaces within the light-transmitting substrate, the at least one second internal surface being a flat partially reflective surface that deflects light from the eye toward the optical sensor, and as a result, the deflected light reaching the optical sensor is not guided by the light-transmitting substrate, and the deflection of light by the optical direction reversal arrangement occurs within the light-transmitting substrate, An optical system comprising: at least one processor electrically coupled to the optical sensor and configured to process signals from the optical sensor to derive the current gaze direction of the eye.

14. An optical system, A light-transmitting substrate having at least two parallel main surfaces, wherein the first of the at least two parallel main surfaces is positioned opposite the viewer's eye, and the position of the eye relative to the light-transmitting substrate defines an eye motion box, and the light-transmitting substrate is configured to induce image illumination corresponding to an infinitely collimated image by internal reflection between the two parallel main surfaces of the light-transmitting substrate, and the light-transmitting substrate includes at least two first internal surfaces positioned between the two parallel main surfaces at an oblique angle to the two parallel main surfaces, wherein the at least two first internal surfaces are flat, parallel to each other, and are partially reflective surfaces for coupling out the induced image illumination from the light-transmitting substrate to the viewer's eye, The eye-motion box is illuminated with eye-tracking light, and the lighting arrangement is configured such that a certain ratio of the intensity of the eye-tracking light is reflected by the eye as reflected light. Optical sensors positioned to detect light, An optical direction reversal arrangement comprising at least one second internal surface arranged at an oblique angle with respect to the two parallel main surfaces within the light-transmitting substrate, wherein the at least one second internal surface is a flat partial reflective surface configured to deflect the reflected light toward the optical sensor, the deflected light reaching the optical sensor is not guided by the light-transmitting substrate, and the deflection of the light due to the optical direction reversal arrangement occurs within the light-transmitting substrate; An optical system comprising: at least one processor electrically coupled to the optical sensor and configured to process signals from the optical sensor to derive the current gaze direction of the eye.

15. An optical system, A light-transmitting substrate having at least two parallel main surfaces, wherein the first of the two parallel main surfaces is positioned facing the viewer's eye, and the light-transmitting substrate is configured to induce image illumination corresponding to an infinitely collimated image by internal reflection between the two parallel main surfaces of the light-transmitting substrate, At least two first internal surfaces arranged at an oblique angle to the two parallel main surfaces within the light-transmitting substrate, wherein the at least two first internal surfaces are flat, parallel to each other, and are partially reflective surfaces for coupling out the induced image illumination from the light-transmitting substrate to the viewer's eyes, Optical sensors positioned to detect light, An optical reversal arrangement comprising at least one second internal surface positioned at an oblique angle with respect to the two parallel main surfaces within the light-transmitting substrate, wherein the at least one second internal surface is a flat partial reflective surface and is configured to deflect light from the eye toward the optical sensor, so that the deflected light reaching the optical sensor is not guided by the light-transmitting substrate, the deflection of light by the optical reversal arrangement occurs within the light-transmitting substrate, the optical reversal arrangement deflects a first set of light rays from the eye, through the imaging lens toward the optical sensor, so as to form a first image of at least a portion of the eye, and the optical system deflects a second set of light rays from the eye, through the imaging lens toward the optical sensor, so as to form a second image of at least a portion of the eye, further comprising a second optical reversal arrangement, An optical system comprising: at least one processor electrically coupled to the optical sensor and configured to process signals from the optical sensor to derive the current gaze direction of the eye, wherein the at least one processor is further configured to process signals from the optical sensor corresponding to the first and second images to determine the distance between the eye and the first of the main surfaces.

16. An optical system, A light-transmitting substrate having at least two parallel main surfaces, wherein the first of the two parallel main surfaces is positioned facing the viewer's eye, and the light-transmitting substrate is configured to induce image illumination corresponding to an infinitely collimated image by internal reflection between the two parallel main surfaces of the light-transmitting substrate, At least two first internal surfaces arranged at an oblique angle to the two parallel main surfaces within the light-transmitting substrate, wherein the at least two first internal surfaces are flat, parallel to each other, and are partially reflective surfaces for coupling out the induced image illumination from the light-transmitting substrate to the viewer's eyes, Optical sensors positioned to detect light, An optical reversal arrangement comprising at least one second internal surface positioned at an oblique angle with respect to the two parallel main surfaces within the light-transmitting substrate, wherein the at least one second internal surface is a flat partial reflective surface and is configured to deflect light from the eye toward the optical sensor, so that the deflected light reaching the optical sensor is not guided by the light-transmitting substrate, the deflection of light by the optical reversal arrangement occurs within the light-transmitting substrate, the optical reversal arrangement deflects a first set of light rays from the eye, through the imaging lens toward the optical sensor, so as to form a first image of at least a portion of the eye, and the optical system deflects a second set of light rays from the eye, through the imaging lens toward the optical sensor, so as to form a second image of at least a portion of the eye, further comprising a second optical reversal arrangement, An optical system comprising: at least one processor electrically coupled to the optical sensor and configured to process signals from the optical sensor to derive the current gaze direction of the eye, wherein the optical direction reversal arrangement includes at least one partially reflective surface located within a first portion of the light-transmitting substrate, and one of a diffracting element provided on at least a first portion of one of the main surfaces of the light-transmitting substrate, or a selectively reflective surface provided on at least a first portion of one of the main surfaces of the light-transmitting substrate, and the second optical direction reversal arrangement includes at least one partially reflective surface located within a second portion of the light-transmitting substrate, a diffracting element provided on at least a second portion of one of the main surfaces of the light-transmitting substrate, or a selectively reflective surface provided on at least a second portion of one of the main surfaces of the light-transmitting substrate, wherein the selectively reflective surface is designed to be highly transmittant to light in the light-adapted spectrum and highly reflectant to light having wavelengths in the eye-tracking spectrum.

17. An optical system, A light-transmitting substrate having at least two parallel main surfaces, wherein the first of the two main surfaces is positioned facing the viewer's eye, the light-transmitting substrate is configured to induce image illumination corresponding to an infinitely collimated image by internal reflection between the two parallel main surfaces of the light-transmitting substrate, and the light-transmitting substrate is integrated as part of a near-eye display. At least two first internal surfaces arranged at an oblique angle to the two parallel main surfaces within the light-transmitting substrate, wherein the at least two first internal surfaces are flat, parallel to each other, and are partially reflective surfaces for coupling out the induced image illumination from the light-transmitting substrate to the viewer's eyes, Optical sensors positioned to detect light, An optical direction reversal arrangement comprising at least one second internal surface arranged at an oblique angle to the two parallel main surfaces within the light-transmitting substrate, wherein the at least one second internal surface is a flat partial reflective surface that deflects light from the eye toward the optical sensor, and as a result, the deflected light reaching the optical sensor is not guided by the light-transmitting substrate, and the deflection of light due to the optical direction reversal arrangement occurs within the light-transmitting substrate, An optical system comprising: at least one processor electrically coupled to the optical sensor and configured to process signals from the optical sensor to derive the current gaze direction of the eye.

18. An optical system, A light-transmitting substrate having at least two parallel main surfaces, wherein the first of the two main surfaces is positioned facing the viewer's eye, the light-transmitting substrate is configured to induce image illumination corresponding to an infinitely collimated image by internal reflection between the two parallel main surfaces of the light-transmitting substrate, and the light-transmitting substrate is integrated as part of a head-up display. At least two first internal surfaces arranged at an oblique angle to the two parallel main surfaces within the light-transmitting substrate, wherein the at least two first internal surfaces are flat, parallel to each other, and are partially reflective surfaces for coupling out the induced image illumination from the light-transmitting substrate to the viewer's eyes, Optical sensors positioned to detect light, An optical direction reversal arrangement comprising at least one second internal surface arranged at an oblique angle to the two parallel main surfaces within the light-transmitting substrate, wherein the at least one second internal surface is a flat partial reflective surface that deflects light from the eye toward the optical sensor, and as a result, the deflected light reaching the optical sensor is not guided by the light-transmitting substrate, and the deflection of light due to the optical direction reversal arrangement occurs within the light-transmitting substrate, An optical system comprising: at least one processor electrically coupled to the optical sensor and configured to process signals from the optical sensor to derive the current gaze direction of the eye.

19. An optical system, A light-transmitting substrate having at least two parallel main surfaces, wherein the first of the two parallel main surfaces is positioned facing the viewer's eye, and the light-transmitting substrate is configured to induce image illumination corresponding to an infinitely collimated image by internal reflection between the two parallel main surfaces of the light-transmitting substrate, At least two first internal surfaces arranged at an oblique angle to the two parallel main surfaces within the light-transmitting substrate, wherein the at least two first internal surfaces are flat, parallel to each other, and are partially reflective surfaces for coupling out the induced image illumination from the light-transmitting substrate to the viewer's eyes, Optical sensors positioned to detect light, An optical reversal arrangement comprising at least one second internal surface positioned at an oblique angle to the two parallel main surfaces within the light-transmitting substrate, wherein the at least one second internal surface is a flat partial reflective surface that deflects light from the eye toward the optical sensor, and as a result, the deflected light reaching the optical sensor is not guided by the light-transmitting substrate, the deflection of light by the optical reversal arrangement occurs within the light-transmitting substrate, and the light from the eye, deflected by the optical reversal arrangement, undergoes at most one reflection within the light-transmitting substrate before reaching the optical sensor, An optical system comprising: at least one processor electrically coupled to the optical sensor and configured to process signals from the optical sensor to derive the current gaze direction of the eye.

Images