Waveguide systems for in-field SMI eye tracking

Abstract

Systems, devices, and methods for eye tracking and other sensing using self-mixing interferometry (SMI) through a waveguide in a near-eye device, i.e., in-field SMI eye tracking. The near-eye display device may include a waveguide having SMI coupler(s) and eye-side coupler(s) where one or more SMI(s): (i) project light into the SMI coupler(s) through the waveguide and out the eye-side coupler(s) to the user's eye; and (ii) receive light reflected back from the user's eye into the eye-side coupler(s) through the waveguide and out the SMI coupler(s). The one or more SMIs mix / modulate the received reflected light with presently projected light to provide an electrical signal for eye tracking. The SMI and / or eye-side couplers may be reflective, refractive, and / or diffractive; interior / exterior gratings, lenses, and / or coatings may be utilized in / on the waveguide to redirect, modulate, and / or otherwise alter the projected / received light.

Description

WAVEGUIDE SYSTEMS FOR IN-FIELD SMI EYE TRACKINGCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This patent application claims the benefit of priority to U.S. Provisional Patent Application Serial No. 63 / 553,928 filed on February 15, 2024.TECHNICAL FIELD

[0002] This patent application relates generally to eye tracking, and in particular to one or more self-mixing interferometers (SMIs) being coupled to the waveguide of a near-eye display device to act as eye tracking projectors / sensors.BACKGROUND

[0003] With recent advances in technology, prevalence and proliferation of content creation and delivery has increased greatly in recent years. In particular, interactive content such as virtual reality (VR) content, augmented reality (AR) content, mixed reality (MR) content, and other content within and / or associated with a real and / or virtual environment (e.g., a “metaverse”) have become more widely available to consumers. This interactive content includes simulated three-dimensional (3D) environments, objects, images, representations, art and the like.

[0004] To facilitate delivery of this and other related content, service providers have endeavored to provide various forms of wearable display systems. One such example may be a near-eye display device, such as, e.g., a wearable headset or headmounted display (HMD) device, a wearable eyewear, or eyeglasses (e.g., “smartglasses”). In some examples, the head-mounted display (HMD) device may project or direct light to may display virtual objects or combine images of real objects with virtual objects, as in virtual reality (VR), augmented reality (AR), or mixed reality (MR) applications. For example, in an augmented reality (AR) system, a user may view both images of virtual objects (e.g., computer-generated images (CGIs)) and the surrounding environment. Head-mounted display (HMD) devices may also present interactive content, where a user’s (wearer’s) gaze may be used as input for the interactive content.

[0005] Wearable display devices, such as virtual reality (VR), augmented reality (AR), and / or mixed reality (MR) glasses, may require increasingly complex and intricate lens assembly structures, as well as increasingly complex and intricate electronic structures, etc., thereby complicating, inter alia, the manufacturing process. Moreover, the need for both electronics and optics to have a relatively small size and negligible weight for portability and user comfort, as well as the ability to operate in awide variety of environments, produces a host of challenges and competing concerns, in areas such as, for example, eye tracking.SU M MARY

[0006] According to an aspect of the present invention, there is provided a neareye display device, comprising: a frame; a waveguide disposed in the frame to transmit light by internal reflection and to provide Augmented Reality (AR)ZVirtual Reality (VR) images to a user’s eye; an eye tracking self-mixing interferometer (SMI) disposed in the frame to provide projected light to the user’s eye and to receive reflected light from the projected light on the user’s eye; an SMI-side coupler on the waveguide to receive into the waveguide the projected light from the eye tracking SMI and to project from the waveguide to the eye tracking SMI the reflected light from the user’s eye; and an eye-side coupler on the waveguide to project the projected light from the waveguide onto the user’s eye and to receive into the waveguide the reflected light from the user’s eye; wherein the eye tracking SMI is to further modulate the received reflected light with presently projected light to provide an electrical signal corresponding to the modulated light, wherein non-image-based eye tracking is performed based on the electrical signal.

[0007] Optionally, at least one of the waveguide, the SMI-side coupler, or the eye-side coupler is further to redirect the projected light from the eye tracking SMI in a desired direction towards the user’s eye.

[0008] Optionally, at least one of the waveguide, the SMI-side coupler, or the eye-side coupler is further to add a desired angular orientation, polarization, or focusing power to the projected light as the projected light is projected to the user’s eye.

[0009] Optionally, the eye tracking SMI is integrated with another eye tracking system in a sensor fusion framework.

[0010] Optionally, the eye tracking SMI is operated in at least one of a coherent sensing or Frequency Modulated Continuous Wave (FMCW) sensing mode.

[0011] Optionally, the SMI-side coupler or the eye-side coupler comprises at least one of a prism, a surface, a coating, a reflective grating, a refractive grating, a diffractive grating, a metasurface, a metalens, or a lens.

[0012] Optionally, the diffractive grating comprises at least one of a Polarization Volumetric Hologram-based (PVH) grating, a Surface Relief Grating (SRG), or a Volume Bragg Gratings (VBG).

[0013] Optionally, the near-eye display device further comprises: an in-field waveguide camera disposed on the frame.

[0014] Optionally, the near-eye display device further comprises: a waveguide coupler on the waveguide to receive light from the SMI-side coupler and project it to the eye-side coupler and to receive light from the eye-side coupler and project it to the SMI-side coupler.

[0015] Optionally, the eye tracking SMI comprises a vertical cavity self-emitting laser (VCSEL).

[0016] According to a further aspect of the present invention there is provided a near-eye display device, comprising: a frame; a waveguide disposed in the frame to transmit light by internal reflection and to provide Augmented Reality (AR)A / irtual Reality (VR) images to a user’s eye; a plurality of eye tracking self-mixing interferometers (SMIs) disposed in the frame to provide projected light for the user’s eye and to receive reflected light from the projected light on the user’s eye; a plurality of SMI-side couplers on the waveguide to receive into the waveguide the projected light from the eye tracking SMI and to project from the waveguide to the eye tracking SMI the reflected light from the user’s eye; and a plurality of eye-side couplers on the waveguide to project the projected light from the waveguide onto the user’s eye and to receive into the waveguide the reflected light from the user’s eye; wherein the eye tracking SMI is to further modulate the received reflected light with presently projected light to provide an electrical signal corresponding to the modulated light, wherein non- image-based eye tracking is performed based on the electrical signal.

[0017] Optionally, the plurality of eye tracking SMIs are sparsely populated on the surface of the waveguide or disposed as a group or array on the surface of the waveguide.

[0018] Optionally, each of the plurality of eye tracking SMIs has a matching one of the plurality of SMI-side couplers and a matching one of the plurality of eye-side couplers, and wherein the projected light from each of the eye tracking SMIs incouples only through its matching SMI-side coupler and out-couples only through its matching eye-side coupler.

[0019] Optionally, at least one of the plurality of eye-side couplers directs the projected light from its matching eye tracking SMI in a desired direction towards the user’s eye.

[0020] Optionally, at least one of the plurality of eye-side couplers adds adesired angular orientation, polarization, or focusing power to the projected light from its matching eye tracking SMI as the projected light is projected to the user’s eye.

[0021] Optionally, the plurality of eye tracking SMIs are integrated with another eye tracking system in a sensor fusion framework.

[0022] Optionally, each of the plurality of SMI-side couplers or the plurality of eye-side couplers comprises at least one of a prism, a surface, a coating, a reflective grating, a refractive grating, a diffractive grating, a metasurface, a metalens, or a lens.

[0023] According to a further aspect of the present invention, there is provided a method of eye tracking in a near-eye display device, comprising: projecting, by a self-mixing interferometer (SMI) eye tracking sensor in a frame of the near-eye display device, a beam of light to be projected onto an eye of a user of the near-eye display device; receiving, by an SMI coupler of a waveguide of the near-eye display device, the projected beam of light from the SMI eye tracking sensor; modulating, by a modulator in the waveguide, the received beam of light for projecting the beam of light onto the eye of a user; projecting, from the waveguide, the modulated light onto the eye of the user; receiving, by the waveguide, reflected light from the eye of the user by the projected modulated beam of light; providing, by the SMI coupler, the received reflected light to the SMI eye tracking sensor; receiving and mixing, by the SMI eye tracking sensor, the received reflected light from the eye of the user; providing, by the SMI eye tracking sensor, an electrical signal corresponding to the received and mixed light; and performing eye tracking based on the electrical signal provided by the SMI eye tracking sensor.

[0024] Optionally, the modulator in the waveguide comprises at least one of a prism, a surface, a coating, a reflective grating, a refractive grating, a diffractive grating, a metasurface, a metalens, or a lens.

[0025] Optionally, the step of modulating, by a modulator in the waveguide, the received beam of light for projecting the beam of light onto the eye of a user further comprises at least one of: redirecting the beam of light projected by the eye tracking SMI in a desired direction towards the eye of the user; or adding a desired at least one of an angular orientation, polarization, or focusing power to the beam of light projected by the eye tracking SMI as the projected light is projected towards the eye of the user. BRIEF DESCRIPTION OF DRAWINGS

[0026] Features of the present disclosure are illustrated by way of example and not limited in the following figures, in which like numerals indicate like elements. Oneskilled in the art will readily recognize from the following that alternative examples of the structures and methods illustrated in the figures can be employed without departing from the principles described herein.

[0027] FIG. 1 illustrates a block diagram of an artificial reality system environment including a near-eye display device, according to an example.

[0028] FIGS. 2A and 2B illustrate a front prospective view and a back prospective view, respectively, of a near-eye display device in the form of a headmounted display (HMD) device to which examples of the present disclosure may be applied.

[0029] FIGS. 3A and 3B illustrate a perspective view and a top view, respectively, of a near-eye display device in the form of a pair of glasses to which examples of the present disclosure may be applied.

[0030] FIGS. 4A and 4B are simplified block diagrams of a side view and a planar view, respectively, of a waveguide with eye tracking Self-Mixing Interferometers (SMIs), in-couplers, and out-couplers, according to an example.

[0031] FIGS. 5 and 6 are simplified block diagrams of waveguides with eye tracking Self-Mixing Interferometers (SMIs) using Volume Bragg Gratings (VBGs), according to examples.

[0032] FIG. 7 is a flowchart illustrating a method for eye tracking using SelfMixing Interferometer (SMI) eye tracking sensors, according to an example.

[0033] FIGS. 8 and 9 are graphs and a diagram illustrating the coupling efficiency of Volume Bragg Gratings (VBGs) used as couplings with eye tracking SelfMixing Interferometers (SMIs), according to examples.DETAILED DESCRIPTION

[0034] For simplicity and illustrative purposes, the present application is described by referring mainly to examples thereof. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. It will be readily apparent, however, that the present application may be practiced without limitation to these specific details. In other instances, some methods and structures readily understood by one of ordinary skill in the art have not been described in detail so as not to unnecessarily obscure the present application. As used herein, the terms “a” and “an” are intended to denote at least one of a particular element, the term “includes” means includes but not limited to, the term “including” means including but not limited to, and the term “based on”means based at least in part on.

[0035] As used herein, a “near-eye display device” may refer to any display device (e.g., an optical device) that may be in close proximity to a user’s eye. Accordingly, a near-eye display device may be a head-mounted display (HMD) device, such as a wearable eyewear, a wearable headset, and / or “smartglasses,” which may be used for interacting with virtual reality (VR) and / or augmented reality (AR), or any environment of real and virtual elements, such as a “metaverse” — all of which may be referred to herein as “VR / AR.” As used herein, a “wearable device” may refer to any portable electronic device that may be worn on any body part of a user and used to present audio and / or video content, control other devices, monitor bodily functions, and perform similar actions. As used herein, a “user” may refer to a user or wearer of a “near-eye display device” and / or a “wearable display.”

[0036] As referred to above, it may be beneficial for the various components in any near-eye display device to have a relatively small size and negligible weight for portability and user comfort — more specifically, it may be desirable to reduce the size, location, power / energy, and other requirements of the eye tracking system in any near- eye display device to, for example, increase the overall efficiency of the near-eye display device.

[0037] Continued progress in AR / VR system performance for near-eye display device requires eye tracking that works reliably and uniformly for all users, regardless of eye / face shapes, slippage, occlusion caused by eyelids and / or eyelashes, and / or the use of prescription lenses. Eye tracking systems for AR / VR system performance in near-eye display devices also need to operate within tight power envelopes while still providing an optimal user experience (UX). Accordingly, the tracking rates / performance achievable by camera-based eye tracking is typically limited by, for example, power consumption constraints.

[0038] According to examples of the present disclosure, Self-Mixing Interferometers (SMIs) are coupled with the waveguide of a near-eye display device to provide in-field sensing for eye tracking. In some examples, a group of eye tracking SMIs may be employed, where either (1 ) each SMI has its own in-coupler / out-coupler pair, or (2) the group of SMIs share one in-coupler and one out-coupler. In some examples, the group of eye tracking SMIs may be sparsely populated or in an array. In some examples, the in-coupler / out-coupler pairs for the eye tracking SMIs may be the same as, and / or separate from, any in-coupler / out-coupler pairs employed forillumination (such as for, e.g., camera-based eye tracking) and / or projection (such as for, e.g., AR / VR images).

[0039] According to some examples of the present disclosure, the waveguide may be designed to redirect the light projected by the eye tracking SMIs in a desired beam configuration. In some examples, the couplers for the eye tracking SMIs may be designed to achieve any desired angular orientation, polarization, and / or focusing power for each beam as it is projected into the eyebox. In some examples, the eye tracking SMIs may be combined with a camera-based or other eye tracking system in a sensor fusion network. In some examples, the in-field eye tracking SMIs may be combined with an in-field waveguide camera.

[0040] While some advantages and benefits of the present disclosure are discussed herein, there are additional benefits and advantages which would be apparent to one of ordinary skill in the art.NEAR-EYE DISPLAY DEVICE(S)

[0041] FIG. 1 illustrates a block diagram of an artificial reality system environment including a near-eye display device, according to an example. As used herein, a “near-eye display device” may refer to a device (e.g., an optical device) that may be in close proximity to a user’s eye. As used herein, “artificial reality” may refer to aspects of, among other things, a “metaverse” or an environment of real and virtual elements and may include use of technologies associated with virtual reality (VR), augmented reality (AR), and / or mixed reality (MR). As used herein a “user” may refer to a user or wearer of a “near-eye display device.”

[0042] As shown in FIG. 1 , an artificial reality system environment 100 may include a near-eye display device 120, an optional external imaging device 150, and an optional input / output interface 140, each of which may be coupled to an optional console 110. The optional console 110 may be optional in some instances where functions of the optional console 110 may be integrated into the near-eye display device 120. In some examples, the near-eye display device 120 may be implemented in any suitable form-factor, including a head-mounted display (HMD), a pair of glasses, or other similar wearable eyewear or device. In some examples, the near-eye display device 120 may include one or more rigid bodies, which may be rigidly or non-rigidly coupled to each other. In some examples, a rigid coupling between rigid bodies may cause the coupled rigid bodies to act as a single rigid entity, while in other examples,a non-rigid coupling between rigid bodies may allow the rigid bodies to move relative to each other. Specific examples of implementations of the near-eye display device 120 are described further below with respect to FIGS. 2A-2B and 3A-3B.

[0043] In some examples, the near-eye display device 120 may present content to a user, including, for example, audio / visual content, such as, e.g., virtual reality (VR), augmented reality (AR), and / or mixed reality (MR) content. In augmented reality (AR) and / or mixed reality (MR) examples, the near-eye display device 120 may combine images (and / or a see-through view) of a physical, real-world environment external to the near-eye display device 120 and artificial reality / digital content (e.g., computer-generated images, video, sound, etc.) to present an augmented reality (AR) or mixed reality (MR) environment for the user.

[0044] As shown in FIG. 1 , the near-eye display device 120 may include any one or more of one or more processors 121 , display electronics 122, display optics 124, one or more locators 126, one or more position sensors 128, an eye tracking unit 130, an inertial measurement unit (IMU) 132, a wireless communication sub-system 134, one or more outward projectors 172, and / or one or more inward projectors 173. In some examples, the near-eye display device 120 may include additional components; in other examples, the near-eye display device 120 may omit any one or more of the one or more locators 126, the one or more position sensors 128, the eye tracking unit 130, the inertial measurement unit (I MU) 132, the wireless communication sub-system 134, the one or more outward projectors 172, and / or the one or more inward projectors 173. As would be understood by one of ordinary skill in the art, various operational, electronic, communication (for, e.g., control signals), electrical and other such connections may or may not also be included between and among the components of the near-eye display device 120.

[0045] In some examples, the display electronics 122 may display or facilitate the display of images to the user according to data received from control electronics disposed in, for example, the near-eye display device 120, the optional console 110, the input / output interface 140, and / or a system connected by wireless or wired connection with the near-eye display device 120. In some examples, such electronics may include a virtual reality engine, such as, for example, the virtual reality engine 116 in the external console 110 described below, a virtual reality engine implemented, in part or in whole, in electronics in the near-eye display device 120, and / or a virtual reality engine implemented, in whole or in part, in an external system connected bythe wireless communication subsystem 134, etc. In some examples, the display electronics 122 may include one or more display panels, and may include and / or be operationally connected to the display optics 124. In some examples, the display electronics may include one or more of a liquid crystal display (LCD) and / or a lightemitting diode (LED) and may include any number of pixels to emit light of a predominant color such as red, green, blue, white, or yellow. In some examples, the display electronics 122 may display a three-dimensional (3D) image, e.g., using stereoscopic effects produced by two-dimensional panels, to create a subjective perception of image depth.

[0046] In some examples, the display electronics 122 may include and / or be operationally connected to the one or more outward projectors 172 and / or the one or more inward projectors 173; in some examples, the eye tracking unit 130 may also include and / or be operationally connected to the one or more inward projectors 173. As indicated by the striped lined box in FIG. 1 , there may be operational and / or other connections between and among the display electronics 122, the eye tracking unit 130, the one or more outward projectors 172, and / or the one or more inward projectors 173. As indicated above, such connections may also be included between and among these and other components of the near-eye display device 120; the possible connections indicated by the striped lined box in FIG. 1 are shown herein as they are germane to examples of the present disclosure.

[0047] In some examples, the one or more inward projectors 173 may, under the control of the display electronics 122, form an image in angular domain for direct observation by a viewer’s eye through a pupil. In some examples, the same or different one or more inward projectors 173 may, under the control of the eye tracking unit 130, project a fringe or other pattern on the eye (such as the inward projectors 310 of FIGS. 3A and 3B discussed below). As used herein, “eye tracking” may refer to determining an eye’s position or relative position, including orientation, location, and / or gaze of a user’s eye. In examples where at least some of the one or more inward projectors 173 may be used to project a fringe pattern on the eye, reflections from the projected pattern on the eye may be captured by a camera and analyzed (e.g., by the eye tracking unit 130 and / or the eye tracking module 118 in the optional console 110) to determine a position of the eye (the pupil), a gaze, etc. In other examples, the eye tracking unit 130 may capture reflected radio waves emitted by a miniature radar unit. These data associated with the eye may be used to determine or predict eye position,orientation, movement, location, and / or gaze.

[0048] In some examples, the one or more outward projectors 172 may, under the control of the display electronics 122, project a fringe or other pattern on the external environment (such as the outward projectors 315 of FIGS. 3A and 3B). In examples where at least some of the one or more outward projectors 172 may be used to project a fringe pattern on the external environment, reflections from the projected pattern on the external environment may be captured by a camera and analyzed to determine a position of objects in the external environment, distances between the user and objects and / or surfaces of the external environment, etc.

[0049] In some examples, a location of any of the one or more inward projectors 173 and / or the one or more outward projectors 172 may be adjusted to enable any number of design modifications. For example, in some instances, the one or more inward projectors 173 may be disposed in the near-eye display device 120 in front of the user’s eye (e.g., “front-mounted” placement). In a front-mounted placement, in some examples, the one or more inward projectors 173 under control of the display electronics 122 may be located away from a user’s eyes (e.g., “world-side”). In some examples, the near-eye display device 120 may utilize a front-mounted placement to propagate light and project an image on the user’s eye(s).

[0050] In some examples, the one or more outward and / or inward projectors 172 and / or 173 may employ a controllable light source (e.g., a laser) and a microelectromechanical system (ME MS) beam scanner to create a light field from, for example, a collimated light beam. In some examples, the light source of the one or more projectors 172 and / or 173 may include one or more of a liquid crystal display (LCD), a light emitting diode (LED) or micro-light emitting diode (mLED), an organic light emitting diode (OLED), an inorganic light emitting diode (ILED), an active-matrix organic light emitting diode (AMOLED), a transparent organic light emitting diode (TLED), any other suitable light source, and / or any combination thereof. In some examples, the one or more projectors may comprise a single electronic display or multiple electronic displays (e.g., one for each eye of the user).

[0051] In some examples, the display optics 124 may project, direct, and / or otherwise display image content optically and / or magnify image light received from the one or more inward projectors 173 (and / or otherwise created by the display electronics 122), correct optical errors associated with image light created and / or received from the external environment, and / or present the (corrected) image light to a user of thenear-eye display device 120. In some examples, the display optics 124 may include an optical element or any number of combinations of various optical elements as well as mechanical couplings to, for example, maintain relative spacing and orientation of the optical elements in the combination. In some examples, one or more optical elements in the display optics 124 may include an aperture, a Fresnel lens, a refractive lens, a reflective mirror, a diffractive element, a waveguide, a filter, or any other optical element suitable for affecting and / or otherwise manipulating light emitted from the one or more inward projectors 173 (and / or otherwise created by the display electronics 122). In some examples, one or more optical elements in the display optics 124 may have an optical coating, such as an anti-reflective coating, a reflective coating, a filtering coating, and / or a combination of different optical coatings.

[0052] In some examples, the display optics 124 may be used to combine the view of an environment external to the near-eye display device 120 and artificial reality content (e.g., computer-generated images) generated by, e.g., the virtual reality engine 1 16 in the console 110, and projected by, e.g., the one or more inward projectors 173 (and / or otherwise created by the display electronics 122). In such examples, the display optics 124 may augment images of a physical, real-world environment external to the near-eye display device 120 with generated and / or overlaid digital content (e.g., images, video, sound, etc.) projected by the one or more inward projectors 173 (and / or otherwise created by the display electronics 122) to present an augmented reality (AR) to a user.

[0053] In some examples, the display optics 124 may also be designed to correct one or more types of optical errors, such as two-dimensional optical errors, three-dimensional optical errors, or any combination thereof. Examples of two- dimensional errors may include barrel distortion, pincushion distortion, longitudinal chromatic aberration, and / or transverse chromatic aberration. Examples of three- dimensional errors may include spherical aberration, chromatic aberration field curvature, and astigmatism.

[0054] In some examples, the one or more locators 126 may be objects located in specific positions relative to one another and relative to a reference point on the near-eye display device 120. In some examples, the optional console 110 may identify the one or more locators 126 in images captured by the optional external imaging device 150 to determine the artificial reality headset’s position, orientation, or both. The one or more locators 126 may each be a light-emitting diode (LED), a corner cubereflector, a reflective marker, a type of light source that contrasts with an environment in which the near-eye display device 120 operates, or any combination thereof.

[0055] In some examples, the optional external imaging device 150 may include one or more cameras, one or more video cameras, any other device capable of capturing images including the one or more locators 126, or any combination thereof. The optional external imaging device 150 may detect light emitted or reflected from the one or more locators 126 in a field of view of the optional external imaging device 150.

[0056] In some examples, the one or more position sensors 128 may sense motion of the near-eye display device 120 and, in response, generate one or more measurement signals and / or data. Examples of the one or more position sensors 128 may include any number of accelerometers, gyroscopes, magnetometers, and / or other motion-detecting or error-correcting sensors, or any combination thereof.

[0057] In some examples, the inertial measurement unit (IMU) 132 may be an electronic device that generates fast calibration data based on measurement signals received from the one or more position sensors 128. The one or more position sensors 128 may be located external to the inertial measurement unit (IMU) 132, internal to the inertial measurement unit (IMU) 132, or any combination thereof. Based on the one or more measurement signals from the one or more position sensors 128, the inertial measurement unit (IMU) 132 may generate fast calibration data indicating an estimated position of the near-eye display device 120. Estimated positions may be of a reference point on the near-eye display device 120, and estimated positions may be, for example, relative to an initial position of the near-eye display device 120, relative to other objects in an external environment, relative to virtual objects in an artificial environment or augmented / mixed reality, etc., as would be understood by one of ordinary skill in the art. For example, the inertial measurement unit (IMU) 132 may integrate measurement signals received from accelerometers over time to estimate a velocity vector and integrate the velocity vector over time to determine an estimated position of the near-eye display device 120. Alternatively, the inertial measurement unit (IMU) 132 may provide the sampled measurement signals to the optional console 110, which may determine the fast calibration data.

[0058] In some examples, the wireless communication subsystem 134 may include an ultra-wide band (UWB) transceiver. Ultra-wide band (UWB) wireless communication technology is used for short-range, fast, and secure data transmissionenvironments. Ultra-wide band (UWB) wireless communication technology provides high transmission speed, low power consumption, and large bandwidth, in addition to the ability to co-exist with other wireless transmission technologies. The ultra-wide band (UWB) transceiver may be used to detect another user (head-mounted display (HMD) device) within range of communication and within an angle-of-arrival (AoA), then establish line-of-sight (LoS) communication between the two users. The communication may be in audio mode only or in audio / video mode. In other examples, the ultra-wide band (UWB) transceiver may be used to detect the other user, but a different communication technology (transceiver) such as WiFi or Bluetooth Low Energy (BLE) may be used to facilitate the line-of-sight (LoS) communication. In some cases, multiple wireless communication transceivers may be available and one with lowest power consumption, highest communication quality (e.g., based on interfering signals), or user choice may be used. For example, the communication technology may be selected based on a lowest power consumption for a given range.

[0059] In some examples, the one or more processors 121 may be the control electronics (which may include, e.g., an operating system) for the near-eye display device 120. The one or more processors 121 may be employed for controlling one or more of the display electronics 122, the display optics 124, the one or more locators 126, the one or more position sensors 128, the eye tracking unit 130, the inertial measurement unit (IMU) 132, the wireless communication sub-system 134, the one or more outward projectors 172, and / or the one or more inward projectors 173, according to the present disclosure. The one or more processors 121 may be implemented, in whole or in part, as a separate physical component in the near-eye display device 120, as distributed among and / or integrated into one or more components of the near-eye display device 120 (such as, e.g., the display electronics 122), and / or externally to near-eye display device 120, such as being implemented / integrated in, for example, the input / output interface 140 and / or the console 110 (e.g., the eye tracking module 118, the headset tracking module 114, the virtual reality engine 116, the application store 112, etc.), and / or in another external system connected by, for example, the wireless communication subsystem 134. In some examples, the one or more processors 121 of the near-eye display device 120 may receive input, store, and process data, and / or control the components of the near-eye display device 120 in accordance with received input and / or stored / processed data in order to maintain optimal operating conditions of one or more components in the near-eye displaydevice 120.

[0060] In some examples, the one or more processors 121 , any control electronics, and / or any of the other components of the near-eye display device 120 may be implemented in and / or by any number of processors executing instructions stored on any number of non-transitory computer-readable storage media (not shown) disposed on / in and / or communicatively linked to the near-eye display device 120. The one or more processors 121 may include multiple processing units executing instructions in parallel. The non-transitory computer-readable storage medium / media may be any memory, such as a hard disk drive, a removable memory, or a solid-state drive (e.g., flash memory or dynamic random access memory (DRAM)). In some examples, the one or more processors 121 in the near-eye display device 120 may perform one or more functions; in some examples, one or more non-transitory computer-readable storage media in the near-eye display device 120 may store instructions that, when executed by the one or more processors 121 , cause the one or more processors 121 to perform any of the functions described herein and / or to control any of the components described herein. In some examples, functions such as those described below in reference to the optional console 110 (e.g., eye tracking, headset tracking, and the generation of virtual reality images) may be performed by the one or more processors 121 integrated with and / or wired / wirelessly connected to the near-eye display device 120.

[0061] In some examples, the input / output interface 140 may be a device that allows a user to send action requests to the optional console 1 10 and / or the near-eye display device 120. As used herein, an “action request” may be a request to perform a particular action. For example, an action request may be to start or to end an application or to perform a particular action within the application. The input / output interface 140 may include one or more input devices. Example input devices may include a keyboard, a mouse, a game controller, a glove, a button, a touch screen, or any other suitable device for receiving action requests and communicating the received action requests to the optional console 110. In some examples, an action request received by the input / output interface 140 may be communicated to the optional console 110 and / or the near-eye display device 120, either or both of which may perform an action corresponding to the requested action.

[0062] In some examples, the optional console 110 may provide content to the near-eye display device 120 for presentation to the user in accordance withinformation received from one or more of the near-eye display device 120, the input / output interface 140, and / or the external imaging device 150. For example, as shown in the example of FIG. 1 , the optional console 110 may include an application store 112, a headset tracking module 114, a virtual reality engine 116, and an eye tracking module 118. In some examples, the optional console 110 may include different or additional modules than those described herein, and the functions described further below may be distributed among the components of the optional console 110 in a different manner than is described here (or may be distributed, in part or whole, in one or more components in the near-eye display device 120). It should be appreciated that the optional console 110 may or may not be needed, or the optional console 110 may be integrated, in whole or in part, with the input / output interface 140 and / or the near-eye display device 120, or the optional console 1 10 may be separate from the input / output interface 140 and / or the near-eye display device 120. In some examples, the optional console 1 10 may include a processor and a non-transitory computer-readable storage medium storing instructions executable by the processor (including, for example, the application store 1 12).

[0063] In some examples, the application store 112 may store one or more applications for execution by one or more processors in at least one of the optional console 110, the near-eye display device 120, the input / output interface 140, and / or the optional external imaging device 150. An application may include a group of instructions that, when executed by a processor, generates content for presentation to the user. Examples of the applications may include gaming applications, conferencing applications, video playback application, or other suitable applications.

[0064] In some examples, the virtual reality engine 116 may execute applications within the artificial reality system environment 100 and receive position / acceleration / velocity information of the near-eye display device 120, predicted future positions of the near-eye display device 120, or any combination thereof from the headset tracking module 114. In some examples, the virtual reality engine 116 may also receive estimated eye position and orientation information from the eye tracking module 118. Based on the received information, the virtual reality engine 116 may determine content including, e.g., virtual reality images, to provide to the near- eye display device 120 for presentation to the user.

[0065] In some examples, the eye tracking module 118, which may be implemented as a processor, may receive eye tracking data from the eye tracking unit130 and determine the position of the user’s eye based on the eye tracking data. In some examples, the position of the eye may include an eye’s orientation, location, or both relative to the near-eye display device 120 or any element thereof. Accordingly, in these examples, because the eye’s axes of rotation change as a function of the eye’s location in its socket, determining the eye’s location in its socket may allow the eye tracking module 1 18 to more accurately determine the eye’s orientation.

[0066] Generally speaking, any one or more of the components and / or functionalities described in reference to any of the drawings / figures herein may be implemented by hardware, software, and / or any combination thereof, according to examples of the present disclosure. In some examples, the components and / or functionalities may be implemented by at least one of any type of application, program, library, script, task, service, process, or any type or form of executable instructions executed on hardware such as circuitry that may include digital and / or analog elements (e.g., one or more transistors, logic gates, registers, memory devices, resistive elements, conductive elements, capacitive elements, and / or the like, as would be understood by one of ordinary skill in the art). In some examples, the hardware and data processing components used to implement the various processes, operations, logic, and circuitry described in connection with the examples described herein may be implemented with a general purpose single- and / or multi-chip processor, a single- and / or multi-core processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic, discrete hardware components, and / or any combination thereof suitable to perform the functions described herein. A general purpose processor may be any conventional processor, microprocessor, controller, microcontroller, and / or state machine. In some examples, the memory / storage may include one or more components (e.g., random access memory (RAM), read-only memory (ROM), flash or solid state memory, hard disk storage, etc.) for storing data and / or computer-executable instructions for completing and / or facilitating the processing and storage functions described herein. In some examples, the memory / storage may be volatile and / or non-volatile memory, and may include database components, object code components, script components, or any other type of information structure suitable for implementing the various activities and storage functions described herein.

[0067] FIGS. 2A and 2B illustrate a front prospective view and a backprospective view, respectively, of a near-eye display device in the form of a headmounted display (HMD) device 200 which may be implemented with an inward-facing and / or an outward-facing projection system to which examples of the present disclosure may be applied. In some examples, the head-mounted display (HMD) device 200 may be a specific implementation of the near-eye display 120 of FIG. 1 , and may be configured to operate as a virtual reality (VR) system, an augmented reality (AR) system, a mixed reality (MR) system, and / or as part of any such system that uses displays or wearables, or any combination thereof. In some examples, the head-mounted display (HMD) device 200 may include a display 210, a body 220 and a head strap 230. In some examples, the head-mounted display (HMD) device 200 may include additional, fewer, and / or different components than shown and / or described in reference to FIGS. 2A-2B.

[0068] FIG. 2A is a frontal prospective view 200A showing a front side 225, a bottom side 223, and a right side 229 of the body 220, as well as the display 210 and the head strap 230 of the head-mounted display (HMD) device 200. FIG. 2B is a bottom rear prospective view 200B showing the bottom side 223, the front side 225, and a left side 227 of the body 220, as well as the display 210 and the head strap 230 of the head-mounted display (HMD) device 200. In some examples, the head strap 230 may have an adjustable or extendible length. In particular, in some examples, there may be a sufficient space between the body 220 and the head strap 230 of the head-mounted display (HMD) device 200 for allowing a user to mount the headmounted display (HMD) device 200 onto the user’s head. For example, the length of the head strap 230 may be adjustable to accommodate a range of user head sizes.

[0069] In some examples, the head-mounted display (HMD) device 200 (including, e.g., the display 210) in FIGS. 2A-2B may include any number of processors, display electronics, and / or display optics similar to the one or more processors 121 , the display electronics 122, and the display optics 124 described in reference to FIG. 1. In some examples, the display electronics and display optics of the head-mounted display (HMD) device 200 may display and / or facilitate the display of media or other digital content including virtual and / or augmented views of a physical, real-world environment with computer-generated elements. Examples of the media or digital content presented by the head-mounted display (HMD) device 200 may include images (e.g., two-dimensional (2D) or three-dimensional (3D) images), videos (e.g., 2D or 3D videos), audio, or any combination thereof. In some examples, the displayelectronics may display a three-dimensional (3D) image, e.g., using stereoscopic effects produced by two-dimensional panels, to create a subjective perception of image depth. In some examples, the display optics in the head-mounted display (HMD) device 200 may include a single optical element or any number of combinations of various optical elements, such as waveguides, gratings, optical lenses, optical couplers, mirrors, etc., as well as mechanical couplings to maintain relative spacing and orientation of the optical elements in the combination, such as are described above in reference to the display optics 124 in FIG. 1 .

[0070] In some examples, the head-mounted display (HMD) device 200 in FIGS. 2A-2B may include one or more inward / outward projectors, similar to the one or more inward projectors 173 and / or one or more outward projectors 172 of FIG. 1. In some examples, the one or more inward projectors of the head-mounted display (HMD) device 200 may project an image for direct observation by the user’s eye and / or project a fringe or other pattern on the eye. In some examples, the one or more outward projectors of the head-mounted display (HMD) device 200 may project a fringe or other pattern on the external environment and / or objects / surfaces within the external environment in order to, for example, perform 3-dimensional (3D) mapping of the external environment. In some examples, the one or more inward / outward projectors of the head-mounted display (HMD) device 200 may include one or more of a liquid crystal display (LCD) and / or a light-emitting diode (LED); more specifically, the one or more inward / outward projectors of the head-mounted display (HMD) device 200 may include, e.g., one or more of a liquid crystal display (LCD), a light emitting diode (LED) or micro-light emitting diode (mLED), an organic light emitting diode (OLED), an inorganic light emitting diode (ILED), an active-matrix organic light emitting diode (AMOLED), a transparent organic light emitting diode (TLED), any other suitable light source, and / or any combination thereof. It should be appreciated that in some examples, the inward projectors of the head-mounted display (HMD) device 200 may be placed near and / or closer to a user’s eye (e.g., “eye-side”). It should be appreciated that, in some instances, utilizing a back-mounted inward projector may help to reduce size or bulkiness of any required housing required for a display system, which may also result in a significant improvement in user experience for a user.

[0071] In some examples, the head-mounted display (HMD) device 200 may also include an eye tracking system, one or more locators, one or more position sensors, and an inertial measurement unit (IMU), similar to the eye tracking unit 130,the one or more locators 126, the one or more position sensors 128, and the inertial measurement unit (I MU) 132, respectively, described in reference to FIG. 1. In some examples, the head-mounted display (HMD) device 100 may include various other sensors, such as depth sensors, motion sensors, image sensors, light sensors, and / or the like. Some of these sensors may sense any number of structured or unstructured light patterns projected by the one or more inward / outward projectors of the headmounted display (HMD) device 200 for any number of purposes, including, e.g., sensing, eye tracking, and / or the creation of virtual reality (VR) content.

[0072] In some examples, the head-mounted display (HMD) device 200 may include and / or be operably connected to a virtual reality engine (not shown), similar to the virtual reality engine 116 described in reference to FIG. 1 , that may execute applications within the head-mounted display (HMD) device 200 and receive depth information, position information, acceleration information, velocity information, predicted future positions, or any combination thereof of the head-mounted display (HMD) device 200 from the various sensors. In some examples, the information received by the virtual reality engine may be used for producing a signal (e.g., display instructions) to the one or more display assemblies. In some examples, the headmounted display (HMD) device 200 may include locators (not shown), similar to the one or more locators 126 described in reference to FIG. 1 , which may be located in fixed positions on the body 220 of the head-mounted display (HMD) device 200 relative to one another and relative to a reference point. Each of the locators may emit light that is detectable by an external imaging device. This may be useful for the purposes of head tracking or other movement / orientation. It should be appreciated that other elements or components may also be used in addition or in lieu of such locators.

[0073] As stated above, the head-mounted display (HMD) device 200 may include additional, fewer, and / or different components than shown and / or described in reference to FIGS. 2A-2B. In some examples, the head-mounted display (HMD) device 200 may include an input / output interface (similar to the input / output interface 140 in FIG. 1 ), a console (similar to the console 110 described in reference to FIG. 1 ), and / or a camera to capture images or videos of the user’s environment to present the user with, e.g., augmented reality (AR)Zvirtual reality (VR) content. In some examples, the head-mounted display (HMD) device 200 may include one or more cameras to capture reflections of patterns projected by the one or more inward / outward projectors.

[0074] FIGS. 3A and 3B illustrate a perspective view 300A and a top view 300B, respectively, of a near-eye display device 300 in the form of a pair of glasses having both an inward-facing and an outward-facing projection systems to which examples of the present disclosure may be applied. In some examples, the near-eye display device 300 may be a specific implementation of the near-eye display device 120 of FIG. 1 , and may be configured to operate as a virtual reality (VR) system, an augmented reality (AR) system, a mixed reality (MR) system, and / or as part of any such system that uses displays or wearables, or any combination thereof. As shown in FIGS. 3A- 3B, the near-eye display device 300 may include a frame 305, one or more outward pattern projectors 310, one or more eye tracking projectors 315 (which effectively operate as inward pattern projectors), an outward-facing camera(s) 320, an eye tracking camera(s) 325, and a display 390.

[0075] As shown in FIGS. 3A-3B, the near-eye display device 300 may include an inward-facing imaging / projection system, comprising the one or more eye tracking projectors 315 (i.e., inward pattern projectors) and the eye tracking camera(s) 325, and an outward-facing imaging / projection system, comprising the one or more outward pattern projectors 310 and the outward-facing camera(s) 320. In some examples, the inward-facing imaging / projection system of the near-eye display device 300 may be an eye tracking system, where the one or more eye tracking projectors 315 project a pattern directly on the user’s eye(s) and the eye tracking camera(s) 325 captures one or more reflections of the projected pattern on the user’s eye(s), and the eye tracking system uses the captured reflections to track the user’s eye(s). In some examples, the one or more eye tracking projectors 315 may be disposed on the temple arms of the frame 305 of the near-eye display device 300, and may project one or more patterns on eye lens of the near-eye display device 300, which reflects those one or more patterns onto the user’s eye 355. In some examples, the inner surface of the eye lens may be coated with a reflective surface, fabricated with a reflective surface, and / or covered by a metasurface or other type of nanostructure which may be suitably employed for the re-direction of the light projected by the one or more eye tracking projectors 315, as would be understood by one of ordinary skill in the art. In such examples, the inner surface may create the one or more patterns which are projected onto the user’s eye 355, either alone or in combination with the one or more eye tracking projectors 315. In other words, in some examples, the one or more eye tracking projectors 315 may project unstructured light and the inner surface re-directedthe light onto the user’s eye may provide the one or more patterns which may be used for eye tracking. In some examples, the eye tracking camera(s) 325 may be a single photon avalanche diode (SPAD) sensor. In some examples, the one or more eye tracking projectors 315 may project a pattern such as, for example, a structured image (e.g., a fringe pattern) projected onto the eye by a micro-electromechanical system (MEMS) based scanner reflecting light from a light source (e.g., a laser).

[0076] As shown in FIG. 3B, in some examples, the outward-facing imaging / projection system of the near-eye display device 300 may include the one or more outward pattern projectors 310, which project a pattern directly on an external environment 350 and / or one or more objects / surfaces in the external environment 350, and the outward-facing camera(s) 320, which captures one or more reflections of the projected pattern on the one or more objects / surfaces or all or part of the entire external environment 350. In some examples, such an outward-facing imaging / projection system may serve a variety of purposes, including, but not limited to, profilometry, determining surface patterns / structures of objects in the external environment 350, determining distances from the user to one or more objects / surfaces in the external environment 350, determining relative positions of one or more objects / surfaces to each other in the external environment 350, determining relative velocities of one or more objects / surfaces in the external environment 350, etc., as would be understood by one of ordinary skill in the art. In some examples, the outward-facing imaging / projection system of the near-eye display device 300 may also be employed to capture images of the external environment 350. In such examples, the captured images may be processed, for example, by a virtual reality engine to add virtual objects to the captured images or modify physical objects in the captured images, and the processed images may be displayed to the user by the display 390 for augmented reality (AR) and / or mixed reality (MR) applications.

[0077] In some examples, the display 390 may include, in whole or in part, one or more processors, display electronics, and / or display optics similar to the one or more processors 121 , the display electronics 122, and the display optics 124 in FIG. 1 , and may be configured to present media or other content to a user, including, e.g., virtual reality (\ / R), augmented reality (AR) system, and / or mixed reality (MR) content. In some examples, the display 390 may include any number of light sources, such as, e.g., a liquid crystal display (LCD) display panel, a light-emitting diode (LED) display panel, or an optical display panel (e.g., a waveguide display assembly), etc., and anynumber of optical components, such as waveguides, gratings, lenses, mirrors, etc., as would be understood by one of ordinary skill in the art.

[0078] As shown in FIG. 3B, in some examples, the display 390 of the near-eye display device 300 may include optics 391 and a waveguide 393, which may be coupled to a projector (such as, e.g., the one or more inward projectors 173 of FIG. 1 ). In some examples, the display 390 may combine the view of the external environment 350 and artificial reality content (e.g., computer-generated images). In some examples, light from the external environment 350 may traverse a “see-through” region of the waveguide 393 in the display 390 to reach a user’s eye 355 (located somewhere within an eye box), while images are also projected for the user to see as part of an augmented reality (AR) display and / or a mixed reality (MR) display.

[0079] In such examples, the light of images projected by the projector may be coupled into a transparent substrate of the waveguide 393, propagate within the waveguide 393, be coupled with light from the user’s actual environment, and be directed out of the waveguide 393 at one or more locations towards a user’s eye 355 located within the eye box. In such examples, the waveguide 393 may be geometric, reflective, refractive, polarized, diffractive, and / or holographic, as would be understood of one of ordinary skill in the art, and may use any one or more of macro-optics, microoptics, and / or nano optics (such as, e.g., metalenses and / or metasurfaces). In some examples, the optics 391 of the display 390 may include optical polymers, plastic, glass, transparent wafers (e.g., Silicon Carbide (SiC) wafers), amorphous silicon, Silicon Oxide (SiC>2), Silicon Nitride (SiN), Titanium Oxide (TiO), optical nylon, carbonpolymers, and / or any other transparent materials used for such a purpose, as would be understood by one of ordinary skill in the art.

[0080] In some examples, the near-eye display device 300 may further include various sensors on or within a frame 305, such as, e.g., any number of depth sensors, motion sensors, position sensors, inertial sensors, and / or ambient light sensors. In some examples, the various sensors may include any number of image sensors configured to generate image data representing different fields of views in one or more different directions (which may or may not include the outward-facing camera(s) 320). In some examples, the various sensors may be used as input devices to control or influence the displayed content of the near-eye display device 300, and / or to provide an interactive virtual reality (VR), augmented reality (AR), and / or mixed reality (MR) experience to a user of the near-eye display device 300. In some examples, thevarious sensors may also be used for stereoscopic imaging or other similar application.

[0081] In some examples, the near-eye display device 300 may further include one or more illuminators to project light into a physical environment (which may or may not include, e.g., the outward pattern projector(s) 310). The projected light may be associated with different frequency bands (e.g., visible light, infra-red light, ultra-violet light, etc.), and may serve various purposes. In some examples, the one or more illuminators may be used as locators, such as the one or more locators 126 described above with respect to FIG. 1 . In such examples, the near-eye display device 300 may also include an image capture unit (which may or may not include the outward-facing camera(s) 320 and / or the external imaging device 150 of FIG. 1 ), which may capture images of the physical environment in the field of view. In some instances, the captured images may be processed, for example, by a virtual reality engine (such as, e.g., the virtual reality engine 116 of FIG. 1) to add virtual objects to the captured images or modify physical objects in the captured images, and the processed images may be displayed to the user by the display 390 for augmented reality (AR) and / or mixed reality (MR) applications.

[0082] In some examples, a majority of electronic components of the near-eye display device 300 in the form of a pair of glasses may be included in the frame 305 of the glasses (e.g., a top bar, a bridge, a rim, a lens, etc.). Examples of such electronic components included in the frame 305 include, but are not limited to, a camera, a sensor, a projector, a speaker, a battery, a microphone, and a battery management unit (BMU). In some examples, a battery management unit (BMU) may be an electronic system that may be used to manage charging and discharging of a battery (e.g., a lead acid battery). In some examples, the battery management unit (BMU) may, among other things, monitor a state of the battery, determine and report data associated with the battery, and provide environmental control(s) for the battery. In some examples, the temples 306 may be provided with a tapering profile, based on design considerations for the specific implementation. In such examples, the tapered temples may be utilized to house various electronic components. For example, in some cases, a microphone or speaker may often be placed towards a rear of a temple arm, near a user’s ear, and as such, in many cases, a battery may be more likely to be placed near a front of the temple arm.

[0083] In FIG. 3B, an eye tracking system (such as that described in referenceto eye tracking unit 130, the eye tracking module 118, and the inward projector(s) 173 of FIG. 1 ) may be implemented by the eye tracking projector(s) 315, which project patterns and / or other suitable lighting for performing eye tracking upon the user’s eye 355, the eye tracking camera(s) 325, which receive reflections of the light of the eye tracking projector(s) 315 from the user’s eye 355, and a controller (or controllers) 317, which process the reflections received by the eye tracking camera(s) 325 to perform eye tracking. In some examples, the controller 317 may be similar to the one or more processor(s) 121 in FIG. 1 (and thus may perform a wide variety of functions for the near-eye display device 300), other processor(s) which perform several tasks, and / or a processor(s) dedicated to performing eye tracking.

[0084] In some examples, the controller 317 for performing eye tracking may be communicatively connected with a memory, which may be at least one non- transitory computer-readable storage medium storing instructions executable by the controller 317. The controller 317 may include multiple processing units, and those multiple processing units may further execute instructions in parallel. The at least one non-transitory computer-readable storage medium may be any memory, such as a hard disk drive, a removable memory, or a solid-state drive (e g., flash memory or dynamic random access memory (DRAM)). In various examples, the controller 317 may be further subdivided into multiple devices (for example, the functions of the controller 317 may be separated among various components, such as a digital signal processing (DSP) chip for eye tracking analysis as well as a Central Processing Unit (CPU) for controlling, e.g., the eye tracking projector(s) 315).IN-FIELD SMI WAVEGUIDE SYSTEMS FOR EYE TRACKING

[0085] As mentioned above, it may be desirable to reduce the size, location, power / energy, accuracy, speed, and other requirements of the eye tracking system in any near-eye display device 100, such as, e.g., the head-mounted display (HMD) device 200 in FIGS. 2A-2B and / or the near-eye display device 300 in the form of a pair of glasses in FIGS. 3A-3B, to increase its overall efficiency. It may be beneficial for eye tracking systems to reduce power consumption while also providing an optimal user experience (UX), by working consistently, comfortably, and reliably for all users, regardless of eye / face shape, slippage, occlusion caused by eyelids and / or eyelashes, prescription lenses, and / or any other differences in user’s eyes, faces, and / or personal usage of the near-eye display device 100, as would be understood by one of ordinaryskill in the art.

[0086] Current eye tracking systems often use light projectors, such as groups of LEDs, image-based cameras, and processing units for image processing of data received through the cameras — such as, for example, eye tracking projector(s) 315, eye tracking camera(s) 325, and controller(s) 317 in FIG. 3B. In such systems, the eye tracking cameras are often pointed at difficult angles towards the eye 355, causing poor visibility of the eye. For example, if the eye tracking cameras are disposed at or near the hinges at the upper part of the frame 305, the eye tracking cameras may be pointed at a rather steep angle towards the eye, thus limiting visibility of the eye by being frequently blocked by, e.g., eyebrows, eye lids, eye lashes, and / or other occlusions. Performance may be further limited by poor visibility of the eye, distortion by / from prescription lenses (particularly in certain constructions of near-eye display devices, where for example, the projecting light and / or reflecting light from the eye, as well as the light from the external environment 350, may be strongly impacted by travelling through, in part or whole, the prescription lenses), and power constraints where, for example, eye tracking should use relatively lower power consumption than the components providing and projecting the VR / AR images).

[0087] As mentioned above, the off-axis lighting and / or sensing for eye tracking may be significantly challenged by occlusions, such as, e.g., eyelashes. In-field sensing technology, where the illuminating and / or sensing components for eye tracking are somehow integrated in, near, and / or with, the eye lenses and / or waveguide of the near-eye display device 100 may provide one possible solution for providing intensity, depth, velocity, and other useful information for eye tracking. However, current sparse-based laser solutions, such as, e.g., direct lamination of miniature lasers into and / or on the waveguide / lens, may have integration challenges, including, but not limited to:• costly and time-consuming integration of lasers, lenses, and metal traces on substrate;• increased integration complexity and yield issues when integrating multiple lasers on substrate; and• increased conspicuity with in-field direct lamination of lasers and metal substrate.

[0088] In examples according to the present disclosure, one or more Self-Mixing Interferometers (SMIs) may provide in-field sensing for eye tracking by both projecting light into, and receiving reflected light from the eye back through, the waveguide of the near-eye display device. As an example, the eye tracking SMIs may be disposed to project into, and receive reflected light back through, the waveguide 393 in the near-eye display device 300 of FIG. 3B.

[0089] According to examples of the present disclosure, the eye tracking SMIs may be placed in the frame / enclosure / body of the near-eye display device as an array and / or one or more modules pointing into the waveguide / lens of the near-eye display device. As examples, the eye tracking SMIs may be placed in the frame 305 pointing into the waveguide 393 of the near-eye display device 300 in FIGS. 3A-3B or the SMIs may be placed in the body 220 and / or one or more of the sides 223, 225, 227, and / or 229 pointing into the waveguide of the display 210 of the near-eye display device 200 in FIGS. 2A-2B.

[0090] According to examples of the present disclosure, each eye tracking SMI has its own in-coupler / out-coupler pair, and / or the eye tracking SMIs share one incoupler and one out-coupler as a group, or several different groups. In some examples, the in-coupler / out-coupler pairs for the eye tracking SMIs may be the same as, and / or separate from, any in-coupler / out-coupler pairs employed for illumination (such as for, e.g., camera-based eye tracking) and / or projection (such as for, e.g., AR / VR images).

[0091] According to examples of the present disclosure, the eye-side couplers of the eye tracking SMIs may be sparsely populated or disposed in an array. In some examples, the couplers (and possibly the waveguide) may be designed to redirect the light projected by the eye tracking SMIs in a desired beam configuration. In some examples, the couplers for the eye tracking SMIs (and possibly the waveguide) may be designed to achieve any desired angular orientation, polarization, and / or focusing powerfor each beam as it is projected into the eyebox. In some examples, one, more, and / or all of the projected beams may be collimated, diverging, and / or focused into the eyebox. In some examples, a single SMI module may produce multiple output beams, which might require more sophisticated algorithms, as would be understood by one of ordinary skill in the art. In some examples, a single electronic readout may be provided as the output from an array of eye tracking SMIs. In some examples, the eye tracking SMIs may be combined with a camera-based or other eye tracking system in a sensor fusion network. In some examples, the in-field eye tracking SMIsmay be combined with an in-field waveguide camera. In other examples, the eye tracking SMIs may be employed alone, without the assistance of any imaging sensors and / or any other eye tracking systems.

[0092] According to examples of the present disclosure, using the eye tracking SMIs for non-image-based sensing may be typically more accurate, faster, and more power efficient than camera-based eye tracking (i.e., image-based eye tracking) when detecting changes in the angular velocity of the user’s eye. In some examples, SMIbased non-image eye tracking may be able to accurately track / fix eye movement (i.e., angular velocity) to a range of about ! degree to about a z degree per second within less than 100 microseconds. In some examples, the non-image-based eye tracking provided by one or more SMI eye tracking sensors (e.g., rotational movement) may be combined or otherwise integrated with image-based and / or camera-based eye tracking (e.g., gaze vector sensor or gaze position sensing) with a slower tracking / fixing / determining rate to obtain complete gaze tracking at a higher accuracy and speed. In-field SMIs may also provide depth information which may contribute to more accurate 3D pupil tracking. In some examples, an array of SMIs may be employed to detect the pupil size, center, and / or shape (possibly in 3D) without the assistance of any additional imaging sensors.

[0093] According to examples of the present disclosure, the eye tracking SMIs may be disposed in any location, with any orientation, in any portion of the near-eye display device, provided that suitable optical elements are employed to in-couple the projected light from the eye tracking SMIs appropriately into the waveguide and / or to guide / direct reflected light from the waveguide back to the eye tracking SMIs. Accordingly, in some examples, the eye tracking SMIs may be positioned such that the SMIs do not directly beam and receive light from any side or surface of the waveguide, but rather one or more optical elements, such as prisms, diffraction gratings, lenses, spaceplates, and / or any other suitable optical elements may be employed to re-direct the light appropriately into the waveguide from, and / or appropriately out of the waveguide to, the eye tracking SMIs.

[0094] According to examples of the present disclosure, the eye tracking SMIs may be disposed with, near, and / or merely work in conjunction with (but not located near), light sources / projectors providing AR / VR images through the waveguide and / or providing illumination for camera-based eye tracking. In some examples, the eye tracking SMIs may be paired in the same location as the AR / VR and / or eye trackingprojectors. In such examples, the eye-tracking SMIs may be simply co-located with the AR / VR and / or eye tracking projectors; in other examples, the eye-tracking SMIs and the AR / VR and / or eye tracking projectors may be in the same module(s) and / or components(s); in other examples, the eye-tracking SMIs and the AR / VR and / or eye tracking projectors may be fabricated by the same process, from the same substrate, and thereby be disposed in / on the same one or more integrated circuits / chips. In some examples, the eye tracking SMIs may be located differently and / or operate completely differently than the ARA / R and / or eye tracking projectors.

[0095] According to examples of the present disclosure, the eye tracking SMIs may have in-coupler(s) and / or out-coupler(s) to the waveguide separate from, and / or in addition to, the in-coupler(s) and / or out-coupler(s) for any AR / VR and / or eye tracking projection system. Accordingly, in some examples, the eye tracking SMIs may also use the same means for in- and out-coupling as are employed by the AR / VR display system (for example, if the AR / VR display system employs reflective out- couplers, reflective coatings may be disposed in suitable locations on the waveguide to enable the SMI beams from the eye tracking SMIs to use the same reflective out- couplers). In some examples, each of the eye tracking SMIs may have its own in- coupler / out-coupler pair; in some examples, all, or one or more groupings, of the eye tracking SMIs may have a single in-coupler / out-coupler pair. In some examples, the eye tracking SMIs may be sparsely populated and / or disposed in an array. As would be understood by one of ordinary skill in the art, each coupler may operate as either an in-coupler or an out-coupler, depending on what light path is being considered, i.e., (i) the light projected from the eye tracking SMIs through the waveguide onto the user’s eye, or (ii) the light reflected back from the eye through the waveguide to the eye tracking SMIs.

[0096] According to examples of the present disclosure, the couplers to the waveguide employed by the eye tracking SMIs may include any type or system of suitable optical coupling, such as, for example, refractive elements, reflective elements, diffractive gratings (such as, e.g. Polarization Volumetric Hologram-based (PVH) gratings, Surface Relief Gratings (SRGs), and / or Volume Bragg Gratings (VBGs), as discussed further below), nano-optics (including, e.g., metalenses and metasurfaces), micro-structures (including those fabricated using 3D printing), surface coatings, lithographically-created layers within the waveguide, and / or any other suitable technique, technology, layer, coating, and / or material feasible and / or possibleeither presently or in the future, as would be understood by one of ordinary skill in the art. In some examples, the in-coupler / out-coupler pairs may be designed to couple with / between each other, and the coupling surfaces are fabricated within manufacturing / placement tolerances, whereby the k-vectors may be maintained between them with substantially little or no dispersion. In some examples, the couplers may be laminated on the inner surface and / or outer surface of the waveguide.

[0097] In some examples, putting eye tracking SMIs in-field may be relatively inexpensive (for example, it may be as low as $5-10 for the diffractive couplers and the waveguide), as there is no need for a high index substrate, so a lower-cost, lower- weight polymer may be employed, and / or a surface and / or coating laminated directly on the waveguide. Moreover, although beam roll-off may still exist (because different wavelengths transmitted through the waveguide may have different Total Internal Reflection (TIR) angles), it may not substantially affect the spatial resolution of the system because the wavelength sweeping range of the eye tracking SMIs is not substantial enough (in this case).

[0098] In some examples, the couplers and the waveguide are constructed / fabricated / designed to redirect the light from the eye tracking SMIs in a desired beam configuration to the eyebox for the user’s eye. In some examples, the couplers may be designed to add any desired angular orientation, polarization, and / or focusing power to the beams exiting the waveguide for the eyebox. In some examples, a sensor fusion framework may be employed, where the eye tracking SMIs may be combined with camera-based or other eye tracking systems. In some examples, a single eye tracking SMI may project a beam which is split into a multitude of beams. In such examples, the multitude of beams may or may not be distinguished from each other in detection, but the combination of the multitude of beams may be used in an eye tracking algorithm.

[0099] According to examples of the present disclosure, the eye tracking SMIs may operate in different modes, and use different means to perform eye tracking (e.g., to measure depth and / or velocity). In some examples, the eye tracking SMIs may employ coherent sensing; in other examples, the eye tracking SMIs may employ Frequency Modulated Continuous Wave (FMCW) sensing. In some examples, the eye tracking SMIs may be employed with a camera-based eye tracking system.

[0100] According to examples of the present disclosure, the eye tracking SMIs may be Vertical Cavity Surface Emitting Lasers (VCSELs) disposed in the frame of thenear-eye display device projecting into the waveguide through in-couplers. In some examples, the eye tracking SMI / VCSELs may be disposed together with other VCSELs in the frame of the near-eye display device and project into the waveguide through the same in-couplers. In such examples, the other VCSELs may function as in-field illumination for camera-based eye tracking and / or as projectors for an AR / VR display. In some examples, both the eye tracking SMI / VCSELs and the illumination / projector VCSELs may be manufactured together in the same optical engine. In some examples, both the in-field eye tracking SMI / VCSELs and the in-field illumination / projector VCSELs may be fabricated in the same lithographic process to form an in-field eye tracking and illumination / projection integrated circuit.

[0101] According to some examples of the present disclosure, the eye tracking SMIs may be integrated with a waveguide / in-field camera. In some examples, the eye tracking SMIs, the waveguide camera, and an illumination and / or projection source may be integrated into the same optical engine. In examples where the eye tracking SMIs and the illumination / projection sources are both VCSELs in an integrated circuit, the in-field waveguide camera may also be manufactured in the integrated circuit with the same substrate as both the in-field eye tracking SMI / VCSELs and the in-field illumination / projector VCSELs. Such examples may be beneficial, as VCSELs are an optimal light source for the waveguide camera, which operates optimally when its illumination source is also in-field. In such examples, the illumination configuration may be optimized so that the reflected light from the eye is towards the normal direction.

[0102] According to some examples of the present disclosure, a diffractive grating platform (using, e.g., Volume Bragg Gratings (VBG) and / or Holographic Optical Elements (HOEs)) may be employed to realize a waveguide camera and in-field eye tracking SMIs integrated on the same waveguide with, for example, photopolymer. In such examples, grating / HOE coupler pairs and around-the-frame VCSEL emitters may be leveraged such that each grating in-couples one emitter’s light into the slab waveguide and a dedicated matching HOE out-couples that one emitter’s light to achieve a desired illumination profile at targeted locations. In this manner, around-the- frame illumination may be converted into in-field illumination using the same diffractive grating platform.EXAMPLES

[0103] FIGS. 4A and 4B are simplified block diagrams of a side view 400A and a planar view 400B, respectively, of a waveguide with eye tracking SMIs, in-couplers, and out-couplers according to an example. In FIGS. 4A and 4B, a waveguide 410 may have eye-side grating couplers 405 facing the user’s eye and frame-side grating couplers 415 facing the eye tracking SMIs 420 on the same side of the waveguide 410. While the eye-side grading couplers 405 must face the user’s eye, the frame-side couplers 415 (for coupling with the eye tracking SMIs) may be disposed elsewhere (such as, e.g., on the opposite side and / or at the end of the waveguide 410) and thus may also be referred to herein as “the SMI-side grating couplers.” The components shown in FIGS. 4A and 4B are provided to illustrate an explanation below of this example, and omits aspects, features, and / or components not germane to examples of the present disclosure, as would be understood by one of ordinary skill in the art; moreover, the components shown in FIGS. 4A and 4B may not be shown in accurate aspect and / or ratio of relative sizes. For instance, the relative sizes of the waveguide 410, the couplers, and the eye tracking SMIs 420 in FIGS. 4A and 4B may in no way approximate the relative dimensions of those components. In other words, the sizes, proportions, relative aspects, etc., of the elements shown in FIGS. 4A and 4B are not intended to be accurate, as would be understood by one of ordinary skill in the art, but rather to illustrate examples of the present disclosure.

[0104] According to examples of the present disclosure, the eye tracking SMIs act as both an illumination source and an image sensor for eye tracking (which hereinafter may be referred to, inter alia, as “the SMI eye tracking sensor(s) / projector(s)”). The SMI eye tracking sensor(s) / projector(s) operate by: (1 ) projecting a beam of light on the user’s eye (via the waveguide); (2) receiving the reflection of that light from the user’s eye (via the waveguide); (3) modulating the received beam (by self-mixing both the received beam and the output beam); and (4) producing an electrical signal corresponding to received, modulated beam. As would be understood by one of ordinary skill in the art, this abridged description is provided to illustrate examples of the present disclosure, and may omit steps, processes, aspects, features, and / or components not germane to examples of the present disclosure.

[0105] According to examples of the present disclosure, the eye tracking SMIs may project light / radiation in the infrared (IR) spectrum (e.g., about 750nm-1000nm). In some examples, the eye tracking SMIs may project light / radiation light / radiationfrom any one or more of the ultraviolet spectrum (e.g., about 200-350nm), the visual light spectrum (e.g., about 350nm-750nm), the infrared spectrum, and / or any electromagnetic radiation spectrum capable of being employed for eye tracking in accordance with examples of the present disclosure, as would be understood by one of ordinary skill in the art.

[0106] As shown in FIGS. 4A and 4B, the eye tracking SMIs may be in the form of Vertical Cavity Surface Emitting Laser (VCSELs), i.e., VCSEL / SMI eye tracking sensors / projectors 420, although eye tracking SMIs according to examples of the present disclosure are not limited to VCSEL constructions / fabrications. In FIGS. 4A and 4B, the VCSEL / SMI eye tracking sensors / projectors 420 may project a beam of light into the frame-side grating couplers 415, and that light is reflected internally within the waveguide 410, until it exits the waveguide 410 via the eye-side grating couplers 405 and thereby are projected upon the user’s eye. When projecting, the frame-side grating couplers 415 operate as in-couplers and the eye-side grating couplers 405 operate as out-couplers. The projected light may then be reflected from the eye back to the eye-side grating couplers 405, where it is reflected internally within the waveguide 410, until it exits the waveguide 410 via the frame-side grating couplers 415 and is thereby received by the VCSEL / SMI eye tracking sensors / projectors 420, each of which self-mixes the received, reflected light and produces an electrical signal corresponding thereto. When receiving, the frame-side grating couplers 415 operate as out-couplers and the eye-side grating couplers 405 operate as in-couplers.

[0107] In some examples, each of the VCSEL / SMI eye tracking sensors / projectors 420 may have a matching pair of couplers, i.e., a matching single one of the frame-side grating couplers 415 and a matching single one of the eye-side grating couplers 405, where the beam from that single VCSEL / SMI eye tracking sensor / projector enters and exits only through that matching pair of couplers. In some examples, unlike that shown in FIGS. 4A-4B, each of the SMI eye tracking sensors / projectors may have its own channel, and a single eye-side couplerand single SMI-side coupler is used for all of the channels. In other examples, a combination of individual and / or grouped couplers and / or individual and / or grouped channels may be employed.

[0108] The electrical signals produced by the VCSEL / SMI eye tracking sensors / projectors 420 are transmitted to a controller 430, which may perform eye tracking by processing the received electrical signals. In some examples, the controller430 may also control one, more, or all of the VCSEL / SMI eye tracking sensors / projectors 420 to project the light which is reflected from the user’s eye. In some examples, the controller 430 may be part of the eye tracking unit 130, eye tracking module 118, and / or processor(s) 121 in FIG. 1 , the controller 317 in FIG. 3B, and / or any other one or more processing units disposed on or in the frame 305 of the near-eye display device 300 in FIGS. 3A-3B, or on, in, and / or communicatively connected to the near-eye display device 200 in FIGS. 2A-2B. The controller 430 may include processor 433 and memory 435 which may store instructions executable by the processor 433 to perform the methods described herein.

[0109] In some examples, each of the VCSEL / SMI eye tracking sensors / projectors 420 may include a top reflector layer, a middle reflector layer, and a bottom reflector layer, where the top and middle reflector layers enclose an active region / optical cavity, and the middle and bottom layers enclose a photodetector (PD)Zactive region layer. The active region / optical cavity between the top reflector layer and the middle reflector layer is part of the light generating source. The photodetector (PD) / active region layer between the middle and bottom layers is part of the electrical signal generation and may include, for example, a resonant cavity photodetector (RCVP); whereas, in other examples, other suitable light detecting element(s) may be included, as would be understood by one of ordinary skill in the art. In some examples, the reflector layers are Bragg reflectors, where, for example, the top reflector layer may include a p-type distributed Bragg reflector (DBR), and the middle and bottom reflector layers may include an n-type distributed Bragg reflector (DBR).

[0110] In some examples, the active region / optical cavity may be employed to generate and project coherent light which is guided through the waveguide and couplers to a user’s eye and at least a portion of that projected light may be reflected back to the eye tracking SMI, where it enters the active region / optical cavity interferes with presently generated projected light, thereby producing a modulated feedback light which is received by the PD layer. In some examples, the controller 430 controls each of the VCSEL / SMI eye tracking sensors / projectors 420 to project the coherent light towards the user’s eye and then receives signals / data representing the modulated feedback light from each of the VCSEL / SMI eye tracking sensors / projectors 420. In some examples, the controller 430 processes this received data / signals to generate position and / or movement information regarding the user’s eye based on, e.g.,intensity, power, and / or other measurements / calculations of the modulated feedback light as detected by the PD layer.

[0111] As mentioned above, the VCSEL / SMI eye tracking sensors / projectors 420 in the example of FIGS. 4A-4B have VCSEL constructions, architectures, and / or fabrications. In such VCSEL examples, the VCSEL may include, for example, a VCSEL with multiple active regions (e.g., a bipolar cascade VCSEL); a tunnel junction VCSEL; a tunable VCSEL which may employ, e.g., a micro-electromechanical system (MEMS); a wafer-bonded and / or wafer-fused VCSEL; a Vertical External Cavity Surface Emitting Laser (VECSEL); a Vertical Cavity Semiconductor Optical Amplifier (VCSOA) which may be optimized as amplifiers as opposed to oscillators; two or more Vertical Cavity Surface Emitting Lasers (VCSELs) disposed on top of one another (i.e., vertically) such that each one pumps the one on top of it (e.g., monolithically optically pumped VCSELs); any other suitable VCSEL construction, architecture, and / or fabrication, as would be understood by one of ordinary skill in the art in light of the examples of the present disclosure; and / or other constructions, architectures, and / or fabrications suitable for the present disclosure may be employed besides a VCSEL, such as — with appropriate architectural modifications, for example, an Edge-Emitting Laser (EEL), a Horizontal Cavity Surface Emitting Laser (HC-SEL), a Quantum Dot Laser (QDL), a Quantum Cascade Laser (QCL), a micro-Light Emitting Diode (mLED), any other form of solid state laser, and / or any light source suitable for examples according to the present disclosure, as would also be understood by one of ordinary skill in the art.

[0112] According to examples of the present disclosure, the frame-side grating couplers 415 and the eye-side grating couplers 405 may be used for in-field waveguide eye tracking by employing a configuration such as shown in FIGS. 4A-4B. The frameside couplers 415 in FIGS. 4A-4B are one type of SMI-side couplers (i.e., where the eye tracking SMIs are located within the frame, facing the inner surface of the waveguide) and, in some contexts, the term “frame-side coupler” should be interpreted as meaning the broader term SMI-side couplers. In some examples, the frame- side / SMI-side grating couplers 415 may form various shapes and / or configurations corresponding to the locations and beam directions of the eye tracking SMIs 420. In some examples, the frame-side / SMI-side grating couplers 415 and the eye-side grating couplers 405 may be in the 300-500 microns range, or more generally the 100- 900 microns range, in width. In some examples, the eye-side grating couplers mayform an array, such as a 1 D array or line across the waveguide 410 or a 2D array covering an area of the waveguide 410.

[0113] In some examples, various types of couplers may be employed to both receive and project the projected light and the reflected light, including, for example, diffraction gratings such as Polarization Volumetric Hologram-based (PVH) gratings, Surface Relief Gratings (SRGs), and / or Volume Bragg Gratings (VBGs). In some examples, other suitable optical couplings may be employed, such as, for example, prisms, nano-optics (including, e.g., metalenses and metasurfaces), surface coatings, refractive elements, reflective elements, 3D printed microstructures and / or nanostructures, lithographically-created layers within the waveguide, and / or any other suitable technique, technology, layer, coating, and / or material feasible and / or possible either presently or in the future, as would be understood by one of ordinary skill in the art.

[0114] When using Polarization Volumetric Hologram-based (PVH) gratings, a two-layer stack between both gratings may be utilized to enhance the output coupling where each layer handles one of the polarizations. When the expected Signal-to- Noise Ratio (SNR) for the collimated beams upon the scattering surfaces of the eye (e.g., the sclera, the iris, perhaps the retina, etc.) may be potentially limiting optimal performance, 3D printed microlenses, diffractive or holographic optical elements with optical power including metasurface / metalenses, Pancharatnam Berry Phase (PBP) lenses, and / or VBG lenses may be employed, if necessary. When using Volume Bragg Gratings (VBGs), a high coupling around the near IR range may be achieved, as shown by the examples in FIGS. 5 and 6 described below.

[0115] FIGS. 5 and 6 are simplified block diagrams of waveguides with eye tracking SMIs using Volume Bragg Gratings (VBGs), according to examples of the present disclosure. More specifically, FIG. 5 shows a configuration where a VBG grating is used on the outer surface of the waveguide (facing the external environment) and a VBG lens is used on the inner surface of the waveguide (facing the eye); while FIG. 6 shows a configuration where a VBG lens is used on the outer surface of the waveguide (facing the external environment) and an Anti-Reflective (AR) coating is used on the inner surface of the waveguide (facing the eye).

[0116] The components shown in FIGS. 5 and 6 are provided to illustrate an explanation below of these examples, and omits aspects, features, and / or components not germane to examples of the present disclosure, as would be understood by oneof ordinary skill in the art; moreover, the components shown in FIGS. 5 and 6 may not be shown in accurate aspect and / or ratio of relative sizes. For instance, the relative sizes of the waveguides, the input gratings / prisms, and the eye tracking SMIs in FIGS. 5 and 6 may in no way approximate the relative dimensions of those components. In other words, the sizes, proportions, relative aspects, etc., of the elements shown in FIGS. 4A and 4B are intended to illustrate examples of the present disclosure, and not intended to show accurate proportions, etc., as would be understood by one of ordinary skill in the art.

[0117] FIG. 5 shows a waveguide 510 with eye tracking SMI(s) 520, an input grating / prism 515, a VBG grating 507 on the outer surface of the waveguide, and a VBG lens 505 on the inner surface of the waveguide 510, according to an example. Similarly to FIGS. 4A-4B, the eye tracking SMI(s) 520 is / are connected to a controller 530 with a processor 533 and a memory 535. In FIG. 5, the rays of the light projected from the eye tracking SMI(s) 520 are represented by solid gray arrows, while the rays of the light reflected back from the eye are represented by light gray dashed arrows.

[0118] In FIG. 5, the projected light (the solid gray arrows) by the eye tracking SMI(s) 520 enters the waveguide 510 through the input grating / prism 515, travels by reflection through the waveguide 510 until the projected light is reflected by the VBG grating 507 towards the VBG lens 505 and thereby focused onto the user’s eye. The reflected light (the dashed light gray arrows) from the user’s eye reflects back through the VBG lens 505 through the waveguide 510 to the VBG grating 507, which re-directs the reflected light into / through the waveguide 510 back to the input grating / prism 515 which projects the reflected light into the eye tracking SMI(s) 520. As discussed above, the eye tracking SMI(s) 520 self-mixes the reflected and projected light and produces an electrical signal corresponding to that self-mixed / modulated light, which is transmitted to the controller 530, which may perform, in whole or in part, eye tracking functions. As mentioned above, in the various examples of the present disclosure, the projected light may not be focused, but may rather be diverging or collimating. As also mentioned above, implementations of the present disclosure may be designed to redirect the light projected by the eye tracking SMIs in any desired beam configuration, modulation, orientation, direction, polarization, etc., as would be understood by one of ordinary skill in the art. For instance, the couplers for the eye tracking SMIs (and possibly the waveguide and / or other optical components, such as coatings, reflectors, diffractive surfaces, etc.) may be designed to achieve any desired intensity,distribution, power, angular orientation, polarization, etc., for the projected beams, as would be understood by one of ordinary skill in the art.

[0119] FIG. 6 shows a waveguide 610 with eye tracking SMI(s) 620, an input grating / prism 615, a VBG lens 607 on the outer surface of the waveguide, and an anti- reflective (AR) coating 605 on the inner surface of the waveguide 610, according to an example. In some examples, the VBG lens 607 may be replaced by a transmission or similar lens, as would be understood by one of ordinary skill in the art. Similarly to FIGS. 4A-4B and 5, the eye tracking SMI(s) 620 is / are connected to a controller 630 with a processor 633 and a memory 635. In FIG. 6, the rays of the light projected from the eye tracking SMI(s) 620 are represented by solid gray arrows, while the rays of the light reflected back from the eye are represented by light gray dashed arrows.

[0120] In FIG. 6, the projected light (solid gray arrows) by the eye tracking SMI(s) 620 enters the waveguide 610 through the input grating / prism 615, travels by reflection through the waveguide 610 until the projected light is reflected and directed by the VBG lens 607 towards the user’s eye through the AR coating 605. In some examples, the VBG lens 607 may be employed as a transmissive lens and / or be disposed at another side of the waveguide as well. The light reflected from the user’s eye (light gray dashed arrows) reflects back through the AR coating 605 into the waveguide 610 to the VBG lens 607, which re-directs the reflected light into / through the waveguide 610 back to the input grating / prism 615 which projects the reflected light into the eye tracking SMI(s) 620, which, as discussed above, self-mixes the reflected and projected light and produces an electrical signal corresponding to that modulated light which is transmitted to the controller 630, which may perform, in whole or in part, eye tracking functions.

[0121] As mentioned above, although the eye tracking SMI(s) 520 and 620 are (most likely) placed either in or on the frame / body of the near-eye display device so as to point into the input grating / prisms 515 and 615 as shown in FIGS. 5 and 6, the eye tracking SMI(s) according to other examples of the present disclosure may be disposed in any location with any relative orientation (provided there are suitable optics to re-direct the projected light into the in-couplers on the waveguide). Similarly, although the in-couplers for the eye tracking SMI(s) in FIGS. 4A (i.e., frame-side couplers 415), 5 (i.e., input grating / prism 515), and 6 (i.e., input grating / prism 615) are shown at the far end of the inner surface of the waveguide, the in-couplers according to other examples of the present disclosure may be disposed elsewhere on thewaveguide, such as, for example, the top, bottom, and / or side surfaces of the waveguide.

[0122] As mentioned above, the out-couplers on the inner surface of the waveguide facing the eye (e.g., the eye-side couplers 405 in FIGS. 4A-4B and the input gratings / prisms 515 & 615 in FIGS. 5 & 6) may also have beamforming, focusing, and / or light deflecting properties according to examples of the present disclosure. In some examples, the out-couplers on the outer edges of the waveguide may focus their out-going beams at a greater angle inwards so as to be normal to the rounded surface of the eye. Although the couplers discussed more specifically above were diffraction gratings (such as, e.g. PVH gratings, SRGs, VBGs, etc.), any other technique, technology, layer, coating, and / or material suitable, appropriate, and / or capable of providing the performance characteristics as shown by the examples of the present disclosure may be employed, as would be understood by one of ordinary skill in the art. Such couplers include structures, materials, and / or coatings in the waveguide; structures, materials, and / or coatings on the waveguide; structures, materials, and / or coatings fabricated as part of the waveguide; structures, materials, and / or coatings laminated, coated, and / or otherwise disposed on the waveguide; etc., such as, e.g., 3D printed microstructures and / or nanostructures, lithographically-created layers, and the like.

[0123] In some examples, the couplers may be metasurfaces which may, or may not, also have a beam-shaping, lensing, focusing, or other function for directing and / or otherwise modifying the light projected upon the user’s eye. In examples using metasurface as couplers, the metasurface may be on the order of nanometers and may therefore be thinner than other optical elements and / or layers, while also providing a much higher precision and may be much more resilient than other optical elements. In some examples, the metasurface may be a planar structure which can modulate the local properties (e.g., amplitude, phase, polarization, etc.) of an optical light beam which is transmitted through it. In some examples, the metasurface may comprise a large array of nanostructures, where the shape and dimensions of these nanostructures determine the phase and / or amplitude at which light is scattered from them, thus facilitating substantially complete control over the phase and / or amplitude profile of the light beam which emerges from the metasurface. In such examples, in contrast to conventional optical components which rely on thickness variation to induce a phase profile, the metasurface may realize arbitrary phase distributions usinglarge arrays with sub-wavelength and ultrathin (tens of nanometers) features. In such examples, the metasurface may be easily realized using a single lithographic step and may be highly suited for patterning a variety of substrates, including nonplanar and soft surfaces. In some examples, the metasurface may include optical polymers, plastic, glass, transparent wafers (e.g., Silicon Carbide (SiC) wafers), amorphous silicon, Silicon Oxide (SiO2), Silicon Nitride (SiN), Titanium Oxide (TiO), optical nylon, carbon-polymers, and / or any other transparent materials used for such a purpose, as would be understood by one of ordinary skill in the art.

[0124] In some examples, the controllers 430 / 530 / 630, the eye tracking SMI(s) 420 / 520 / 620, the in-couplers 415 / 515 / 615, and / or out-couplers 405 / 505 / 507 / 607 may be integrated into and / or employed as a component in other systems and / or for other functionalities of a near-eye display device, as would be understood by one of ordinary skill in the art. For instance, the eye tracking SMI(s) may be integrated into and / or collocated with the components projecting AR / VR images to the user through the waveguide, such as, e.g., the inward projector(s) 173 and / or display electronics 122 of FIG. 1. As another instance, the controllers of the eye tracking SMI(s) may be integrated into, and / or implemented as part of, the eye tracking system of the near- eye display device (such as, e.g., the eye tracking module 130 and / or the eye tracking unit 118 in FIG. 1 or the controller(s) 317 in FIG. 3B).

[0125] As shown in FIGS. 4A-4B, 5, and / or 6, the controllers 430, 530, and / or 630, respectively, may include the processors 433, 533, and / or 633, respectively, and the memories 435, 535, and / or 635, respectively. In some examples, the controller 430, 530, and / or 630 may be implemented as hardware, software, and / or a combination of hardware and software in the near-eye display device. In some examples, the controller 430, 530, and / or 630 may be implemented, in whole or in part, by at least one of any type of application, program, library, script, task, service, process, or any type or form of executable instructions executed on hardware such as circuitry that may include digital and / or analog elements (e.g., one or more transistors, logic gates, registers, memory devices, resistive elements, conductive elements, capacitive elements, and / or the like, as would be understood by one of ordinary skill in the art). In some examples, the processor433, 533, and / or633 may be implemented with a general purpose single- and / or multi-chip processor, a single- and / or multi-core processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic device,discrete gate or transistor logic, discrete hardware components, and / or any combination thereof suitable to perform the functions described herein. A general purpose processor may be any conventional processor, microprocessor, controller, microcontroller, and / or state machine. In some examples, the memory 435, 535, and / or 635 may be implemented by one or more components (e.g., random access memory (RAM), read-only memory (ROM), flash or solid state memory, hard disk storage, etc.) for storing data and / or computer-executable instructions for completing and / or facilitating the processing and storage functions described herein. In such examples, the memory 435, 535, and / or 635 may be volatile and / or non-volatile memory, and may include database components, object code components, script components, or any other type of information structure suitable for implementing the various activities and storage functions described herein.

[0126] In another aspect, methods and / or systems for eye tracking and / or other sensing in a near-eye display device using one or more SMIs coupled to a waveguide of the near-eye display device may be employed. In yet another aspect, a system for manufacturing a waveguide to be coupled to one or more SMIs for eye tracking and / or other sensing in a near-eye display device may be employed.

[0127] FIG. 7 is a flowchart illustrating a method for eye tracking using one or more SMI eye tracking sensors, according to an example of the present disclosure. The method 700 shown in FIG. 7 is provided by way of example and may only be one part of an entire process, procedure, ongoing operation, method, etc., as would be understood by one of ordinary skill in the art. The method 700 may further omit parts of any process, procedure, ongoing operation, method, etc., involved in eye tracking not germane to examples of the present disclosure, as would be understood by one of ordinary skill in the art. Each block shown in FIG. 7 may further represent one or more steps, processes, methods, or subroutines, as would be understood by one of ordinary skill in the art. For the sake of convenience and ease of explanation, the blocks in FIG. 7 may refer to the components shown in the Figures described herein; however, the method 700 is not limited in any way to the components, apparatuses, and / or constructions described and / or shown in any of the Figures herein.

[0128] In FIG. 7, some of the processes indicated by the blocks may overlap and / or may occur substantially simultaneously and, moreover, the blocks may be performed by different processing components. For instance, eye tracking is a continuous, on-going process, and thus all of the blocks in FIG. 7 may be referred toas being performed “at the same time,” i.e., they may be substantially continually being performed (although at different stages).

[0129] In FIG. 7, any of blocks may be performed by the same one or more processors / controllers or by different processors / controllers, as would be understood by one of ordinary skill in the art. In some examples, the components performing any of blocks may be controlled / directed to do so by one or more controllers / processors (such as, for example, the controller(s) 430, 530, and / or 630 in FIGS. 4, 5, and / or 6, respectively; the controller(s) 317 in FIG. 3B; the eye tracking unit 130, the eye tracking module 118, and / or the processor(s) 121 of FIG. 1 , and / or any other suitable processor / controller, as would be understood by one of ordinary skill in the art).

[0130] At block 710 in FIG. 7, one or more SMI eye tracking sensors may project light intended for a user’s eye into a waveguide. In some examples, the one or more SMI eye tracking sensors may be the eye tracking sensor projectors 420 in FIG. 4A, the eye tracking SMI(s) 510 in FIG. 5, and / or the eye tracking SMI(s) 620 in FIG. 6. In some examples, one or more controllers / processors may control the one or more SMI eye tracking sensors to perform block 710, such as, e.g., the controller(s) 430, 530, and / or 630 in FIGS. 4, 5, and / or 6, respectively; the controller(s) 317 in FIG. 3B; the eye tracking unit 130, the eye tracking module 118, and / or the processor(s) 121 of FIG. 1 , and / or any other suitable processor / controller, as would be understood by one of ordinary skill in the art.

[0131] At block 720, one or more SMI couplers on the waveguide receive the projected light from the one or more SMI eye tracking sensors. In some examples, the one or more SMI couplers may be the frame-side grating couplers 415 in FIG. 4A, the input grating / prism 515 in FIG. 5, and / or the input grating / prism 615 in FIG. 6.

[0132] At block 730, one or more modulators in / on the waveguide module the projected light. In some examples, the one or more modulators may include the VBG grating 507 and / or the VBG lens 505 of FIG. 5, the VBG lens 607 and / or the AR coating 605 in FIG. 6, the in-couplers and / or out-couplers of the waveguide (such as, e.g., the frame-side grating couplers 415 and / or the eye-side grating couplers 405 in FIGS. 4A- 4B), and / or any other modulating technology suitable for the task as described herein, as would be understood by one of ordinary skill in the art. In some examples, the one or more modulators in block 730 may include, at least one of a prism, a surface, a coating, a reflective grating, a refractive grating, a diffractive grating, a metasurface, a metalens, and / or a lens. In some examples, the modulating performed in block 730may include, e.g., one or more of: redirecting the beam of light projected by the eye tracking SMI in a desired direction towards the eye of the user, and / or adding a desired at least one of an angular orientation, polarization, or focusing power to the beam of light projected by the eye tracking SMI when the light is projected towards the eye of the user.

[0133] At block 740, the modulated light is projected out from the waveguide to the user's eye. In some examples, this may be done through an out-coupling, such as, e.g., the eye-side grating couplers 405 in FIG. 4B, the VBG lens 505 of FIG. 5, and / or the AR coating 605 in FIG. 6.

[0134] At block 750, the waveguide receives light reflected back from the user’s eye, including reflections of the projected modulated light from block 740. In some examples, this may be done through the same couplers as in block 740, i.e., the couplers are both in-couplers and out-couplers. In some examples, this may be done through the eye-side grating couplers 405 in FIG. 4B, the VBG lens 505 of FIG. 5, and / or the AR coating 605 in FIG. 6.

[0135] At block 760, one or more SMI couplers (which may or may not be the same one or more SMI couplers from block 720) projects the received reflected light out from the waveguide toward the one or more SMI eye tracking sensors from block 710. Even if not the same identical devices as used in block 720, the functional description of the SMI coupler(s) in block 760 is the same, as would be understood by one of ordinary skill in the art.

[0136] At block 770, the one or more SMI eye tracking sensors from block 710 receive and self-mix the reflected light. In some examples, the reflected light is received in a resonant cavity of the one or more SMI eye tracking sensors, where the changed phase of the reflected light is modulated by mixing with the presently-emitting light which has a different phase (i.e., the phase of the light emitted in block 710).

[0137] At block 780, the one or more SMI eye tracking sensors may generate one or more electrical signals corresponding to the self-mixing in block 770. In some examples, the self-mixed / modulated light from block 770 may be received by one or more light sensors and thereby turned into an electromagnetic signal, which may also be amplified. In some examples, one or more controllers / processors may control all or part of the processing / performing of blocks 770 and / or 780, such as, e.g., the controller(s) 430, 530, and / or 630 in FIGS. 4, 5, and / or 6, respectively; the controller(s) 317 in FIG. 3B; the eye tracking unit 130, the eye tracking module 118, and / or theprocessor(s) 121 of FIG. 1 , and / or any other suitable processor / controller, as would be understood by one of ordinary skill in the art.

[0138] At block 790, eye tracking may be performed using at least the one or more electrical signals generated in block 780. In some examples, one or more controllers / processors may perform all or part of the eye tracking for the near-eye device, such as, e.g., the eye tracking unit 130, the eye tracking module 118, and / or the processor(s) 121 of FIG. 1 ; the controller(s) 317 in FIG. 3B; and / or any other suitable processor / controller, as would be understood by one of ordinary skill in the art. In some examples, eye tracking may be performed in a distributed manner, where multiple controllers / processors perform various roles in the eye tracking, including, for example, the controller(s) 430, 530, and / or 630 in FIGS. 4, 5, and / or 6, respectively; the controller(s) 317 in FIG. 3B; the eye tracking unit 130, the eye tracking module 118, and / or the processor(s) 121 of FIG. 1 , and / or any other suitable processor / controllers, as would be understood by one of ordinary skill in the art.CASE STUDIES

[0139] FIGS. 8 and 9 illustrate the coupling efficiency of Volume Bragg Gratings (VBGs) used as couplings with eye tracking SMI(s) according to examples of the present disclosure. More specifically, FIG. 8 shows graphs illustrating the efficiency of a VBG when the light is normal to the surface of the waveguide, whereas FIG. 9 shows a simplified block diagram of a waveguide illustrating how the angle of the output light is varied by changing the slant and pitch of the grating of the VBG and graphs illustrating the efficiency as the slant and pitch of the VBG are varied. In the graphs shown in FIGS. 8 and 9, the wavelength range is about 830nm to 870nm; the efficiency of s-polarized light is indicated by a dashed line with X’s, and the p-polarized light is indicated by a dashed line with O’s; and the efficiency ranges from 0.0 to 1 .0. In FIGS. 8 and 9, the waveguide used involved glass with a refractive index of 1 .5 of the VBG to air and a 60° Total Internal Reflection (TIR) angle (off the vertical axis). The Rigorous Coupled-Wave Analysis (RCWA) method was employed for analysis.

[0140] In FIG. 8, the incident light and output light are both normal: graph 810 shows the efficiencies when the refractive index difference (Dn) is 0.03, and graph 820 shows the efficiencies when the refractive index difference (Dn) is 0.04. In FIG. 8, graphs 810 and 820 show that the VBG used as a coupler in accordance with an example of the present disclosure has a relatively optimal input coupling efficiency, aswould be understood by one of ordinary skill in the art.

[0141] In FIG. 9, the output angle of the light varies from -15° to +15° by varying the pitch and slant of the grating of a VBG 907 of a waveguide 910. In some examples, the numerical aperture (NA) of the light beam may be 0.26. The waveguide configuration included a 5um thick film with a refractive index difference (Dn) of 0.04, and was operated in reflective mode. Graph 915 shows the efficiencies when an output angle of +15° is projected by having the pitch of the grating be 817nm and the slant of the grating be 64.97°; graph 925 shows the efficiencies when an output angle of +7.5° is projected by having the pitch be 727nm and the slant be 62.50°; graph 935 shows when an output angle of 0° is projected by having the pitch be 654nm and the slant be 60.00°; graph 945 shows when an output angle of -7.5° is projected by having the pitch be 595nm and the slant be 57.50°; and graph 955 shows when an output angle of - 15° is projected by having the pitch be 545nm and the slant be 55.03°. Accordingly, in FIG. 9, graphs 915-955 show that the VBG used as a coupler in accordance with an example of the present disclosure has a relatively optimal output coupling efficiency, as would be understood by one of ordinary skill in the art.

[0142] According to examples, systems and apparatuses for in-field SMI eye tracking in a near-eye display device are described herein. One or more methods for eye tracking using in-field SMIs in a near-eye display device are also described herein. A non-transitory computer-readable storage medium may have an executable stored thereon, which when executed instructs a processor to perform any of the methods described herein.

[0143] In the foregoing description, various examples are described, including devices, systems, methods, and the like. For the purposes of explanation, specific details are set forth in order to provide a thorough understanding of examples of the disclosure. However, it will be apparent that various examples may be practiced without these specific details. For example, devices, systems, structures, assemblies, methods, and other components may be shown as components in block diagram form in order not to obscure the examples in unnecessary detail. In other instances, well- known devices, processes, systems, structures, and techniques may be shown without necessary detail in order to avoid obscuring the examples.

[0144] The figures / drawings and description are not intended to be restrictive. The terms and expressions that have been employed in this disclosure are used as terms of description and not of limitation, and there is no intention in the use of suchterms and expressions of excluding any equivalents of the features shown and described or portions thereof. The word "example" is used herein to mean "serving as an example, instance, or illustration." Any embodiment or design described herein as "example1is not necessarily to be construed as preferred or advantageous over other embodiments or designs.

[0145] Although the methods and systems as described herein may be directed mainly to digital content, such as videos or interactive media, it should be appreciated that the methods and systems as described herein may be used for other types of content or scenarios as well. Other applications or uses of the methods and systems as described herein may also include social networking, marketing, content-based recommendation engines, and / or other types of knowledge or data-driven systems.

Claims

CLAIMS1. A near-eye display device, comprising: a frame; a waveguide disposed in the frame to transmit light by internal reflection and to provide Augmented Reality (AR)A / irtual Reality (VR) images to a user’s eye; an eye tracking self-mixing interferometer (SMI) disposed in the frame to provide projected light to the user’s eye and to receive reflected light from the projected light on the user’s eye; an SMI-side coupler on the waveguide to receive into the waveguide the projected light from the eye tracking SMI and to project from the waveguide to the eye tracking SMI the reflected light from the user’s eye; and an eye-side coupler on the waveguide to project the projected light from the waveguide onto the user’s eye and to receive into the waveguide the reflected light from the user’s eye; wherein the eye tracking SMI is to further modulate the received reflected light with presently projected light to provide an electrical signal corresponding to the modulated light, wherein non-image-based eye tracking is performed based on the electrical signal.

2. The near-eye display device of claim 1 , wherein at least one of the waveguide, the SMI-side coupler, or the eye-side coupler is further to redirect the projected light from the eye tracking SMI in a desired direction towards the user’s eye.

3. The near-eye display device of claim 1 or 2, wherein at least one of the waveguide, the SMI-side coupler, or the eye-side coupler is further to add a desired angular orientation, polarization, or focusing power to the projected light as the projected light is projected to the user’s eye.

4. The near-eye display device of any preceding claim, wherein the eye tracking SMI is integrated with another eye tracking system in a sensor fusion framework.

5. The near-eye display device of any preceding claim, wherein the eye tracking SMI is operated in at least one of a coherent sensing or Frequency Modulated Continuous Wave (FMCW) sensing mode.

6. The near-eye display device of any preceding claim, wherein the SMI-side coupler or the eye-side coupler comprises at least one of a prism, a surface, acoating, a reflective grating, a refractive grating, a diffractive grating, a metasurface, a metalens, or a lens, in which case optionally wherein the diffractive grating comprises at least one of a Polarization Volumetric Hologram-based (PVH) grating, a Surface Relief Grating (SRG), or a Volume Bragg Gratings (VBG).

7. The near-eye display device of any preceding claim, further comprising: an in-field waveguide camera disposed on the frame.

8. The near-eye display device of any preceding claim, further comprising: a waveguide coupler on the waveguide to receive light from the SMI-side coupler and project it to the eye-side coupler and to receive light from the eye-side coupler and project it to the SMI-side coupler.

9. The near-eye display device of any preceding claim, wherein the eye tracking SMI comprises a vertical cavity self emitting laser (VCSEL).

10. A near-eye display device, comprising: a frame; a waveguide disposed in the frame to transmit light by internal reflection and to provide Augmented Reality (AR)ZVirtual Reality (VR) images to a user’s eye; a plurality of eye tracking self-mixing interferometers (SMIs) disposed in the frame to provide projected light for the user’s eye and to receive reflected light from the projected light on the user’s eye; a plurality of SMI-side couplers on the waveguide to receive into the waveguide the projected light from the eye tracking SMI and to project from the waveguide to the eye tracking SMI the reflected light from the user’s eye; and a plurality of eye-side couplers on the waveguide to project the projected light from the waveguide onto the user’s eye and to receive into the waveguide the reflected light from the user’s eye; wherein the eye tracking SMI is to further modulate the received reflected light with presently projected light to provide an electrical signal corresponding to the modulated light, wherein non-image-based eye tracking is performed based on the electrical signal.11 . The near-eye display device of claim 10, wherein the plurality of eye tracking SMIs are sparsely populated on the surface of the waveguide or disposed as a group or array on the surface of the waveguide.

12. The near-eye display device of claim 10 or 11 , wherein each of the plurality of eye tracking SMIs has a matching one of the plurality of SMI-side couplers and a matching one of the plurality of eye-side couplers, and wherein the projected light from each of the eye tracking SMIs in-couples only through its matching SMI-side coupler and out-couples only through its matching eye-side coupler; in which case optionally any one of more of: a) wherein at least one of the plurality of eye-side couplers directs the projected light from its matching eye tracking SMI in a desired direction towards the user’s eye; or b) wherein at least one of the plurality of eye-side couplers adds a desired angular orientation, polarization, or focusing power to the projected light from its matching eye tracking SMI as the projected light is projected to the user’s eye.

13. The near-eye display device of any one of claims 10 to 12, and any one or more of: a) wherein the plurality of eye tracking SMIs are integrated with another eye tracking system in a sensor fusion framework; or b) wherein each of the plurality of SMI-side couplers or the plurality of eyeside couplers comprises at least one of a prism, a surface, a coating, a reflective grating, a refractive grating, a diffractive grating, a metasurface, a metalens, or a lens.

14. A method of eye tracking in a near-eye display device, comprising: projecting, by a self-mixing interferometer (SMI) eye tracking sensor in a frame of the near-eye display device, a beam of light to be projected onto an eye of a user of the near-eye display device; receiving, by an SMI coupler of a waveguide of the near-eye display device, the projected beam of light from the SMI eye tracking sensor; modulating, by a modulator in the waveguide, the received beam of light for projecting the beam of light onto the eye of a user; projecting, from the waveguide, the modulated light onto the eye of the user; receiving, by the waveguide, reflected light from the eye of the user by the projected modulated beam of light; providing, by the SMI coupler, the received reflected light to the SMI eye tracking sensor;receiving and mixing, by the SMI eye tracking sensor, the received reflected light from the eye of the user; providing, by the SMI eye tracking sensor, an electrical signal corresponding to the received and mixed light; and performing eye tracking based on the electrical signal provided by the SMI eye tracking sensor.

15. The eye tracking method of claim 14, and any one or more of: a) wherein the modulator in the waveguide comprises at least one of a prism, a surface, a coating, a reflective grating, a refractive grating, a diffractive grating, a metasurface, a metalens, or a lens; or b) wherein the step of modulating, by a modulator in the waveguide, the received beam of light for projecting the beam of light onto the eye of a user further comprises at least one of: redirecting the beam of light projected by the eye tracking SMI in a desired direction towards the eye of the user; or adding a desired at least one of an angular orientation, polarization, or focusing power to the beam of light projected by the eye tracking SMI as the projected light is projected towards the eye of the user.

Images