Gaze tracking circuit with optical range finder

Abstract

A gaze tracking circuit with an optical range finder is disclosed. An eyewear, such as a head-mounted device, may include an adjustable prescription lens and / or may include a display. The eyewear may include a gaze tracking circuit that tracks a gaze direction of a user. The gaze tracking circuit may include a range finder that determines an eye pitch using a phase-based optical coherence tomography. The rangefinder may include one or more emitters, such as lasers, that emit infrared light into a beam splitter that splits the light into signal light that travels to a free space path of the eye and reference light that travels toward a fixed reference path of a plurality of image sensors located around a perimeter of the eye. The signal light is specularly reflected away from the eye (thereby producing an eye flash), and the reflected signal light is combined with the reference light to produce an interference pattern that can be detected by the image sensors and analyzed to determine an eye distance.

Description

[0001] This application claims priority to U.S. Patent Application No. 18 / 984,695, filed on December 17, 2024, and U.S. Provisional Patent Application No. 63 / 551,876, filed on February 9, 2024, which are hereby incorporated by reference in their entireties. Technical Field

[0002] The present invention relates generally to electronic devices, and more particularly to wearable electronic devices such as head-mounted devices. Background Art

[0003] Head-mounted devices and other eyewear can use gaze tracking circuitry to track the user's gaze.

[0004] Designing gaze tracking circuits with satisfactory performance can be challenging. If care is not taken, gaze tracking circuits may produce inaccurate measurements or may exhibit other performance limitations, such as excessive power consumption. Summary of the Invention

[0005] Eyewear, such as a head-mounted device, may include adjustable prescription lenses and / or may include a display. The lenses and display may be mounted to a support structure, such as a support frame or other head-mounted support structure.

[0006] The eyewear may include gaze tracking circuitry that tracks the direction of a user's gaze. The gaze tracking circuitry may include a camera, a light source, and a rangefinder, or the gaze tracking circuitry may include only a rangefinder.

[0007] The rangefinder can use phase-based optical coherence tomography to determine eye distance. The rangefinder may include one or more transmitters, each of which includes one or more lasers or other coherent light sources. The lasers can emit infrared light of known time-correlated wavelengths into a beam splitter, which splits the light into a signal light that travels a free-space path length to the eye and a reference light that travels a fixed reference path length toward multiple image sensors. The signal light reflects off the eye's mirror (generating an eye glint), and the reflected signal light combines with the reference light to produce an interference pattern that can be captured by the image sensor and analyzed to determine glint location, eye distance, eye velocity, and eye acceleration. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a top view of an illustrative head-mounted device that may include gaze tracking circuitry, according to one embodiment.

[0009] Figure 2 is a rear view of an illustrative head-mounted device that may include gaze tracking circuitry, according to one embodiment.

[0010] Figure 3 is a schematic diagram of an illustrative head-mounted device that may include gaze tracking circuitry, according to one embodiment.

[0011] Figure 4 is a top view of an illustrative gaze tracking circuit for tracking a user's gaze direction, according to one embodiment.

[0012] Figure 5 is a schematic diagram of an illustrative rangefinder for determining eye distance using optical coherence tomography, according to one embodiment.

[0013] Figure 6 According to one embodiment, Figure 5 Rear view of an illustrative head-mounted device for a rangefinder of the type shown.

[0014] Figure 7 is a side view of an illustrative rangefinder having an image sensor including a pixel array, a holographic optical element, a phase grating, and a lens, according to one embodiment.

[0015] Figure 8 is a side view of an illustrative phase grating having edges offset from the apex of overlapping lenses, according to one embodiment.

[0016] Figure 9 is a front view of an illustrative image captured by a pixel array and having two focal spots slightly out of phase with each other, according to one embodiment. DETAILED DESCRIPTION

[0017] Eyewear (such as a pair of glasses or other head-mounted device) may include one or more eye monitoring components such as gaze tracking circuitry for determining the direction of a user's gaze. The gaze tracking circuitry may include one or more rangefinders (e.g., distance sensors) that use optical coherence tomography to measure the path length of specular reflections off the sclera and cornea of the eye (sometimes referred to as glints). This path length information may then be used to determine the direction of the user's gaze. The gaze tracking circuitry may rely entirely on distance sensing to track the direction of the user's gaze, or the gaze tracking circuitry may use both a distance sensor and one or more cameras in a hybrid gaze tracking approach to capture images of the pupil and one or more glints.

[0018] A distance sensor based on optical coherence tomography may include one or more coherent light sources, such as lasers, that emit infrared beams into a beam splitter. The infrared light beams may have wavelengths that vary slightly over time in a known or measured manner. The beam splitter may include a star coupler that allows light from each laser to be detected by multiple sensors at different locations around the eye. The beam splitter may split the infrared light into a signal light that travels along a free-space path to the eye and a reference light that travels along a fixed reference path to one or more image sensors. The signal light emitted toward the eye produces a specular reflection (e.g., a glint) at the eye and reflects toward the image sensor. The reflected signal light from the eye is combined with the reference light from the fixed reference path, and the corresponding interference pattern is captured by the image sensor. If the free-space path length to the eye is equal to the fixed reference path length, the appearance of the image of the glint will remain unchanged. If the two path lengths are different, the corresponding interference pattern can be captured by the image sensor and analyzed to determine the glint location, eye distance, and eye velocity and acceleration (e.g., the first two derivatives of eye distance).

[0019] In a hybrid gaze tracking arrangement, an optical coherence tomography-based rangefinder can be combined with a light source and a camera to determine pupil position and glint position. For example, a light source such as a light emitting diode can produce a glint on the user's eye and / or illuminate the user's pupil and iris. Pupil position and glint position can be determined based on an image of the eye captured by a camera. The rangefinder can use phase-based optical coherence tomography to determine the distance to the eye (sometimes referred to as interocular distance). The user's gaze direction can be determined based on the location of the glint, the location of the pupil, and the distance to the eye.

[0020] Using a hybrid gaze tracking circuit that includes both a camera and light source for flash detection and a rangefinder for distance sensing can allow the use of fewer flashes (e.g., using only a single flash from a single light-emitting diode, if desired) to determine gaze direction. This can be beneficial in arrangements where fewer light sources are desired and / or in scenarios where multiple flashes cannot be obtained from a particular user's eyes. In some arrangements, the camera, light source, and rangefinder can remain powered during operation of device 10 and can be used continuously or periodically to track gaze direction. In other arrangements, power savings can be achieved by keeping the camera and / or light source off (or otherwise in a low-power state) until the rangefinder detects a change in eye distance (which can indicate a change in gaze direction). When a change in eye distance is detected by the rangefinder, the camera and light source can be turned on to help determine the user's new gaze direction.

[0021] In some arrangements, flash detection may not be necessary to determine gaze direction. For example, a camera may be used to capture information about the pupil, iris, eyelid, and / or other features of the eye, and this information may be supplemented with one or more ranging measurements from optical coherence tomography measurements of the path length of specular reflections off the surface of the user's eye.

[0022] Figure 1 A top view of an exemplary head mounted device or other eyewear is shown in FIG. Figure 1 As shown in FIG, a head-mounted device such as electronic device 10 may have a head-mounted support structure such as housing 12. Housing 12 may include a portion (e.g., support structure 12T) for allowing device 10 to be worn on a user's head. Support structure 12T may be formed from fabric, polymer, metal, and / or other materials. Support structure 12T may form a strap or other head-mounted support structure to help support device 10 on the user's head. The main support structure of housing 12 (e.g., main housing portion 12M) may support electronic components such as display 14. Main housing portion 12M may include a housing structure formed from metal, polymer, glass, ceramic, and / or other materials. For example, housing portion 12M may have a housing wall located on the front face F and housing walls located on adjacent top, bottom, left, and right sides, these housing walls being formed from a rigid polymer or other rigid support structure, and these rigid walls may optionally be covered with electronic components, fabric, leather, or other soft materials, etc. The walls of housing portion 1254 may enclose internal components 38 within interior region 34 of device 10 and may separate interior region 34 from the environment surrounding device 10 (exterior region 36). Internal components 38 may include integrated circuits, actuators, batteries, sensors, and / or other circuits and structures for device 10. Housing 12 may be configured to be worn on a user's head and may form glasses, a hat, a helmet, goggles, and / or other head-mounted devices. Configurations in which housing 12 forms goggles are sometimes described herein as an example.

[0023] The front side F of the housing 12 can face outward, away from the user's head and face. The opposite rear side R of the housing 12 can face the user. The portion of the housing 12 located on the rear side R (e.g., the portion of the main housing 12M) can form a cover, such as cover 12C (sometimes referred to as a blind). The presence of cover 12C on the rear side R can help hide the internal housing structure, interior components 38, and other structures in the interior area 34 from the user.

[0024] Device 10 may have left and right optical modules 40. Each optical module may include a corresponding display 14, a lens 30, and a support structure 32. Support structure 32, which may sometimes be referred to as a lens barrel or an optical module support structure, may include a hollow cylindrical structure with an open end or other support structure for accommodating display 14 and lens 30. Support structure 32 may, for example, include a left lens barrel that supports left display 14 and left lens 30, and a right lens barrel that supports right display 14 and right lens 30.

[0025] Display 14 may include an array of pixels or other display devices to produce images. Display 14 may include, for example, organic light-emitting diode pixels formed on a substrate having thin-film circuitry and / or formed on a semiconductor substrate, pixels formed from a crystalline semiconductor die, liquid crystal display pixels, a scanning display device, and / or other display devices for producing images.

[0026] Lens 30 may include one or more lens elements for providing image light from display 14 to corresponding eyepiece 13. Lens 30 may be implemented using a refractive glass lens element, using a mirror lens structure (a catadioptric lens), using a Fresnel lens, using a holographic lens, and / or other lens systems.

[0027] When the user's eyes are located in the eye zone 13, the displays (display panels) 14 operate together to form the display of the device 10 (e.g., the user's eyes can view images provided by the corresponding left and right optical modules 40 in the eye zone 13, thereby creating a stereoscopic image for the user). When the user views the display, the left image from the left optical module is fused with the right image from the right optical module.

[0028] If desired, device 10 may include an additional lens, such as lens 50. Lens 50 may be a fixed lens or may be an adjustable lens, such as a liquid crystal lens, a liquid-filled lens, or other suitable adjustable lens. Lens 50 may be configured to accommodate different focal length ranges and / or correct vision defects, such as myopia, hyperopia, presbyopia, astigmatism, higher-order aberrations, and / or other vision defects. For example, lens 50 may be an adjustable prescription lens having a first set of optical properties for a first user having a first prescription and a second set of optical properties for a second user having a second prescription. Lens 50 may be removably or permanently attached to housing 12. In arrangements where lens 50 is removable, lens 50 may have mating engagement features, magnets, clamps, or other attachment structures that allow lens 50 to be attached to housing 12 (e.g., individually or in pairs).

[0029] If desired, the device 10 may be used purely for vision correction (e.g., the device 10 may be a pair of eyeglasses, spectacles, etc.), and the Figure 114, lens 30, and optical module 40. In other arrangements, device 10 (sometimes referred to as eyewear 10, glasses 10, head-mounted device 10, etc.) may include a display that displays virtual reality, mixed reality, and / or augmented reality content. With this type of arrangement, lens 50 may be a prescription lens and / or may be used to move content between focal planes from the user's perspective. Lens 50 may be omitted if desired. Arrangements in which device 10 is a head-mounted device with one or more displays are sometimes described herein as illustrative examples.

[0030] It may be desirable to monitor the user's eyes while they are within the eye-viewing zone 13. For example, it may be desirable to use a camera to capture an image of the user's iris (or other portion of the user's eye) for user authentication. It may also be desirable to monitor the direction of the user's gaze. Gaze tracking information may be used as a form of user input and / or may be used to determine where within the image the image content resolution should be locally enhanced in a fovea imaging system. To ensure that the device 10 can capture a satisfactory image of the user's eyes when they are within the eye-viewing zone 13, each optical module 40 may be provided with gaze tracking circuitry 62. Gaze tracking circuitry 62 may include one or more cameras (such as camera 42), one or more light sources (such as light source 44 (e.g., a light emitting diode, a laser, a lamp, etc.)), and one or more rangefinders (such as rangefinder 48). The device 10 may include gaze tracking circuitry 62 for each eye (e.g., left and right eye), or the device 10 may include gaze tracking circuitry 62 for a single eye.

[0031] Camera 42 and light emitting diode 44 may operate at any suitable wavelength (visible, infrared, and / or ultraviolet). In the exemplary configuration sometimes described herein as an example, diode 44 emits infrared light that is invisible (or nearly invisible) to the user. This allows eye monitoring operations to continue without interfering with the user's ability to view images on display 14.

[0032] The rangefinder 48 (sometimes referred to as the depth sensor 48) can be any suitable rangefinder, such as an optical rangefinder (e.g., a light source and light sensor that collects time-of-flight measurements, phase-based measurements, a self-mixing sensor, a light detection and ranging (laser) sensor, a structured light sensor, and / or a depth sensor based on a stereo imaging device that captures three-dimensional images, etc.), an ultrasonic rangefinder (e.g., one or more capacitive micromachined ultrasonic transducers, piezoelectric micromachined transducers, and / or other suitable ultrasonic transducers for emitting and / or detecting acoustic signals), and / or any other suitable rangefinder.

[0033] In some arrangements, the rangefinder 48 may use phase-based optical coherence tomography to determine the distance to the eye. With this type of arrangement, the rangefinder 48 may include one or more coherent light sources, such as wavelength-tunable lasers, that transmit light into a beam splitter while simultaneously sweeping the wavelength of the emitted light over a given tunability range. Laser light emitted with this type of wavelength sweep is sometimes referred to as laser chirping. The beam splitter splits the light into a signal arm that produces a specular reflection (e.g., a flash) at the user's eye and a local oscillator arm that interferes with the light collected from the signal arm. The rangefinder 48 may include one or more image sensors (e.g., an array of pixels) that capture an image of the signal light and reference light. The image may include an interference pattern based on constructive and destructive interference between the signal light from the eye and the reference light from the local oscillator arm. The captured image may be processed to determine the path length of the specular reflection and the corresponding distance to the eye. If desired, only the portion of the pixel array that includes the flash image may be read out for processing (e.g., to conserve processing power). Specifically, a first laser chirp can be used to identify the portion of the pixel array that includes the flash, and a second laser chirp can be used to determine the distance to the flash using only that portion of the pixel array (e.g., by reading out only the subset of pixels that includes the flash). If desired, a third laser chirp can be used to determine a first temporal derivative of the distance to the eye (e.g., eye velocity). If desired, the third and fourth laser chirps can be used to determine the first two temporal derivatives of the distance to the eye (e.g., eye velocity and eye acceleration).

[0034] Not all users have the same interpupillary distance (IPD). To provide device 10 with the ability to adjust the interpupillary distance (IPD) between modules 40, and thus the distance (IPD) between eye zones 13, along the lateral dimension X to accommodate different user interpupillary distances, device 10 may be provided with actuator 43. Actuator 43 may be manually controlled and / or computer-controlled (e.g., a computer-controlled motor) for moving support structures 32 relative to one another. For example, camera 42 may be used to collect information about the position of the user's eyes. The position of eye zones 13 may then be adjusted accordingly.

[0035] like Figure 2 As shown, cover 12C can cover back surface F while leaving lens 30 of optical module 40 uncovered (e.g., cover 12C can have an opening that aligns with and receives module 40). Modules 40 move relative to a fixed housing structure, such as the walls of main portion 12M, and relative to each other as modules 40 move relative to each other along dimension X to accommodate different interpupillary distances for different users.

[0036] Figure 3 A schematic diagram of an illustrative electronic device such as a head-mounted device or other wearable device is shown in FIG. Figure 3Device 10 may operate as a standalone device and / or resources of device 10 may be used to communicate with external electronic equipment. For example, communication circuitry in device 10 may be used to send user input information, sensor information, and / or other information to external electronic devices (e.g., wirelessly or via a wired connection). Each of these external devices may include Figure 3 Components of the type shown in device 10.

[0037] like Figure 3 As shown, a head-mounted device such as device 10 may include control circuitry 20. The control circuitry 20 may include storage and processing circuitry for supporting the operation of the device 10. The storage and processing circuitry may include storage devices, such as non-volatile memory (e.g., flash memory or other electrically programmable read-only memory configured to form a solid-state drive), volatile memory (e.g., static or dynamic random access memory), and the like. The processing circuitry in the control circuitry 20 may be used to collect inputs from sensors and other input devices, and may be used to control output devices. The processing circuitry may be based on one or more microprocessors, microcontrollers, digital signal processors, baseband processors and other wireless communication circuitry, power management units, audio chips, application-specific integrated circuits, and the like. During operation, the control circuitry 20 may provide visual and other outputs to the user using the display 14 and other output devices.

[0038] To support communication between device 10 and external equipment, control circuitry 20 may communicate using communication circuitry 22. Circuitry 22 may include an antenna, radio frequency transceiver circuitry, and other wireless communication circuitry and / or wired communication circuitry. Circuitry 22 (which may sometimes be referred to as control circuitry and / or control and communication circuitry) may support two-way wireless communication via a wireless link between device 10 and external equipment (e.g., a companion device such as a computer, cellular phone, or other electronic device, an accessory such as a pointing device, a computer stylus, or other input device, a speaker, or other output device, etc.). For example, circuitry 22 may include radio frequency transceiver circuitry, such as a wireless local area network (WLAN) transceiver circuitry configured to support communication via a WLAN link, a near field communication (NFC) transceiver circuitry configured to support communication via a NFC link, a cellular phone transceiver circuitry configured to support communication via a cellular phone link, or a transceiver circuitry configured to support communication via any other appropriate wired or wireless communication link. For example, communication may be performed via a WLAN link. link, Wireless communications may be supported by a link, a wireless link operating at a frequency between 10 GHz and 400 GHz, a 60 GHz link or other millimeter wave link, a cellular telephone link, or other wireless communication link. Device 10 may include power circuitry for transmitting and / or receiving wired and / or wireless power, if desired, and may include a battery or other energy storage device. For example, device 10 may include a coil and a rectifier to receive wireless power for supplying to circuits in device 10.

[0039] Device 10 may include input-output devices, such as device 24. Input-output devices 24 may be used to collect user input, to collect information about the user's surroundings, and / or to provide output to the user. Device 24 may include one or more displays, such as display 14. Display 14 may include one or more display devices, such as an organic light emitting diode display panel (a panel having organic light emitting diode pixels formed on a polymer or silicon substrate that contains pixel control circuitry), a liquid crystal display panel, a microelectromechanical system display (e.g., a two-dimensional mirror array or scanning mirror display device), a display panel having an array of pixels formed from crystalline semiconductor light emitting diode dies (sometimes referred to as micro-LEDs), and / or other display devices.

[0040] The sensors 16 in the input-output device 24 may include force sensors (e.g., strain gauges, capacitive force sensors, resistive force sensors, etc.), audio sensors (such as microphones), touch and / or proximity sensors (such as capacitive sensors, such as touch sensors forming buttons, touchpads, or other input devices), and other sensors. If desired, sensors 16 may include optical sensors (such as optical sensors that emit and detect light), ultrasonic sensors, optical touch sensors, optical proximity sensors and / or other touch sensors and / or proximity sensors, monochrome and color ambient light sensors, image sensors, fingerprint sensors, iris scan sensors, retina scan sensors and other biometric sensors, temperature sensors, sensors for measuring three-dimensional contactless gestures (“air gestures”), pressure sensors, sensors for detecting position, orientation and / or motion (for example, accelerometers, magnetic sensors such as compass sensors, gyroscopes and / or inertial measurement units that include some or all of these sensors), health sensors such as blood oxygen sensors, heart rate sensors, blood flow sensors and / or other health sensors, radio frequency sensors, depth sensors (for example, structured light sensors and / or depth sensors based on stereoscopic imaging devices that capture three-dimensional images), optical sensors such as self-mixing sensors and light detection and ranging (laser) sensors that collect time-of-flight measurements, humidity sensors, wetness sensors, gaze tracking sensors, electromyography sensors that sense muscle activation, facial sensors and / or other sensors. In some arrangements, device 10 may use sensors 16 and / or other input-output devices to collect user input. For example, buttons may be used to collect button press input, a touch sensor overlying the display may be used to collect user touch screen input, a touchpad may be used to collect touch input, a microphone may be used to collect audio input, an accelerometer may be used to detect when a finger contacts an input surface and thus may be used to collect finger press input, etc.

[0041] If desired, the electronic device 10 may include additional components (see, for example, other devices 18 in input-output devices 24). Additional components may include tactile output devices, actuators for moving the movable housing structure, audio output devices such as speakers, light-emitting diodes for status indicators, light sources such as light-emitting diodes for illuminating portions of the housing and / or display structure, other optical output devices, and / or other circuits for collecting input and / or providing output. The device 10 may also include a battery or other energy storage device, a connector port for supporting wired communication with auxiliary equipment and for receiving wired power, and other circuitry.

[0042] Figure 4 is a top view of an exemplary gaze tracking circuit 62. Figure 4As shown, gaze tracking circuitry 62 may include one or more cameras (such as camera 42), one or more light sources (such as light source 44), and one or more rangefinders (such as rangefinder 48). Rangefinder 48 may be configured to measure the distance to eye 58 (e.g., the distance to a specular reflection point on the cornea, sometimes referred to as interocular distance). As the user's gaze moves around (e.g., from position P1 to position P2), the distance to eye 58 may change. At position P1, rangefinder 48 may measure distance D1 to eye 58. At position P2, rangefinder 48 may measure distance D2 to eye 58.

[0043] Rangefinder 48 may include one or more transmitters, such as transmitter 48T, and one or more receivers, such as receiver 48R. Transmitter 48T may be configured to transmit a signal 56 toward a user's eye 58. Signal 56 may reflect from eye 58, and reflected signal 54 may be detected by receiver 48R. If desired, rangefinder 48 may include more than one transmitter 48T and / or more than one receiver 48R. For example, rangefinder 48 may include a second transmitter 48T and a second receiver 48R for redundancy. Arrangements in which rangefinder 48 includes three or more transmitters 48T and / or three or more receivers 48R may also be used.

[0044] In some arrangements, a single device may function as both transmitter 48T and receiver 48R. For example, a flexible membrane in a transducer may be used to detect ultrasonic signals (when used as receiver 48R) and also to transmit ultrasonic signals (when used as transmitter 48T). As another example, a self-mixing interferometer may function as both transmitter 48T and receiver 48R.

[0045] The rangefinder 48 can be any suitable sensor configured to measure distance. In an arrangement where the rangefinder 48 is an optical sensor (e.g., an optical sensor that collects time-of-flight measurements, a self-mixing sensor, a light detection and ranging (laser) sensor, a structured light sensor, a phase-based optical coherence tomography sensor, and / or a depth sensor based on a stereoscopic imaging device that captures three-dimensional images, etc.), the transmitted signal 56 and the reflected signal 54 can be optical signals. When the rangefinder 48 is formed by a phase-based sensor (such as an optical sensor based on optical coherence tomography), the rangefinder 48 can be configured to achieve a smaller resolvable time interval than a time-of-flight-based sensor. In an arrangement where the rangefinder 48 is an ultrasonic sensor (e.g., one or more capacitive micromachined ultrasonic transducers, piezoelectric micromachined transducers, and / or other suitable ultrasonic transducers for transmitting and / or detecting acoustic signals), the transmitted signal 56 and the reflected signal 54 can be ultrasonic signals.

[0046] If desired, transmitter 48T may be co-located with receiver 48R. In other arrangements, transmitter 48T and receiver 48R may be mounted in different locations. Camera 42 and light source 44 may be co-located with each other or may be mounted in different locations. One or both of transmitter 48T and receiver 48R may be co-located with camera 42 and / or light source 44, or transmitter 48T and receiver 48R may be mounted separately from camera 42 and / or light source 44.

[0047] During operation, light source 44 may be used to emit light 50 toward eye 58. Light 50 may reflect from eye 58, and reflected light 52 may be detected by camera 42. Emitted light 50 may produce a glint on eye 58. Camera 42 may capture an image of eye 58 including the glint produced by light 50. Based on the captured image, gaze tracking circuitry 62 may determine the location of the glint and the location of the user's pupil. In some arrangements, there may be multiple light sources 44 that produce multiple glints on the user's eye. If a sufficient number of glints are produced on eye 58, gaze control circuitry 62 may determine the shape and / or position of the user's eye or a portion of the user's eye (e.g., the user's cornea), which in turn may be used to determine gaze direction (e.g., without requiring rangefinder 48).

[0048] In some arrangements, such as when fewer light sources 44 are desired, or when a sufficient number of flashes cannot be captured due to the shape of a particular user's eyes, the gaze control circuitry 62 may combine flash detection with distance sensing to determine gaze direction. In this type of scenario, a single light source 44 may produce a single flash on the eye 58, and the camera 42 may capture an image of the eye 58 that includes the single flash. Based on the captured image, the gaze tracking control circuitry 62 may determine the location of the flash and the location of the pupil. Because the eye is mostly spherical (e.g., first order), the flash on the eye 58 will mostly remain in the same location as the eye moves around, but the position of the pupil relative to the flash will change as the gaze direction changes. In particular, as the eye moves around to different gaze directions (e.g., from position P1 to position P2), the position of the pupil relative to the flash will change by a scaling factor that depends on the distance to the eye. By using rangefinder 48 to determine the distance to the eye at positions P1 and P2, gaze tracking circuitry 62 can determine the scaling factor and, therefore, map the pupil and glint position at P1 to a first gaze direction (e.g., based on distance D1), and map the pupil and glint position at P2 to a second gaze direction (e.g., based on distance D2). In scenes where no glint is visible to camera 42, multiple rangefinders 48 or a single rangefinder 48 can be combined with image information from camera 42 (e.g., eye features and / or facial features), which together enable an accurate assessment of the position and orientation of eye 58.

[0049] Figure 5 is to use phase-based optical coherence tomography to determine the distance to the user's eyes (e.g., as combined with Figure 4 Schematic diagram of an exemplary rangefinder 48 as discussed above. Figure 5 As shown, rangefinder 48 may include one or more light sources such as transmitter 48T. Transmitter 48T may include one or more coherent light sources such as coherent light source 74 on an integrated circuit such as integrated circuit die 72. Coherent light source 74 may include one or more lasers such as lasers 74A, 74B, 74C, and 74D. Laser 74 may be a vertical cavity surface emitting laser, a distributed feedback (DFB) laser, a distributed Bragg reflector (DBR) laser, a microelectromechanical system (MEMs) tunable laser, a thermally tuned laser, or any other suitable coherent light source. Laser 74 may be a wavelength tunable laser that is configured to emit light while sweeping the wavelength of the light over a given tunability range. For example, laser 74 may be configured to emit light centered at 905 nm with 1.8 nm of tunability (e.g., from 904.1 nm to 905.9 nm). This example is illustrative only. If desired, the light emitted from laser 74 may be centered at other wavelengths and / or may have other ranges of tunability.

[0050] Each laser 74 may emit a laser chirp having a given swept frequency into a corresponding input of a beam splitter, such as beam splitter 102. Beam splitter 102 may be a planar lightwave circuit (e.g., one or more passive optical components) or other suitable beam splitter for splitting light from laser 72 into signal light, such as signal light 56, and reference light, such as reference light 106. Laser 72A may emit light into input 76-1 of beam splitter 102, laser 72B may emit light into input 76-2 of beam splitter 102, laser 74C may emit light into input 76-3 of beam splitter 102, and laser 74D may emit light into input 76-4 of beam splitter 102.

[0051] Beam splitter 102 can be configured to split the light from laser 74 into a signal arm directed toward eye 58 and a local oscillator arm that will interfere with the light collected from the signal arm. Figure 5As shown, beam splitter 102 can direct half of the light from laser 74A to path 84-1 and the other half to path 104-1; can direct half of the light from laser 74B to path 84-2 and the other half to path 104-2; can direct half of the light from laser 74C to path 84-3 and the other half to path 104-3; and can direct half of the light from laser 84D to path 84-4 and the other half to path 104-4. The light directed to paths 104-1, 104-2, 104-3, and 104-4 can be used as reference light (e.g., in a local oscillator arm forming a fixed reference path), while the light directed to paths 84-1, 84-2, 84-3, and 84-4 can be used as signal light (e.g., in a signal arm forming a free-space path to eye 58).

[0052] Paths 104-1, 104-2, 104-3, and 104-4 can serve as inputs to an optical coupler, such as a star coupler 82, for coupling reference light 106 from corresponding lasers 74 to one or more sensors, such as image sensor 70. Star coupler 82 can have one or more outputs, such as output 78. If desired, star coupler 82 can have more outputs than inputs. For example, star coupler 82 can have four inputs and twelve outputs, eight inputs and twelve outputs, two inputs and ten outputs, and / or any other suitable number of inputs and outputs. Star coupler 82 can be configured to couple reference light 106 from inputs 104-1, 104-2, 104-3, and 104-4 to output 78. Output 78 can direct reference light 106 toward one or more image sensors 70. There can be one, two, three, ten, twelve, more than twelve, or fewer than twelve image sensors 70 in rangefinder 48. Each image sensor 70 may have a two-dimensional array of pixels.If desired, the star coupler 82 may include an additional output 78 coupled to a wavelength reference loop 80 (eg, for measuring the actual wavelength of each laser chirp).

[0053] Paths 84-1, 84-2, 84-3, and 84-4 can be configured to couple signal light 56 to a given emission aperture, such as emission apertures 68-1 and 68-2 (sometimes referred to as exit apertures). If desired, rangefinder 48 can have more than two emission apertures (e.g., three, four, or more than four emission apertures). Emission apertures 68-1 and 68-2 can be positioned at different locations relative to eye 58. Signal light 56 from paths 84-1 and 84-2 is coupled out of aperture 68-1 toward eye 58. Signal light 56 from paths 84-3 and 84-4 is coupled out of aperture 68-2 toward eye 58. Signal light 56 generates a specular reflection (e.g., a glint) at the user's eye, and the reflected signal light 54 is reflected toward one or more sensors 70, where it is combined with reference light 106.

[0054] Each image sensor 70 can capture an image of a flash of light produced on the eye 58 by the signal light 56 from a given laser 74. If the free space path length to the eye 58 is equal to the reference path length, then there will be no change in the image of the flash of light captured by the image sensor 70. If the two path lengths are different, constructive interference will occur at some frequencies of light, while destructive interference will occur at other frequencies of light. The frequency at which the signal modulates between constructive and destructive interference is proportional to the difference in path length between the free space path reflected off the eye 58 and a fixed reference path of known path length. In this way, the distance to the eye 58 (e.g., Figure 4 The distance D1 or the distance D2) may be determined based on an interference pattern (eg, a pattern of fringes) generated on the image sensor 70.

[0055] exist Figure 5 In the example of FIG, the rangefinder 48 includes a first emitter 48T and a second emitter 48T, each of which includes a first laser and a second laser. For example, the first emitter 48T-1 having an exit aperture 68-1 may include laser 74A and laser 74B, and the second emitter 48T-2 having an exit aperture 68-2 may include laser 74C and laser 74D. Using multiple emitters 48T can allow for diversity in path length measurement. If desired, there can be three, four, or more than four emitters 48T. Some of the emitters 48T can be used for direct path measurement, in which the emitters 48T emit light directly toward the sensor 70 on a free-space path (e.g., without reflecting off the eye 58) to monitor changes in the shape of the eyewear 10 that may be worn and removed daily. In an arrangement with four emitters 48T, for example, the rangefinder 48 can be configured to determine the x, y, and z position coordinates of each emitter 48T using direct path measurement.

[0056] Each transmitter 48T consists of two lasers Figure 5 The example of is merely illustrative. If desired, each emitter 48T may include three, four, or more than four lasers, or may include only one laser. Using multiple lasers 74 in each emitter 48T allows for sequential emission of multiple laser chirps during a given measurement while allowing the laser 74 to have a rest period, such as 1 ms, after firing (e.g., for thermal management). Emitting multiple laser chirps in a given measurement allows the first of the chirps to be used to determine flash location, and subsequent chirps to be used to determine eye distance and its preceding one or two temporal derivatives (eye velocity and eye acceleration).

[0057] For example, during a given measurement, laser 74A may fire a first chirp with a first sweep frequency (e.g., a sweep from 905.9 nm to 904.1 nm) over a first sweep duration (e.g., 2 ms or other suitable duration). Beam splitter 102 may split the first chirp from laser 74A into signal light 56 that exits aperture 68-1 and reference light 106 that exits output 78 toward sensor 70. Image sensor 70 may capture one or more images of the reflected signal light 54 and reference light 106. Based on the captured images, rangefinder 48 may determine the location of the flash (e.g., it may determine which set of pixels on image sensor 70 includes the flash). This allows rangefinder 48 to read out only the specific set of pixels where the flash was located during subsequent chirps. For example, if sensor 70 includes a 50-pixel by 50-pixel array, a subset area of 2 pixels by 4 pixels (corresponding to the flash location) may be read out in subsequent chirps for measuring path length and its derivative (as an example).

[0058] After the first chirp, laser 74B may fire a second chirp with a second swept frequency (e.g., from 904.1 nm to 905.9 nm) over a second sweep duration different from the first sweep duration (e.g., 0.5 ms or other suitable duration). Beam splitter 102 may split the first chirp from laser 74B into signal light 56 exiting aperture 68-1 and reference light 106 exiting output 78 toward sensor 70. Image sensor 70 may capture one or more images of the reflected signal light 54 and reference light 106. If desired, only a subset of pixels where the flash occurred (e.g., the subset of pixels identified from the first chirp) may be read out to measure the second chirp. Based on the captured images, rangefinder 48 may determine the free-space path length and the corresponding distance to eye 58.

[0059] After the second chirp, laser 74A may fire again with a third chirp having a third swept frequency (e.g., from 905.9 nm to 904.1 nm) over a third sweep duration different from the first sweep duration (e.g., 1.5 ms or other suitable duration). Beam splitter 102 may split the third chirp from laser 74A into signal light 56 that exits aperture 68-1 and reference light 106 that exits output 78 toward sensor 70. Image sensor 70 may capture one or more images of the reflected signal light 54 and reference light 106. If desired, only a subset of pixels where the flashes occurred (e.g., the subset of pixels identified from the first chirp) may be read out to measure the third chirp. Based on the captured images, rangefinder 48 may determine a first time derivative of the free-space path length (e.g., eye velocity).

[0060] After the third chirp, laser 74B may fire again with a fourth chirp having a fourth swept frequency (e.g., from 904.1 nm to 905.9 nm) over a fourth sweep duration different from the first sweep duration (e.g., 0.5 ms or other suitable duration). Beam splitter 102 may split the fourth chirp from laser 74B into signal light 56 that exits aperture 68-1 and reference light 106 that exits output 78 toward sensor 70. Image sensor 70 may capture one or more images of the reflected signal light 54 and reference light 106. If desired, only a subset of pixels where the flash occurred (e.g., the subset of pixels identified from the first chirp) may be read out to measure the fourth chirp. Based on the captured images, rangefinder 48 may determine a second time derivative of the free-space path length (e.g., eye acceleration).

[0061] In an arrangement where the rangefinder 48 includes multiple transmitters 48T, the transmitters 48T may be active in alternating measurements. For example, the lasers 74A and 74B of transmitter 48T-1 may fire four chirps as described above in a first measurement, while the lasers 74C and 74D of transmitter 48T-2 may be inactive (e.g., not emitting light). In a second, subsequent measurement, the lasers 74C and 74D of transmitter 48T-2 may fire four chirps (as described above for lasers 74A and 74B), while the lasers 74A and 74B of transmitter 48T-1 may not emit light.

[0062] If desired, the rangefinder 48 can use a signal processing technique such as PROFIT to process the signal from the sensor 70. Using this type of technique, the rangefinder 48 can determine the frequency of an isolated sinusoidal signal by taking N+1 data samples and then performing a Fast Fourier Transform on the first N samples and the last N samples. Each will show a peak around the location of the sinusoidal signal, but with a different phase. The frequency of the signal can be estimated based solely on the phase difference (e.g., rather than using the amplitude in the frequency bins outside the peak). However, this is merely illustrative. If desired, other signal processing techniques can be used to process the signal from the image sensor 70.

[0063] Figure 6 is a rear view of the device 10, showing Figure 5 illustrative locations of some of the components of . Image sensor 70 and emission apertures 68-1 and 68-2 may face the eye zone and may be mounted to a support structure such as support structure 66. Support structure 66 may be an annular frame member that extends around the perimeter of the user's eyes. Support structure 66 may be part of housing 12 of device 10, part of optical module 40 of device 10, and / or may be a dedicated support structure for gaze tracking circuitry 62. In arrangements where device 10 includes a display such as display 14, support structure 66 may extend around some or all of the perimeter of display 14, or support structure 66 may be mounted elsewhere in device 10 (e.g., surrounding a portion of a housing 12 of device 10). Figure 1 Lens 30, surrounded by Figure 1 The support structure 66 may have a hole for allowing the emitted signal light 56 from the emission holes 68 - 1 and 68 - 2 to exit the support structure 66 , and may include a hole for allowing the reflected signal light 54 to be detected by the image sensor 70 .

[0064] The rangefinder 48 may include one or more optical fibers, such as optical fibers 64 in the support structure 66. The optical fibers 64 may be configured to direct laser light (e.g., via total internal reflection) from the beam splitter 102 to a desired location within the support structure 66. For example, a first set of optical fibers 64 may be configured to direct reference light 106 from the beam splitter 102 (e.g., from a given output 78 of the star coupler 82) to a corresponding image sensor 70. A second set of optical fibers 64 may be configured to direct signal light 56 from the beam splitter 102 (e.g., from a corresponding path among paths 84-1, 84-2, 84-3, and 84-4) to a corresponding emission aperture, such as exit apertures 68-1 and 68-2. The reflected signal light 54 may interfere with the reference light 106, and the corresponding interference pattern captured by the image sensor 70 may be analyzed to determine a flash location, eye rate, eye velocity, and / or eye acceleration, as combined with Figure 5 discussed.

[0065] Figure 7 FIG is a side view of an exemplary image sensor 70 that may be used in the rangefinder 48. Figure 7 As shown, the image sensor 70 may include a two-dimensional array of pixels 96 on a substrate such as a semiconductor substrate 94. The image sensor 70 may be a complementary metal oxide semiconductor (CMOS) image sensor or other suitable image sensor. If desired, the image sensor 70 may include one or more additional optical components overlapping the pixels 96, such as a lens 86, a phase grating 88, and a holographic optical element 90. The lens 86 may be used to focus the reflected signal light 54 onto the pixels 96. The phase grating 88 (sometimes referred to as a diffraction grating 88) may be interposed between the holographic optical element 90 and the lens 86.

[0066] The holographic optical element 90 can be configured to couple the reference light out of the output 78 at an angle generally parallel to the incident, reflected signal light 54. The holographic optical element 90 can be a relatively sparse hologram with a small duty cycle (e.g., a metallic pattern with a 5% density or other suitable density). Making the reference wavefront parallel or nearly parallel to the signal wavefront reduces the spatial frequency of fringes between the two optical paths and helps ensure that the interference pattern can be captured by the pixel 96 even when the pixel 96 is not too small (e.g., 3 microns by 3 microns or other suitable size). Allowing relatively large pixels can help reduce the number of pixels that need to be digitized, thereby saving power.

[0067] Phase grating 88 can be configured to separate specular reflections (e.g., glint) from eye 58 into two spots on image pixel 96. Depending on whether the free-space path is longer or shorter than the fixed reference path, one of the spots will lead the other (e.g., by 160 degrees of phase difference). Thus, rangefinder 48 can disambiguate the sign (e.g., positive or negative) associated with the difference in path length between the free-space path and the fixed reference path. For example, if rangefinder 48 measures a one-millimeter difference in path length, phase grating 88 can be used to determine whether the free-space path is one millimeter shorter or one millimeter longer than the fixed reference path based on which spot leads the other spot on pixel 96.

[0068] Figure 8 8 is a side view of the phase grating 88 and lens 86 of the rangefinder 48. The phase grating 88 can be a surface relief grating formed by modulating the thickness of a medium (e.g., having ridges 108 and grooves 110 forming stripes in the medium). If desired, the phase grating 88 can be formed from two different types of plastic with known etch depths and known refractive indices. The height H of the ridges 108 relative to the grooves 110 can be equal to approximately half the wavelength of the signal light, so that one of the spots generated on the pixel 96 leads the other.

[0069] like Figure 8 As shown, lens 86 can be laterally offset from the edges of phase grating 88. Specifically, the vertex of lens 86 is laterally offset from all edges of phase grating 88, such as edge 112 of ridge 108. This lateral offset between lens 86 and phase grating 88 can be tuned to produce asymmetry, where one of the spots on pixel 96 leads the other, as shown in FIG. Figure 9 As shown. Figure 9 As shown, Figure 8 Phase grating 88 can separate reflected signal light 54 into two images, such as flash spot 98-1 and flash spot 98-2. Depending on whether the free-space path is longer or shorter than the fixed reference path, one of flash spots 98-1 and 98-2 will lead the other. For example, when the free-space path length is longer than the fixed reference path length, flash spot 98-1 may lead flash spot 98-2, while when the free-space path length is shorter than the fixed reference path length, flash spot 98-2 may lead flash spot 98-1. Rangefinder 48 can therefore disambiguate the sign (e.g., positive or negative) associated with the calculated path length difference based on which of flash spots 98-1 and 98-2 leads the other.

[0070] As described above, one aspect of the present technology is to collect and use information, such as information from input-output devices. The present disclosure contemplates that, in some cases, data including personal information data may be collected that uniquely identifies or can be used to contact or locate a specific person. Such personal information data may include demographic data, location-based data, phone numbers, email addresses, social media information, home addresses, data or records related to the user's health or fitness level (e.g., vital sign measurements, medication information, exercise information), date of birth, username, password, biometric information, or any other identifier or personal information.

[0071] This disclosure recognizes that the use of such personal information in the present technology can be used to benefit users. For example, this personal information data can be used to deliver targeted content that is of particular interest to the user. Thus, the use of such personal information data enables users to exercise planned control over the content delivered. Furthermore, this disclosure contemplates other uses of personal information data that can benefit users. For example, health and fitness data can be used to provide insights into a user's overall health or as positive feedback to individuals using technology to pursue health goals.

[0072] This disclosure anticipates that entities responsible for the collection, analysis, disclosure, transmission, storage, or other use of such personal information data will adhere to robust privacy policies and / or privacy practices. Specifically, such entities should implement and adhere to privacy policies and practices that are recognized as meeting or exceeding industry or government requirements for maintaining the privacy and security of personal information data. Such policies should be easily accessible to users and updated as the collection and / or use of data changes. Personal information from users should be collected for legitimate and reasonable entity purposes and should not be shared or sold outside of those legitimate purposes. Furthermore, such collection / sharing should be conducted after receiving the user's informed consent. Furthermore, such entities should consider taking any necessary steps to protect and safeguard access to such personal information data and ensure that other entities with access to personal information data comply with the other entity's privacy policies and procedures. Furthermore, such entities may subject themselves to third-party assessments to demonstrate compliance with widely accepted privacy policies and practices. Furthermore, policies and practices should be tailored to the specific type of personal information data collected and / or accessed, as well as to applicable laws and standards, including jurisdictional considerations. For example, in the United States, the collection or access of certain health data may be governed by federal and / or state laws such as the Health Insurance Portability and Accountability Act (HIPAA), while health data in other countries may be subject to other regulations and policies and should be handled accordingly. Therefore, different privacy measures should be advocated for different types of personal data in each country.

[0073] Notwithstanding the foregoing, the present disclosure also contemplates implementation schemes in which users selectively block the use or access of personal information data. That is, the present disclosure contemplates providing hardware elements and / or software elements to prevent or block access to such personal information data. For example, the technology of the present invention may be configured to allow users to “opt in” or “opt out” of collecting personal information data at any time during or after registration for a service. In another example, a user may choose not to provide certain types of user data. In yet another example, a user may choose to limit the length of time that user-specific data is maintained. In addition to providing “opt-in” and “opt-out” options, the present disclosure also contemplates providing notifications related to access or use of personal information. For example, a user may be notified that their personal information data will be accessed when downloading an application (“app”), and the user may be subsequently reminded again before the personal information data is accessed by the app.

[0074] Furthermore, it is the intention of this disclosure that personal information data should be managed and processed in a manner that minimizes the risk of unintentional or unauthorized access or use. Risks can be minimized by limiting the collection of data and deleting data once it is no longer needed. In addition, and when applicable, including in certain health-related applications, data de-identification can be used to protect the privacy of users. Where appropriate, de-identification can be facilitated by removing specific identifiers (e.g., date of birth, etc.), controlling the amount or characteristics of stored data (e.g., collecting location data at a city level rather than an address level), controlling how data is stored (e.g., aggregating data across users), and / or other methods.

[0075] Thus, while this disclosure broadly encompasses the use of information that may include personal information data to implement one or more of the various disclosed embodiments, this disclosure also contemplates that various embodiments may be implemented without requiring access to personal information data. That is, various embodiments of the present technology will not be unable to function properly due to the absence of all or a portion of such personal information data.

[0076] Physical Environment: The physical environment refers to the physical world that people can sense and / or interact with without the aid of electronic systems. A physical environment, such as a physical park, includes physical objects, such as physical trees, physical buildings, and physical people. People can directly sense and / or interact with the physical environment, such as through sight, touch, hearing, taste, and smell.

[0077] Computer-generated reality: In contrast, a computer-generated reality (CGR) environment refers to a fully or partially simulated environment that people perceive and / or interact with via electronic systems. In CGR, a subset of a person's physical movements, or representations thereof, is tracked, and in response, one or more characteristics of one or more virtual objects simulated in the CGR environment are adjusted in a manner consistent with at least one law of physics. For example, a CGR system can detect a person's head rotation and, in response, adjust the graphical content and sound field presented to the person in a manner similar to how such views and sounds change in a physical environment. In some cases (e.g., for accessibility reasons), adjustments to the characteristics of virtual objects in a CGR environment can be made in response to representations of physical movement (e.g., voice commands). People can sense and / or interact with CGR objects using any of their senses, including vision, hearing, touch, taste, and smell. For example, a person can sense and / or interact with audio objects that create a 3D or spatial audio environment that provides the perception of a point audio source in 3D space. As another example, audio objects can enable audio transparency, which selectively introduces ambient sounds from the physical environment with or without computer-generated audio. In some CGR environments, people can sense and / or interact only with audio objects. Examples of CGR include virtual reality and mixed reality.

[0078] Virtual Reality: A virtual reality (VR) environment is a simulated environment designed to be based entirely on computer-generated sensory input to one or more senses. A VR environment includes multiple virtual objects that a person can sense and / or interact with. For example, trees, buildings, and computer-generated images representing human avatars are examples of virtual objects. A person can sense and / or interact with virtual objects in a VR environment through the simulation of the person's presence within the computer-generated environment and / or through the simulation of a subset of the person's physical movement within the computer-generated environment.

[0079] Mixed Reality: In contrast to VR environments, which are designed to be based entirely on computer-generated sensory input, mixed reality (MR) environments are simulated environments designed to incorporate sensory input from the physical environment, or representations thereof, in addition to computer-generated sensory input (e.g., virtual objects). On the virtuality continuum, a mixed reality environment is anything between, but not including, a fully physical environment at one end and a virtual reality environment at the other. In some MR environments, computer-generated sensory input can respond to changes in sensory input from the physical environment. Additionally, some electronic systems used to render MR environments can track position and / or orientation relative to the physical environment to enable virtual objects to interact with real objects (i.e., physical items from the physical environment, or representations thereof). For example, the system can cause virtual trees to appear stationary relative to the physical ground. Examples of mixed reality include augmented reality and augmented virtuality. Augmented Reality: An augmented reality (AR) environment is a simulated environment in which one or more virtual objects are superimposed on a physical environment, or a representation of a physical environment. For example, the electronic system used to render an AR environment may have a transparent or semi-transparent display through which a person can directly view the physical environment. The system can be configured to present virtual objects on a transparent or translucent display, allowing a person using the system to perceive the virtual objects superimposed on a physical environment. Alternatively, the system can have an opaque display and one or more imaging sensors that capture images or videos of the physical environment, which are representations of the physical environment. The system combines the images or videos with the virtual objects and presents the combination on the opaque display. A person uses the system to indirectly view the physical environment via the images or videos of the physical environment and perceive the virtual objects superimposed on the physical environment. As used herein, a video of the physical environment displayed on an opaque display is referred to as "transparent video," meaning that the system uses one or more image sensors to capture images of the physical environment and uses those images when presenting the AR environment on the opaque display. Further alternatively, the system can have a projection system that projects virtual objects into the physical environment, for example as holograms or on a physical surface, allowing a person using the system to perceive the virtual objects superimposed on the physical environment. An augmented reality environment also refers to a simulated environment in which a representation of the physical environment is transformed by computer-generated sensory information. For example, in providing pass-through video, the system can transform one or more sensor images to apply a selected perspective (e.g., a viewpoint) that is different from the perspective captured by the imaging sensor. For another example, a representation of a physical environment can be transformed by graphically modifying (e.g., enlarging) a portion thereof, such that the modified portion is a representative, but not a true, version of the original captured image. For another example, a representation of a physical environment can be transformed by graphically eliminating or blurring a portion thereof.Augmented Virtual: An augmented virtual (AV) environment is a simulated environment in which a virtual or computer-generated environment incorporates one or more sensory inputs from the physical environment. The sensory input can be a representation of one or more characteristics of the physical environment. For example, an AV park can have virtual trees and virtual buildings, but people's faces are realistically reproduced from images taken of physical people. In another example, a virtual object can adopt the shape or color of a physical object imaged by one or more imaging sensors. In another example, a virtual object can adopt a shadow that conforms to the position of the sun in the physical environment.

[0080] Hardware: There are many different types of electronic systems that enable a person to sense and / or interact with various CGR environments. Examples include head-mounted systems, projection-based systems, heads-up displays (HUDs), vehicle windshields with integrated display capabilities, windows with integrated display capabilities, displays formed as lenses designed to be placed on a person's eyes (e.g., similar to contact lenses), headphones / earpieces, speaker arrays, input systems (e.g., wearable or handheld controllers with or without haptic feedback), smartphones, tablet computers, and desktop / laptop computers. A head-mounted system may have one or more speakers and an integrated opaque display. Alternatively, a head-mounted system may be configured to accept an external opaque display (e.g., a smartphone). A head-mounted system may incorporate one or more imaging sensors for capturing images or video of the physical environment, and / or one or more microphones for capturing audio of the physical environment. Instead of an opaque display, a head-mounted system may have a transparent or translucent display. A transparent or translucent display may have a medium through which light representing the image is directed to the person's eyes. The display can utilize digital light projection, OLED, LED, μLED, liquid crystal on silicon, laser scanning light source, or any combination of these technologies. The medium can be an optical waveguide, a holographic medium, an optical combiner, an optical reflector, or any combination thereof. In one embodiment, a transparent or translucent display can be configured to selectively become opaque. Projection-based systems can employ retinal projection technology that projects graphic images onto a person's retina. Projection systems can also be configured to project virtual objects into a physical environment, for example, as a hologram or onto a physical surface.

[0081] According to one embodiment, an eyewear is provided, comprising: a support structure; a first lens and a second lens, the first lens and the second lens being mounted to the support structure; and a gaze tracking circuit, the gaze tracking circuit being configured to track gaze direction, wherein the gaze tracking circuit comprises: a coherent light source, the coherent light source being configured to emit infrared light; a beam splitter, the beam splitter being configured to split the infrared light into signal light that travels a free space path and reference light that travels a fixed reference path, wherein the signal light produces eye glint; and an image sensor, the image sensor being configured to capture an image of the reference light and the signal light including the eye glint, the eye distance being determined based on the captured image.

[0082] According to another embodiment, the coherent light source optionally comprises a vertical cavity surface emitting laser, and the infrared light optionally sweeps across a range of wavelengths during emission.

[0083] According to another embodiment, the beam splitter optionally includes a planar lightwave circuit that splits the infrared light into the signal light and the reference light.

[0084] According to another embodiment, the beam splitter optionally includes a star coupler having an input for receiving the reference light and an output for directing the reference light toward the image sensor.

[0085] According to another embodiment, the image sensor optionally comprises a two-dimensional array of pixels.

[0086] According to another embodiment, the image sensor optionally includes a lens overlying the two-dimensional array of pixels.

[0087] According to another embodiment, the image sensor optionally includes a phase grating interposed between the two-dimensional array of pixels and the lens, and the vertex of the lens is laterally offset from all edges of the phase grating.

[0088] According to another embodiment, the image sensor optionally includes a holographic optical element interposed between the phase grating and the array of pixels, wherein the holographic optical element is configured to redirect the reference light to be parallel to the signal light.

[0089] According to another embodiment, the coherent light source and the image sensor optionally have different positions.

[0090] According to another embodiment, the image sensor is optionally one of a plurality of image sensors that detect the signal light and the reference light from the coherent light source.

[0091] According to one embodiment, a distance sensor is provided, comprising: a first emitter and a second emitter, wherein each emitter comprises a first laser that fires a first chirp and a second laser that fires a second chirp after the first chirp; a beam splitter that separates the first chirp and the second chirp into signal light and reference light, wherein the signal light travels a free space path length and produces a mirror reflection when the reference light travels a fixed reference path length; and an image sensor configured to capture images of the signal light and the reference light of the first chirp and the second chirp, wherein a position of the mirror reflection is determined based on the first chirp, and wherein a difference between the free space path length and the fixed reference path length is determined based on the second chirp.

[0092] According to another embodiment, the first transmitter and the second transmitter are optionally configured to be active during alternating measurements.

[0093] According to another embodiment, the image sensor optionally includes a pixel array and a lens overlapping the pixel array.

[0094] According to another embodiment, the distance sensor optionally includes a phase grating, which is interposed between the lens and the pixel array, wherein the phase grating is configured to produce a double image of the mirror reflection on the pixel array, the double image indicating whether the free space path length is longer or shorter than the fixed reference path length.

[0095] According to another embodiment, the distance sensor optionally includes a holographic optical element interposed between the lens and the pixel array, wherein the holographic optical element is configured to redirect the reference light to be parallel to the signal light.

[0096] According to one embodiment, a head-mounted device is provided, comprising: a display configured to present an image; a lens through which the image can be viewed from an eye-viewing zone; an image sensor distributed around the periphery of the display; and a first emitter and a second emitter, wherein the first emitter and the second emitter are configured to emit coherent infrared light, the coherent infrared light being divided into signal light heading toward the eye-viewing zone and reference light heading toward the image sensor, wherein the signal light is mirror-reflected at the eye-viewing zone, and wherein the mirror-reflected signal light is combined with the reference light to produce a corresponding interference pattern, which is captured by the image sensor and used to determine the eye distance.

[0097] According to another embodiment, each of the first emitter and the second emitter optionally includes at least a first laser and a second laser, and the head-mounted device optionally includes a first optical fiber that guides the reference light from the first laser and the second laser to the image sensor and a second optical fiber that guides the signal light from the first laser and the second laser to corresponding exit holes facing the eye-adaptive zone.

[0098] According to another embodiment, the image sensors each optionally include an array of pixels, wherein the first laser is configured to emit a first chirp of the coherent infrared light and the second laser is configured to emit a second chirp of the coherent infrared light after the first chirp, the first chirp being used to identify a portion of the array of pixels that includes a flash of light produced by the mirror-reflected signal light, and the second chirp being used to determine the interocular distance.

[0099] According to another embodiment, only the portion of the array of pixels comprising the flash is optionally read out for processing the second chirp.

[0100] According to another embodiment, at least one of the image sensors optionally includes: a pixel array; a lens, which overlaps with the pixel array; a phase grating, which is inserted between the pixel array and the lens; and a holographic optical element, which is inserted between the phase grating and the pixel array.

[0101] The foregoing is merely illustrative and various modifications may be made to the described embodiments. The foregoing embodiments may be implemented individually or in any combination.

Claims

1. An eyewear comprising: Support structure; a first lens and a second lens, the first lens and the second lens being mounted to the support structure; as well as A gaze tracking circuit configured to track a gaze direction, wherein the gaze tracking circuit comprises: a coherent light source configured to emit infrared light; a beam splitter configured to split the infrared light into a signal light that travels a free-space path and a reference light that travels a fixed reference path, wherein the signal light produces an eye glint; and An image sensor is configured to capture images of the reference light and the signal light including the eye glint, wherein the interocular distance is determined based on the captured images.

2. The eyewear of claim 1 , wherein the coherent light source comprises a vertical cavity surface emitting laser, and wherein the infrared light is swept across a range of wavelengths during emission.

3. The eyewear of claim 1 , wherein the beam splitter comprises a planar lightwave circuit that splits the infrared light into the signal light and the reference light.

4. The eyewear of claim 1 , wherein the beam splitter comprises a star coupler having an input for receiving the reference light and an output for directing the reference light toward the image sensor.

5. The eyewear of claim 1 , wherein the image sensor comprises a two-dimensional array of pixels.

6. The eyewear of claim 5, wherein the image sensor comprises a lens overlapping the two-dimensional array of pixels.

7. An eyewear according to claim 6, wherein the image sensor includes a phase grating interposed between the two-dimensional array of pixels and the lens, and wherein the vertex of the lens is laterally offset from all edges of the phase grating.

8. The eyewear of claim 7, wherein the image sensor comprises a holographic optical element interposed between the phase grating and the array of pixels, wherein the holographic optical element is configured to redirect the reference light to be parallel to the signal light.

9. The eyewear of claim 1 , wherein the coherent light source and the image sensor have different positions.

10. The eyewear of claim 1 , wherein the image sensor is one of a plurality of image sensors that detect the signal light and the reference light from the coherent light source.

11. A distance sensor, comprising: a first transmitter and a second transmitter, wherein each transmitter comprises a first laser that fires a first chirp and a second laser that fires a second chirp after the first chirp; a beam splitter that splits the first chirp and the second chirp into signal light and reference light, wherein the signal light travels a free space path length and generates specular reflection when the reference light travels a fixed reference path length; as well as an image sensor configured to capture images of the signal light and the reference light of the first chirp and the second chirp, wherein a position of the specular reflection is determined based on the first chirp, and wherein a difference between the free space path length and the fixed reference path length is determined based on the second chirp. 12 . The distance sensor of claim 11 , wherein the first transmitter and the second transmitter are configured to be active during alternating measurements. 13 . The distance sensor of claim 11 , wherein the image sensor comprises a pixel array and a lens overlapping the pixel array.

14. The distance sensor of claim 13 , further comprising a phase grating interposed between the lens and the pixel array, wherein the phase grating is configured to produce a double image of the mirror reflection on the pixel array, the double image indicating whether the free space path length is longer or shorter than the fixed reference path length. 15 . The distance sensor of claim 13 , further comprising a holographic optical element interposed between the lens and the pixel array, wherein the holographic optical element is configured to redirect the reference light to be parallel to the signal light.

16. A head-mounted device comprising: a display configured to present an image; a lens through which the image can be viewed from the eye zone; image sensors distributed around a periphery of the display; as well as A first emitter and a second emitter, wherein the first emitter and the second emitter are configured to emit coherent infrared light, wherein the coherent infrared light is divided into signal light heading toward the eye-pleasing zone and reference light heading toward the image sensor, wherein the signal light is mirror-reflected at the eye-pleasing zone, and wherein the mirror-reflected signal light is combined with the reference light to generate a corresponding interference pattern, and the corresponding interference pattern is captured by the image sensor and used to determine the eye distance.

17. The head-mounted device of claim 16 , wherein each of the first emitter and the second emitter comprises at least a first laser and a second laser, and the head-mounted device further comprises a first optical fiber that guides the reference light from the first laser and the second laser to the image sensor and a second optical fiber that guides the signal light from the first laser and the second laser to corresponding exit holes facing the eye-adaptive zone.

18. A head-mounted device according to claim 16, wherein the image sensors each include an array of pixels, wherein the first laser is configured to emit a first chirp of the coherent infrared light, and the second laser is configured to emit a second chirp of the coherent infrared light after the first chirp, wherein the first chirp is used to identify a portion of the array of pixels that includes a flash of light produced by the mirror-reflected signal light, and wherein the second chirp is used to determine the interocular distance.

19. A head-mounted device according to claim 18, wherein only the portion of the array of pixels that includes the flash is read out for processing the second chirp.

20. The head mounted device of claim 16, wherein at least one of the image sensors comprises: pixel array; a lens, the lens overlapping the pixel array; a phase grating interposed between the pixel array and the lens; as well as A holographic optical element is interposed between the phase grating and the pixel array.

Images