Eye tracking device with virtual light source

Abstract

A device may include a frame portion. A device may include a lens coupled to the frame portion. A device may include an illuminator coupled to the frame portion and configured to transmit electromagnetic radiation towards the lens, the lens including a reflective grating and a reflective coating. A device may include the reflective grating configured to redirect a portion of the electromagnetic radiation as redirected electromagnetic radiation towards an eye position via the reflective coating.

Description

EYE TRACKING DEVICE WITH VIRTUALLIGHT SOURCETECHNICAL FIELD

[0001] This description relates to an eye tracking device.BACKGROUND

[0002] Eye tracking is used extensively in augmented reality7, virtual reality , mixed reality, and medical applications. Eye trackers use a light source and a camera to measure eye positions and eye movements. Any combination of the position and shape of the pupil of the eye and the rotational position (gaze direction) of the eye may be used to track the eye.SUMMARY

[0003] The present disclosure describes ways to provide an efficient, more precise, compact eye tracking device that can be positioned on a side portion of a frame, such as a temple arm of a glasses frame. The eye tracking device includes an illuminator that emits an electromagnetic signal that, when reflected off a reflection grating on a surface of a lens, illuminates an eye of a user with at least two glints, or specular reflections off the cornea, from a single illuminator beam. The returned electromagnetic radiation may be reflected via a reflective coating on the lens back to a camera. In this way, it is possible to create a more accurate, efficient and / or compact eye tracker which may be integrated into a wide variety7of frame sides (e.g.. frame side portions), including temple arms of glasses.

[0004] In some aspects, the techniques described herein relate to a head mounted device, including: a frame portion; a lens coupled to the frame portion; and an illuminator coupled to the frame portion and configured to transmit electromagnetic radiation towards the lens, the lens including a reflective grating and a reflective coating, wherein the reflective grating is configured to redirect a portion of the electromagnetic radiation, as redirected electromagnetic radiation, towards an eye position via the reflective coating.

[0005] In some aspects, the techniques described herein relate to a method for eye tracking including: transmitting, via an illuminator, electromagnetic radiation, the illuminator coupled to a frame portion and configured to transmit the electromagnetic radiation towards a lens coupled to the frame portion; reflecting a portion of the electromagnetic radiationtowards an eye position via a reflective grating on the lens, as redirected electromagnetic radiation; and receiving a portion of the redirected electromagnetic radiation, as a returned electromagnetic radiation, reflected off the lens via a reflective coating.

[0006] In some implementations, the method may further comprise: coupling an illuminator to a frame portion, the illuminator being configured to transmit electromagnetic radiation towards an eye position via reflection at a lens; and coupling the lens to the frame portion, the lens comprising a reflective grating configured to redirect a portion of the electromagnetic radiation towards an eye position to generate a redirected electromagnetic radiation and a reflective coating; and coupling a camera to the frame portion, the camera being configured to receive a portion of the redirected electromagnetic radiation, as a returned electromagnetic radiation, reflected off the lens via a reflective coating.

[0007] In some aspects, the techniques described herein relate to a method for assembling an eye tracking device, including: coupling an illuminator to a frame portion, the illuminator being configured to transmit electromagnetic radiation towards an eye position via reflection at a lens; and coupling the lens to the frame portion, the lens including a reflective grating configured to redirect a portion of the electromagnetic radiation towards an eye position to generate a redirected electromagnetic radiation and a reflective coating; and coupling a camera to the frame portion, the camera being configured to receive a portion of the redirected electromagnetic radiation, as a returned electromagnetic radiation, reflected off the lens via a reflective coating.

[0008] In some aspects, the techniques described herein relate to an eye-tracking device, including: an illuminator configured to be coupled to a frame portion, the illuminator transmitting electromagnetic radiation and configured to transmit electromagnetic radiation towards a lens; and the lens configured to be coupled to a frame portion, the lens including a reflective coating and a reflective grating configured to redirect a portion of the electromagnetic radiation, as redirected electromagnetic radiation, towards an eye position.BRIEF DESCRIPTION OF THE DRAWINGS

[0009] FIG. 1 A depicts a front view of a head mounted device, according to implementations described throughout this disclosure.

[0010] FIG. IB depicts a rear view- of the head mounted device, according to implementations described throughout this disclosure.

[0011] FIG. 1C depicts a block diagram of a head mounted device, according toimplementations described throughout this disclosure.

[0012] FIG. 2A depicts a perspective view of an eye tracking device, according to implementations described throughout this disclosure.

[0013] FIG. 2B depicts a top view of the eye tracking device, according to implementations described throughout this disclosure.

[0014] FIG. 2C depicts a top view of the eye tracking device, according to implementations described throughout this disclosure.

[0015] FIG. 2D depicts a detail of a reflective grating, according to implementations.

[0016] FIG. 2E depicts a perspective view of the eye tracking device, according to implementations.

[0017] FIG. 2F depicts an image of an eye taken with an eye tracking device, according to implementations.

[0018] FIG. 3A depicts a method, according to implementations described throughout this disclosure.

[0019] FIG. 3B depicts a method, according to implementations described throughout this disclosure.DETAILED DESCRIPTION

[0020] The present disclosure describes an improved eye tracking device. Eye tracking devices comprise at least one illumination source and one camera operable to measure light emitted from the illumination source and reflected off an eye. From the image data, a position of the pupil in an image of the eye may be determined and used to identify a gaze direction of a user. The gaze direction information may be used to determine where in a head mounted display to place content, or as part of the computations to generate foveated rendering. In implementations of this disclosure, the image of the eye may be used in medical applications, training applications, and / or any other application.

[0021] Eye tracking can function in a desirable fashion when a clear image of the eye is available. For this reason, conventional eye tracking illuminators and sensing devices are mounted on the front frame portion of a frameset in front of a user’s eye, shining light directly on the eye. In head mounted eye tracking applications there is a need to be able to correct for movement of the head worn device relative to the user in order to maintain accurate gaze estimates during use.

[0022] In conventional methods, an eye facing camera is used to image a pupil center and multiple eye-facing illumination sources are each used to generate multiple spatiallyseparated glints (specular reflection off the cornea). The multiple glints, an estimated pupil center, and knowledge of the illumination source placements relative to one another, can be used to estimate the center of the comeal sphere, which can be used to estimate the center of the eyeball in 3D space. Regardless of whether an explicit estimation or a machine learning approach is used to estimate the center of the eyeball in 3D space, the additional information carried by the multiple glints improves algorithm performance and robustness.

[0023] The need for multiple spatially separated illumination sources adds significant complexity to the hardware system, however. The additional illumination sources create a need for additional long flexes, more signal lines, and additional transparent windows (such as NIR transparent windows) in a headset. In the case where the headset is a pair of smart glasses, each of these factors can significantly impact the form factor of frame due to the complicated design and required assembly. At least one technical problem is thus, how to get multiple glints for a high-performance eye tracking device without complicating the design, assembly, and form factor of the headset.

[0024] At least one of the technical solutions described in this disclosure is related to generation of at least one additional virtual illumination source based on a single illumination source. The addition of one or more virtual illumination sources may have at least the technical benefit of increasing the accuracy of the eye tracker without reducing the battery life of the glasses, complicating the form factor, and / or complicating the assembly of the glasses.

[0025] FIG. 1 A depicts a frontal view and FIG. IB depicts a rear view of a head mounted device 100, according to implementations. Head mounted device 100 may be implemented as smart glasses (e g., augmented reality, virtual reality, simulated reality, mixed reality, see-through reality, blended reality, or alternative reality glasses) configured to be worn on a head of a user. Head mounted device 100 may include display capability and computing (e.g., processing) capability. In some implementations, head mounted device 100 may comprise a virtual reality -type frameset.

[0026] Head mounted device 100 includes a frame portion coupled to an illuminator and a lens 110 with a reflective grating. The illuminator, which is positioned within a cavity 142 within an arm portion of a frameset 106 in FIG. 1 A, emits electromagnetic radiation towards the reflective grating on the lens, redirecting the electromagnetic radiation towards an eye position. Once reflected off the eye, the returned electromagnetic radiation can be imaged by a camera as an additional glint, thereby making the eye tracking with head mounted device 100 more precise.

[0027] Head mounted device 100 may include a frame portion. In some implementations, the frame portion may include any portion of a frameset, such as a front frame portion 102 and / or one or more of the two arm portions 104, each respective arm portion being rotatably coupled to the front frame portion 102 by a hinge portions 115, as depicted in the implementations of FIG. s 1A and IB. In some implementations, the frame portion may include any portions of a goggles-like headset as well (not depicted), including a front frame portion integrated with a side portion.

[0028] Front frame portion 102 includes rim portions 123 surrounding respective optical portions in the form of lenses (including lens 110), the rim portions 123 being coupled together by a bridge portion 129 configured to rest on the nose of a user. The two arm portions 104 are coupled, such as, pivotably or rotatably coupled, to the front frame portion 102 at peripheral portions of the respective rim portions 123. In some implementations, the lenses are corrective / prescription lenses. In some implementations, lens 110 is an optical material including glass and / or plastic portions that does not necessarily incorporate corrective / prescription parameters.

[0029] In augmented reality implementations, a user may view the world through the left lens and the right lens. In virtual reality applications, however, front frame portion 102 may include a display area that is opaque to the world beyond the headset. In virtual reality applications, the sensors and electronics described herein as being coupled to the two arm portions 104 may be coupled to the side portions of a cover around the display that, along with one or more straps, keep the frameset in place on a user’s head.

[0030] Head mounted device 100 includes a head mounted device display 140 configured to display information (e.g., text, graphics, image, etc.) for one or both eyes. Head mounted device display 140 may cover all or part of front frame portion 102 of head mounted device 100. Head mounted device display 140 may include one or both of the left and right lens (of which lens 110 is one).

[0031] In implementations, head mounted device 100 may include other sensing devices in addition to an eye tracking device. The head mounted device 100 may include at least one front facing camera 130. Front facing camera 130 may be directed towards a front field-of-view or can include optics to route light from a front field of view to a sensor.

[0032] In implementations, head mounted device 100 may further include at least one orientation sensor implemented as any combination of accelerometers, gyroscopes, and magnetometers combined to form an inertial measurement unit (i.e., IMU) to determine an orientation of a head mounted device. In implementations, head mounted device 100 mayfurther comprise a microphone 114 and / or a speaker 116.

[0033] FIG. 1C depicts a block diagram of head mounted device 100. according to an implementation. Head mounted device 100 may include any combination of components depicted in FIG.s 1 A, IB, and 1C. In FIG. 1C, head mounted device 100 is depicted as including a processor 152, a memory 154, a communications interface 156, a head mounted device display 140, an eye tracking device 158, and an eye tracking module 160.

[0034] Head mounted device 100 includes a processor 152 and a memory 154. In implementations, processor 152 may include multiple processors, and memory 154 may include multiple memories. Processor 152 may be in communication with any cameras, sensors, and other modules and electronics of head mounted device 100. Processor 152 is configured by instructions (e.g., software, application, modules, etc.) to display content or execute any modules included on head mounted device 100. The instructions may include non-transitory computer readable instructions stored in, and recalled from, memory' 154. In implementations, the instructions may be communicated to processor 152 from a computing device or from a network (not pictured) via a communications interface 156.

[0035] Processor 152 of head mounted device 100 is in communication with head mounted device display 140. Processor 152 may be configured by instructions to transmit text, graphics, video, images, etc. to head mounted device display 140.

[0036] Communications interface 156 of head mounted device 100 may be operable to facilitate communication between head mounted device 100 and other computing devices, such as desktop computers, laptop computers, tablet computers, smart phones, wearable computers, servers, or any other ty pe of computing device. In implementations, communications interface 156 may utilize Bluetooth, Wi-Fi, Zigbee, or any other wireless or wired communication methods.

[0037] Head mounted device 100 includes an eye tracking device 158. Eye tracking device 158 includes a frameset 106, an illuminator, and a reflective surface, as is further described below with regards to FIG. 1A.

[0038] In implementations, processor 152 of head mounted device 100 may be configured with instructions to execute eye tracking module 160. Eye tracking module 160 may be operable to perform any combination of the following functions: operate an illuminator, receive a signal from a detector associated with a camera, measure the signal from the detector to generate one or more images of an eye, receive one or more images of an eye, and determine the direction of a gaze or a series of gazes of a user’s eye. as further described below.

[0039] FIG.s 2A-2F depict various features of eye tracking device 158. FIG. 2A depicts a perspective view of eye tracking device 158 coupled to frameset 106. according to an example. FIGs. 2B and 2C depict top views of eye tracking device 158, according to examples. FIG. 2D depicts a detail of reflective grating 230A, according to examples. FIG. 2E depicts a 3D ray diagram of an eye tracking device, according to examples. FIG. 2F depicts an image of an eyeball generated using eye tracking device 158, according to an example.

[0040] Turning to FIG. 2 A, a perspective view of tracking device 158 may be seen with frameset 106. Eye tracking device 158 may be integrated into frameset 106. An eye 204 (of a user) is positioned behind front frame portion 102 for illustration purposes.

[0041] Illuminator 206 is coupled to first arm portion 203. Illuminator 206 is operable to transmit electromagnetic radiation 208 towards a lens 110. In examples, electromagnetic radiation 208 may have an illuminator bandwidth. The illuminator bandwidth may comprise an intrinsic bandwidth of the illuminator 206. The illuminator 206 can transmit / emit electromagnetic radiation 208 having the illuminator bandwidth. In implementations, illuminator 206 may be a laser that generates light in the infrared to visible range that is substantially uniform over a spatial target area. In implementations, illuminator 206 may be a narrow bandwidth laser. In implementations, illuminator 206 may have an illuminator bandwidth of approximately 1 nm, providing for a more efficient illuminator 206. In implementations, illuminator 206 may emit pulsed light. By selecting an illuminator 206 that pulses light with a sufficiently narrow illuminator bandwidth, it may be possible to provide relatively high peak power, irradiance or radiant intensity to obtain adequate signal-to-noise for eye tracking while also using less battery power. In implementations, the peak power maybe between 0.05-1 W. with an average power below 1 mW. Using less battery power to obtain an adequate signal-to-noise ratio may allow for a smaller battery to operate eye tracking device 158, which may in turn allow for a more compact and lower temperature eye tracking device 158. Making eye tracking device 158 more compact may allow for the eye tracking device 158 to be placed in one or more sections of two arm portions 104 instead of front frame portion 102. The more compact eye tracking device 158 may also allow for two arm portions 104 to be more slimline, enabling a further range of frame styles to be used with head mounted device 100.

[0042] In implementations, the illuminator bandwidth of illuminator 206 may be within the near infrared (NIR) spectrum. NIR light may refer to electromagnetic radiation with a wavelength between 750- 2500 nm, or 800-1000 nm. In implementations, theilluminator bandwidth of illuminator 206 may be within the infrared (IR) spectrum. IR light may refer to electromagnetic radiation with a wavelength between about 1 mm to 750 nm. By selecting illuminator bandwidth within the NIR spectrum or the IR spectrum, it may be possible to provide an eye tracking device 158 that does not emit electromagnetic radiation that is visible to the user.

[0043] In implementations, illuminator 206 may comprise a vertical cavity surface emitting laser, or VCSEL. A VCSEL is a type of semiconductor laser diode with laser beam emission perpendicular from the top surface, as opposed to conventional edge-emitting semiconductor lasers. VCSELs include a larger output aperture compared to edge-emitting lasers, producing a lower divergence angle of the output beam. By making illuminator 206 a VCSEL, it may be possible to illuminate only the parts of eye 204 needed to do eye tracking. Avoiding off-target illumination may allow for an even low er powered illuminator 206 that can provide the adequate signal-to-noise ratio needed to perform eye tracking. Moreover, because VCSELs emit from the top surface of the chip, they can be tested on- afer before they are cleaved into individual devices. Selecting a VCSEL to use for illuminator 206 may therefore reduce the fabrication cost of eye tracking device 158.

[0044] FIGs. 2B and 2C depict top view s of eye tracking device 158. FIG. 2B depicts ray tracing between illuminator 206, reflective gratings 230A, 230B, 230C, eye 204, reflective coating 238, and camera 216. FIG. 2C highlights the virtual illumination sources generated by a combination of at least one reflective grating 230A, 230B and illuminator 206. In the figures, the relative positions of lens 1 10, at least one reflective grating 230A, 230B, a reflective coating 238, a frame portion (including at least front frame portion 102 and first arm portion 203), illuminator 206, and camera 216 may be seen with respect to eye 204.

[0045] Illuminator 206 may transmit electromagnetic radiation 208 towards a lens 110. Lens 110, which may be transparent to visible spectrum electromagnetic radiation, may be coupled to frameset 106. As depicted in FIG. 2B, illuminator 206 may be coupled to first arm portion 203 and directed tow ards lens 110, where some of electromagnetic radiation 208 may be reflected off a reflective coating 238 on a surface of lens 110 back towards the area around eye 204. Lens 110 also includes at least one reflective grating 230 A, 230B configured to redirect a portion of the electromagnetic radiation 208 as redirected electromagnetic radiation 210 towards the eye position.

[0046] Redirecting light from illuminator 206 back towards eye 204 from an angle different than light reflecting off reflective coating 238 may generate a virtual illuminator. Turning to FIG. 2C, it is shown that illuminator 206, which emits electromagnetic radiation208 towards reflective coating 238, generates its own virtual illuminator 234. Likewise, reflective grating 230A generates virtual illuminator 232A and reflective grating 230B generates virtual illuminator 232B. The light from reflective grating 230A and reflective grating 230B is redirected back to the eye position, as represented by eye 204 in the figure, thereby generating two additional glints (in addition to the one generated by virtual illuminator 234) that can be imaged by camera 216. While reflective grating 230A may help generate a glint, the illuminator 206 light reflected off reflective coating 238 towards eye 204 may serve as the primary eye illumination in the image that will be generated by eye tracking device 158.

[0047] In implementations, reflective gratings 230A, 230B may be surface relief gratings. In implementations, at least one reflective grating 230A, 230B may include a ruled binary surface grating. In implementations, at least one reflective grating 230A, 230B may include holographic or slanted gratings. The grating pitch d, which may be seen in FIG. 2D, may be determined for at least one reflective grating 230A, 230B based on the following equation: mA = d(sin a + sin / ?) (Equation 1) where m = index of refraction, X = wavelength, d = grating pitch, a = the angle of incidence, and 0 = the angle of diffraction.

[0048] In implementations, angle of incidence a between electromagnetic radiation 208 emitted by illuminator 206 and a normal to lens 110 may span from 15° to 60°. In implementations, the angle of diffraction 0 may span from -45° and 45°. In implementations, the wavelength emitted by illuminator 206 may be between 850 and 940 nm. In implementations, the order of refraction m may be -1 to 1.

[0049] In implementations, the pitch d of reflective grating 230A, 230B may be designed based on where a reflective grating is positioned on a surface of lens 110 configured to redirect electromagnetic radiation 208 towards an eye position. In implementations, pitch d of reflective grating 230A, 230B may range between 0.5 and 2.1 pm.

[0050] In implementations, reflective grating 230A, 230B may be generated with height and fill factors designed to optimize the performance and manufacturability of the grating. In implementations, any combination of gratings 230A, 230B may have a 50% fill factor and a grating height of 200-400 nm to be consistent with existing display surface relief grating manufacturing processes.

[0051] In implementations, reflective grating 230A. 230B may have the same or different surface areas on lens 110. In implementations, a surface area of reflective grating230A, 230B may be any shape. In implementations, reflective grating 230A, 230B may have a length or diameter that measures approximately 0.5 mm. In implementations, reflective grating 230A, 230B may have a surface area with a length or diameter that measures between 0. 1-1.0 mm, or 0.3-0.7 mm.

[0052] In implementations, reflective grating 230A, 230B may be positioned on a surface of lens 110 adjacent to reflective coating 238. By positioning the reflective grating 230 A, 230B on a surface of lens 110, it may be possible to manufacture the reflective gratings on the surface of lens 110 at the same time as any light coupling / decoupling gratings associated with head mounted device display 140. In further examples, reflective grating 230A, 230B may be positioned on a surface of reflective coating 238 opposite lens 110. Reflective grating 230A. 230B may be etched into reflective coating 238.

[0053] In implementations, eye tracking device 158 may include any number of reflective gratings 230A, 230B configured to generate any number of additional virtual illuminators 232A, 232B. FIG. D depicts a 3D view of light rays between illuminator 206, reflective gratings 230, eye 204, reflective coating 238, and camera 216. In the figure, it maybe seen that any combination / configuration of reflective gratings 230 are possible.

[0054] Eye tracking device 158 may also include reflective coating 238. In implementations, reflective coating 238 may comprise an optical coating through which some electromagnetic radiation, such as visible light, may be transmitted and some is reflected. In implementations, reflective coating 238 may reflect at least some of the illuminator bandwidth while leaving other electromagnetic radiation unaffected. Visible light may refer to electromagnetic radiation with a wavelength between about 380 nm to about 750 nm. In implementations, reflective coating 238 may be positioned on the eye side of lens 110. However, the coating 122 may also be arranged on the other side of lens 110, i.e., on the side of lens 110 that faces away from the user’s eye.

[0055] In implementations, reflective coating 238 may comprise a variation of an anti-reflective coating. In implementations, reflective coating 238 may comprise a specialized dichromatic beam splitter reflective to NIR and / or IR light, while allowing visible light to pass through. Reflective coating 238 may comprise a NIR reflective coating. Reflective coating 238 may cover all or only a portion of the surface on the eye-side of lens 1 10. Reflective coating 238 may cover the area around and between reflective gratings 230A, 230B.

[0056] In implementations, reflective gratings 230A, 230B may be formed from any material on a surface on lens 110. In implementations, reflective gratings 230A, 230B may beformed from a polymer film. In implementations, reflective coating 238 may be made with a dielectric thin film coating such as SiO2 or TiO2.

[0057] In implementations, reflective coating 238 may not be 100% efficient at reflecting the illuminator bandwidth. Therefore, in the example where reflective grating 230A, 230B is positioned on a surface of lens 110 between lens 110 and reflective coating 238, some electromagnetic radiation 208 may pass through and be reflected towards the eye position. Because a glint may saturate pixels in an image captured with camera 216, reflective coating 238 may therefore still be able to provide adequate electromagnetic radiation to an eye position to provide a glint even with reflective grating 230A, 230B being positioned under reflective coating 238.

[0058] FIG. 2B depicts a light ray diagram, according to an example. The light ray diagram depicts the journey that light takes in eye tracking device 158 between illuminator 206 and camera 216. Electromagnetic radiation 208 is emitted from illuminator 206. A portion of electromagnetic radiation 208 is reflected off reflective grating 230A as redirected electromagnetic radiation 210. Redirected electromagnetic radiation 210 is incident on eye 204. A portion of redirected electromagnetic radiation 210 is reflected towards reflective coating 238 and is eventually received at camera 216.

[0059] Eye tracking device 158 further includes camera 216. In implementations, camera 216 may include a sensor configured to provide a signal proportional to the amount of light that falls incident upon it, optics, and a housing. In implementations, the sensor within camera 216 may include a global shutter CMOS image sensor, a two-dimensional charge- coupled device (CCD) array detector, or any other t pe of detector. In implementations, a signal from camera 216 may be digitized and converted to one or more images by processor 152. The one or more images may be used by eye tracking module 160 to determine an eye gaze direction. In implementations, eye tracking module 160 may use a centroiding algorithm.

[0060] The signal provided by camera 216 may be used by processor 152 to measure returned electromagnetic radiation 218 and generate a two-dimensional image of eye 204 from which eye tracking information can be derived. FIG. 2F depicts an eye image 270. Eye image 270 captures an image of eye 204 with a pupil 272. The image of eye 204 and pupil 272 is generated primarily from electromagnetic radiation 208 from illuminator 206 reflected off reflective coating 238 towards eye 204. In and around pupil 272, multiple glints 274 may be seen. The multiple glints 274 are generated primarily from light redirected via reflective grating 230A, 230B, 230C.

[0061] FIG. 3A depicts method 300A, in accordance with an example. Method 300A may be executed to determine a direction of a gaze with an eye tracker. Method 300 A may include any combination of steps 302-310.

[0062] Method 300A may begin with step 302. In step 302, electromagnetic radiation may be transmitted via an illuminator, the illuminator coupled to a frame portion and configured to transmit the electromagnetic radiation towards a lens coupled to the frame portion. Illuminator 206 may transmit electromagnetic radiation 208 towards lens 110, as described above.

[0063] Method 300A may continue with step 304. In step 304, a portion of the electromagnetic radiation may be reflected towards an eye position via a reflective grating on the lens, as redirected electromagnetic radiation. Redirected electromagnetic radiation 210 may be reflected towards eye 204 via reflective grating 230A, as described above.

[0064] Method 300A may continue with step 306. In step 306, a portion of the redirected electromagnetic radiation may be received as a returned electromagnetic radiation, reflected off the lens via a reflective coating. Returned electromagnetic radiation 218 may be reflected off reflective coating 238, as described above.

[0065] Method 300A may continue with step 308. In step 308, a signal may be generated via a sensor associated with a camera coupled to the frame portion based the returned electromagnetic radiation. Camera 216 may generate eye image 270 based on returned electromagnetic radiation 218, as described above.

[0066] Method 300A may continue with step 310. In step 310, an eye gaze direction may be determined, using a processor, based on an image generated based on the signal. An eye gaze direction may be determined based on eye image 270.

[0067] FIG. 3B depicts method 300B, in accordance with an example. Method 300B may be executed to assemble an eye tracking device. Method 300B may include any combination of steps 352-356.

[0068] Method 300A may begin with step 352. In step 352, an illuminator may be coupled to a frame portion, the illuminator being configured to transmit electromagnetic radiation towards an eye position via reflection at a lens. Illuminator 206 may be coupled to first arm portion 203. In implementations, illuminator 206 may be inset into first arm portion 203.

[0069] Method 300A may continue with step 354. In step 354, the lens may be coupled to the frame portion, the lens comprising a reflective grating configured to redirect, i.e. reflect, a portion of the electromagnetic radiation towards an eye position to generate aredirected electromagnetic radiation and a reflective coating. Lens 110 may be coupled to front frame portion 102 of head mounted device 100 by press fitting it into rim portions 123. as described above.

[0070] Method 300A may continue with step 356. In step 356, a camera may be coupled to the frame portion, the camera being configured to receive a portion of the redirected electromagnetic radiation as a returned electromagnetic radiation reflected off the lens via a reflective coating. Camera 216 may be coupled to first arm portion 203 and oriented to receive returned electromagnetic radiation 218 reflected off reflective coating 238, as described above. In implementations, camera 216 may be inset into first arm portion 203 in a cavity adjacent to illuminator 206.

[0071]

[0072] In some aspects, the techniques described herein relate to a head mounted device, wherein the illuminator is a laser flood illuminator.

[0073] In some aspects, the techniques described herein relate to a head mounted device, wherein the illuminator is configured to transmit the electromagnetic radiation having a near infrared spectrum and the reflective coating is a near infrared reflective coating.

[0074] In some aspects, the techniques described herein relate to a head mounted device, wherein the reflective grating is positioned between a surface of the lens and the reflective coating.

[0075] In some aspects, the techniques described herein relate to a head mounted device, wherein the reflective grating is a surface relief grating.

[0076] In some aspects, the techniques described herein relate to a head mounted device, wherein the reflective grating has a pitch between 0.5 and 2.1 pm.

[0077] In some aspects, the techniques described herein relate to a head mounted device, further including: a camera, wherein the camera and the illuminator are coupled to a side portion of the frame portion.

[0078] In some aspects, the techniques described herein relate to a method for eye tracking, further including: generating a signal, via a sensor associated with a camera coupled to the frame portion, based the returned electromagnetic radiation.

[0079] In some aspects, the techniques described herein relate to a method, wherein the camera generating the signal and the illuminator transmitting the electromagnetic radiation are coupled to a side portion of the frame portion.

[0080] In some aspects, the techniques described herein relate to a method, furtherincluding: determining, via a processor, an eye gaze direction based on an image generated based on the signal.

[0081] In some aspects, the techniques described herein relate to a method, wherein the illuminator is configured to transmit the electromagnetic radiation having a near infrared spectrum and the reflective coating is a near infrared reflective coating.

[0082] In some aspects, the techniques described herein relate to a method, wherein the reflective grating reflecting the portion of the electromagnetic spectrum is positioned between a surface of the lens and the reflective coating.

[0083] In some aspects, the techniques described herein relate to a method, wherein the reflective grating reflecting the portion of the electromagnetic spectrum is a surface relief grating.

[0084] In some aspects, the techniques described herein relate to a method, wherein the reflective grating reflecting the portion of the electromagnetic spectrum has a pitch between 0.5 and 2.1 pm.

[0085] In some aspects, the techniques described herein relate to a method, wherein coupling the illuminator to the frame portion further includes coupling the illuminator to a side portion of the frame portion and coupling the camera to the frame portion further includes coupling the camera to the side portion of the frame portion.

[0086] In some aspects, the techniques described herein relate to a method, wherein the lens including the reflective coating coupled to the frame portion further includes the reflective grating positioned between a surface of the lens and the reflective coating.

[0087] In some aspects, the techniques described herein relate to a method, wherein the lens including the reflective coating coupled to the frame portion further includes the reflective grating being a surface relief grating.

[0088] In some aspects, the techniques described herein relate to a method, wherein the lens including the reflective coating coupled to the frame portion further includes the reflective grating having a pitch between 0.5 and 2. 1 pm.

[0089] In some aspects, the techniques described herein relate to an eye-tracking device, including: an illuminator configured to be coupled to a frame portion, the illuminator transmitting electromagnetic radiation and configured to transmit electromagnetic radiation towards a lens; and the lens configured to be coupled to a frame portion, the lens including a reflective coating and a reflective grating configured to redirect a portion of the electromagnetic radiation, as redirected electromagnetic radiation, towards an eye position.

[0090] The disclosure of the Application describes an eye tracking device thatincludes fewer illuminators while producing multiple glints, thereby providing an accurate eye tracking device with fewer electrical components and system complexity. The eye tracking device design may therefore allow for a smaller volume of electronic components, fewer electrical traces, less power, and lower manufacturing challenges. Additionally, if head mounted device display 140 already includes gratings, such as surface relief grating couplers, on the surface of lens 110, reflective grating 230A , 23 OB may be added with no additional monetary cost or manufacturing complexity.

[0091] V arious examples of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitiy. specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various examples can include examples in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device. Various examples of the systems and techniques described here can be realized as and / or generally be referred to herein as a circuit, a module, a block, or a system that can combine software and hardware aspects. A module may include the functions / acts / computer program instructions executing on a processor or some other programmable data processing apparatus.

[0092] Some of the above examples are described as processes or methods depicted as flow charts. Although the flow-charts describe the operations as sequential processes, many of the operations may be performed in parallel, concurrently or simultaneously. In addition, the order of operations may be re-arranged. The processes may be terminated when their operations are completed but may also have additional steps not included in the figure. The processes may correspond to methods, functions, procedures, subroutines, subprograms, etc.

[0093] Methods discussed above, some of which are illustrated by the flow- charts, may be implemented by hardw are, softw are, firmware, middlew are, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware or microcode, the program code or code segments to perform the necessary tasks may be stored in a machine or computer readable medium such as a storage medium. A processor(s) may perform the necessary- tasks.

[0094] Specific structural and functional details disclosed herein are merely representative for purposes of describing examples. Examples, however, be embodied in many alternate forms and should not be construed as limited to only the implementations setforth herein.

[0095] It will be understood that, although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. A first element could be termed a second element, and, similarly, a second element could be termed a first element, without departing from the scope of the implementations of the disclosure. As used herein, the term and / or includes any and all combinations of one or more of the associated listed items.

[0096] The terminology used herein is for the purpose of describing particular implementations only and is not intended to be limiting. As used herein, the singular forms a, an, and the are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms includes, comprising, includes and / or including, when used herein, specify the presence of stated features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0097] It should also be noted that in some alternative examples, the functions / acts noted may occur out of the order noted in the FIG.s. Two FIG.s shown in succession may in fact be executed concurrently or may sometimes be executed in the reverse order, depending upon the functionality / acts involved.

[0098] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the implementations belong. It will be further understood that terms, e g., those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0099] Portions of the above implementations and corresponding detailed description are presented in terms of software, or algorithms and symbolic representations of operation on data bits within a computer memory. These descriptions and representations are the ones by which those of ordinary skill in the art effectively convey the substance of their work to others of ordinary skill in the art. An algorithm, as the term is used here, and as it is used generally, is conceived to be a self-consistent sequence of steps leading to a desired result. The steps are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of optical, electrical, or magnetic signalscapable of being stored, transferred, combined, compared, and otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

[0100] In the above implementations, reference to acts and symbolic representations of operations (e.g., in the form of flowcharts) that may be implemented as program modules or functional processes include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types and may be described and / or implemented using existing hardware at existing structural elements. Such existing hardware may include one or more Central Processing Units (CPUs), digital signal processors (DSPs), application-specific-integrated-circuits, field programmable gate arrays (FPGAs) computers or the like.

[0101] It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise, or as is apparent from the discussion, terms such as processing or computing or calculating or determining of displaying or the like, refer to the action and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical, electronic quantities within the computer system’s registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices.

[0102] Note also that the software implemented aspects of the Application are typically encoded on some form of non-transitory program storage medium or implemented over some type of transmission medium. The program storage medium may be magnetic (e.g., a floppy disk or a hard drive) or optical (e.g., a compact disk read only memory’, or CD ROM), and may be read only or random access. Similarly, the transmission medium may be twisted wire pairs, coaxial cable, optical fiber, or some other suitable transmission medium known to the art. The examples are not limited by these aspects of any given examples.

[0103] Lastly, it should also be noted that whilst the accompanying claims set out particular combinations of features described herein, the scope of the present disclosure is not limited to the particular combinations hereafter claimed, but instead extends to encompass any combination of features or examples herein disclosed irrespective of whether or not that particular combination has been specifically enumerated in the accompanying claims at this time.

Claims

WHAT IS CLAIMED IS:

1. A head mounted device, comprising: a frame portion; a lens coupled to the frame portion; and an illuminator coupled to the frame portion and configured to transmit electromagnetic radiation towards the lens, the lens including a reflective grating and a reflective coating, the reflective grating configured to redirect a portion of the electromagnetic radiation as redirected electromagnetic radiation towards an eye position via the reflective coating.

2. The head mounted device of claim 1, wherein the illuminator is a laser flood illuminator.

3. The head mounted device of any of claims 1-2, wherein the illuminator is configured to transmit the electromagnetic radiation having a near infrared spectrum and the reflective coating is a near infrared reflective coating.

4. The head mounted device of any of claims 1-3, wherein the reflective grating is positioned between a surface of the lens and the reflective coating.

5. The head mounted device of any of claims 1-4, wherein the reflective grating is a surface relief grating.

6. The head mounted device of any of claims 1-5, wherein the reflective grating has a pitch between 0.5 and 2. 1 pm.

7. The head mounted device of any of claims 1-6, further comprising: a camera, wherein the camera and the illuminator are coupled to a side portion of the frame portion.

8. A method for eye tracking comprising: transmitting, via an illuminator, electromagnetic radiation, the illuminator being coupled to a frame portion and configured to transmit the electromagnetic radiation towards a lens coupled to the frame portion; redirecting a portion of the electromagnetic radiation towards an eye position, via a reflective grating on the lens, as redirected electromagnetic radiation; and receiving a portion of the redirected electromagnetic radiation as a returned electromagnetic radiation reflected off the lens via a reflective coating.

9. The method for eye tracking of claim 8, further comprising: generating a signal, via a sensor associated with a camera coupled to the frame portion, based on the returned electromagnetic radiation.

10. The method of claim 9, wherein the camera generating the signal and the illuminator transmitting the electromagnetic radiation are coupled to a side portion of the frame portion.1 1. The method of claim 9, further comprising: determining, via a processor, an eye gaze direction based on an image generated based on the signal.

12. The method of any of claims 8-10, wherein the illuminator is configured to transmit the electromagnetic radiation having a near infrared spectrum and the reflective coating is a near infrared reflective coating.

13. The method of any of claims 8-12, wherein the reflective grating reflecting the portion of the electromagnetic radiation is positioned between a surface of the lens and the reflective coating.

14. The method of any of claims 8-13, wherein the reflective grating reflecting the portion of the electromagnetic radiation is a surface relief grating.

15. The method of any of claims 8-14, wherein the reflective grating reflecting the portion of the electromagnetic radiation has a pitch between 0.5 and 2. 1 pm.

16. A method for assembling an eye tracking device, comprising: coupling an illuminator to a frame portion, the illuminator being configured to transmit electromagnetic radiation towards an eye position via reflection at a lens: and coupling the lens to the frame portion, the lens comprising a reflective grating configured to redirect a portion of the electromagnetic radiation towards an eye position to generate a redirected electromagnetic radiation and a reflective coating; and coupling a camera to the frame portion, the camera being configured to receive a portion of the redirected electromagnetic radiation, as a returned electromagnetic radiation, reflected off the lens via a reflective coating.

17. The method of claim 16, wherein coupling the illuminator to the frame portion further comprises coupling the illuminator to a side portion of the frame portion and coupling the camera to the frame portion further comprises coupling the camera to the side portion of the frame portion.

18. The method of any of claims 16-17, wherein the lens comprising the reflective coating coupled to the frame portion further comprises the reflective grating positioned between a surface of the lens and the reflective coating.

19. The method of any of claims 16-18, wherein the lens comprising the reflective coating coupled to the frame portion further comprises the reflective grating being a surface relief grating.

20. The method of any of claims 16-19, wherein the lens comprising the reflective coating coupled to the frame portion further comprises the reflective grating having a pitch between 0.5 and 2.1 pm.

Images