Metasurface camera module for a head-mounted device

Abstract

A camera module that can be used in a head-mounted device, such as AR glasses, for eye tracking or hand tracking is disclosed. The camera module can be positioned in a temple arm AR glasses to receive a mirror image of the user's eye from a lens of the AR glasses. The camera module may include one or more metasurfaces that can focus the mirror image of the eye onto a detector, which generates a digital eye-image for eye tracking. The one or more metasurfaces may be able to generate phase profiles that are not possible using conventional lenses, which may have allowed the camera module to be positioned and oriented in a way that is conducive to alignment and that takes up very little volume in the temple arm of the AR glasses.

Description

METASURFACE CAMERA MODULE FOR AHEAD-MOUNTED DEVICEFIELD OF THE DISCLOSURE

[0001] The present disclosure relates to an imaging system and more specifically to a metasurface camera module for a head-mounted device.BACKGROUND

[0002] A head-mounted device (i.e., head- worn device) is a computing device worn on the head of a user, which can be configured to present information (e.g., visual, audio) to the user. The information presented may be based on the user’s physical interaction with the head-worn device. The head-worn device may respond to this physical interaction by changing the information presented so that the user feels as if they are in a virtual environment with the presented information. For example, sensors in the head-mounted device may sense the posit on and movement of the user’s head so that images displayed on a display of the head-mounted device may move in accordance with a head movement of a user. Sensing and tracking eye positions and movements (i.e., eye-tracking) in addition to (or in place ol the head-tracking can help determine the focus and / or intent of the user, which can enhance the accuracy and realism of the virtual environment. Accordingly, a headmounted device may include a camera module for capturing images of the eye (or eyes) of the user for eye-tracking.SUMMARY

[0003] The present disclosure includes a metasurface camera module for capturing images of an eye of a user. The metasurface camera module may include at least one metasurface. A metasurface can be a surface having a structure with very small dimensions (e.g., subwavelength) which can affect the phase of light passing through the structure. The at least one metasurface may help to capture images at angles difficult for conventional lenses, which can eliminate the requirement that the camera module be positioned to point (i) directly at a user’s eye to capture an eye-image or (ii) directly at a mirror to capture a reflection of the user’s eye. As a result, the metasurface camera module may require less space in the head-mounted device than that which would be required by a conventional camera module using traditional refractive lenses. Further, the metasurface camera modulemay be supported and positioned by a body (e.g., frame) of the head-mounted device so that it is easier to set and maintain alignment.

[0004] In some aspects, the techniques described herein relate to a head-mounted device including: a frame that includes a temple arm coupled to a front portion so that when the frame is worn by a user, the front portion is positioned adjacent to the face of the user and the temple arm extends from the front portion towards an ear of the user; a lens supported in front of an eye of the user by the front portion of the frame; a mirror disposed on a surface of the lens to direct light corresponding to a reflection of the eye to a position along the temple arm; and a camera module integrated with the temple arm at the position along the temple arm, the camera module configured to: receive the light from the mirror corresponding to the reflection of the eye; redirect and focus the light using at least one metasurface to generate a real-image of the eye on a detector; and generate a digital eye-image using the detector.

[0005] Generally, a corresponding metasurface camera module may in particular be integrated in a proposed head-mounted device. Accordingly, a proposed head-mounted device may in particular comprise an embodiment of a proposed metasurface camera module.

[0006] In some aspects, the techniques described herein relate to a metasurface camera module to be integrated with a temple arm of a head-mounted device, the metasurface camera module including: a first plurality of nanostructures disposed on a first side of a substrate; a second plurality of nanostructures disposed on a second side of the substrate, the second side opposite the first side; and a detector facing the second plurality of nanostructures and separated from the substrate by a focal length along an optical axis, the optical axis bisecting the substrate and the detector.

[0007] Generally, the metasurface camera module may include at least one metasurface that functions as a lens to focus a real image of the eye on a detector.

[0008] The foregoing illustrative summary, as well as other exemplary objectives and / or advantages of the disclosure, and the manner in which the same are accomplished, are further explained within the following detailed description and its accompanying drawings.BRIEF DESCRIPTION OF THE DRAWINGS

[0009] FIG. 1 A is a front view of a head-mounted device according to a possible implementation of the present disclosure.

[0010] FIG. IB is a rear view of the head-mounted device according to a possible implementation of the present disclosure.

[0011] FIG. 2 is a system block diagram of a head-mounted device according to a possible implementation of the present disclosure.

[0012] FIG. 3 is a top view of a portion of a head-mounted device including a camera module according to a possible implementation of the present disclosure.

[0013] FIG. 4 is a ray -trace of light in a head-mounted device according to a possible implementation of the present disclosure.

[0014] FIG. 5 is a side cross-sectional view of a camera module according to a possible implementation of the present disclosure.

[0015] FIG. 6 is a side-view, ray -trace of light in the camera module of FIG. 4 according to a possible implementation of the present disclosure.

[0016] FIG. 7A is a phase profile of a first metasurface for a dual-metasurface camera module according to a possible implementation of the present disclosure.

[0017] FIG. 7B is a phase profile of a second metasurface for the dual-metasurface camera module according to a possible implementation of the present disclosure.

[0018] FIG. 8A is a perspective of a portion of a metasurface according to a possible implementation of the present disclosure.

[0019] FIG. 8B is a top view of a portion of a metasurface according to a possible implementation of the present disclosure.

[0020] FIG. 9 is a side-view, ray-trace of light in a single-metasurface camera module according to a possible implementation of the present disclosure.

[0021] FIG. 10A illustrates a field-of-view for a dual-metasurface camera module aligned in a first direction according to a possible implementation of the present disclosure.

[0022] FIG. 10B illustrates a field-of-view for a dual-metasurface camera module aligned in a second direction according to a possible implementation of the present disclosure.

[0023] The components in the drawings are not necessarily to scale relative to each other. Like reference numerals designate corresponding parts throughout the several views.DETAILED DESCRIPTION

[0024] A head-mounted device (i.e. , HMD) , such as augmented-reality glasses (i. e. , AR glasses) or smart glasses, may be configured to implement an eye-tracking process to determine where the interest of a user (i.e., wearer) is directed. The eye-tracking process may require an imaging system to capture digital eye-images of the eye or eyes of the user. The digital eye-images can be processed by an image-processing system to identify eye features (e.g., pupil, iris, etc.) and their locations. Based on these locations, and theposition / orientation (i.e., pose) of the head, a point of interest (i.e., POI) of the user relative to the user’s environment may be determined. In a possible implementation, software running on the AR glasses may be configured to present information regarding an object or a person located at the POI.

[0025] Eye-tracking may require an imaging system configured to generate a real image of an eye (or eyes) of the user on a detector. In a possible implementation, the imaging system may further include an illumination system that can illuminate the eye (or eyes) of the user to improve the quality of a digital eye-image captured by the detector. One technical problem with eye-tracking is efficiently and effectively integrating the imaging system and / or the illumination system into a frame of the AR glasses, especially when it is desirable to make the frames less bulky' for comfort and aesthetics. The present disclosure describes the use of metasurface optical elements to (at least) reduce a volume required for an eye-tracking imaging system and / or an eye-tracking illumination system. The use of a metasurface can also ease the requirements for alignment and / or increase an optical performance (e.g., lower f-number) of the imaging system, as compared to conventional approaches.

[0026] FIG. 1 A is a front view of a head-mounted device according to a possible implementation of the present disclosure. As shown, the head-mounted device 100 may be implemented as glasses (e.g., smart glasses, AR glasses). The head-mounted device 100 includes a frame having a front portion 120 configured to support and position a lens 110 (e.g., left lens, right lens). The front portion 120 of the head-mounted device 100 may further include a world-facing camera 210 configured to capture images of the environment from a point-of-view of a user (i.e., wearer) of the head-mounted device 100. In a possible implementation the front portion 120 of the head-mounted device 100 may further include a depth sensor 216 implemented as a second world-facing camera or a ranging device (e g., lidar sensor, structured light sensor) and configured to determine ranges to objects within view of the user.

[0027] FIG. IB is a rear view of the head-mounted device according to a possible implementation of the present disclosure. The front portion 120 of the head-mounted device 100 is positioned adjacent to the face of the user so that the user can view the environment through the lenses of the head-mounted device 100 because the lens is supported in front of an eye of the user. The head-mounted device 100 further includes a temple arm 130 (e.g., left temple-arm, right temple-arm). The temple arm 130 extends from the front portion 120 towards an ear of the user. The temple arm 130 includes an inner surface 135 that faces the user (i.e., wearer) of the head-mounted device 100.

[0028] FIG. 2 is a system block diagram of a head-mounted device according to a possible implementation of the present disclosure. As mentioned, the head-mounted device (i. e. , head- worn device) may be implemented as smart glasses worn by a user. In a possible implementation, the smart glasses may provide a user with (and enable a user to interact with) an augmented reality (AR) environment. In these implementations, the smart glasses may be referred to as AR glasses. In another possible implementation, the head-mounted device may be implemented as a headset or goggles worn by a user. In a possible implementation, the headset or goggles may provide the user (and enable a user to interact with) a virtual reality (VR) environment. While the head-mounted device can be implemented variously, the disclosure may refer to the AR glasses implementation as the head-mounted device throughout the disclosure.

[0029] The head-mounted device 200 may be worn on the head of a user (i.e., wearer) and can be configured to monitor a position and orientation (i.e., pose) of the head of the user. Additionally, the head-mounted device 200 may be configured to monitor the environment of the user. The head-mounted device may be further configured to determine a frame coordinate system based on the pose of the user and a world coordinate system based on the environment of the user. The relationship between the frame coordinate system and the world coordinate system may be used to visually anchor digital objects to real objects in the environment. The digital objects may be displayed to a user in a heads-up display. For these functions the head-mounted device 200 may include a variety of sensors and subsystems.

[0030] The head-mounted device 200 may include a world-facing camera 210. The world-facing camera (i.e., front-facing camera) may be configured to capture images of a front field-of-view 215. The front field-of-view 215 may be aligned with a user’s field of view so that the front-facing camera (i.e., world-facing camera 210) captures images from a point-of-view of the user. In a possible implementation, the head-mounted device 200 may further include a depth sensor 216 (i.e., range detector) configured to measure a range of objects in (at least) the front field-of-view 215.

[0031] The head-mounted device 200 may further include a position / orientation (i.e. pose) sensor. The pose sensor can be implemented as an inertial measurement unit (IMU 290). The IMU may include a plurality of sensor modules to determine its position, orientation, and / or movement. The IMU may have a frame coordinate system (X, Y, Z) and each sensor module may output values relative to each direction of the frame coordinate system.

[0032] The head-mounted device 200 may further include an eye-tracking camera module (i. e. , camera module 220). The camera module 220 may be configured to form a real image from light collected from an eye field-of-view 225. The light collected may include light reflected from an exterior surface of the eye. For example, a mirror may reflect light corresponding to the reflection of the eye towards the eye-tracking camera module 220 (i.e. camera module). The camera module may capture a digital eye-image of the real image. The digital eye image may include features of the eye, such as the pupil and the iris. The features may be analyzed (i.e., compared to the frame coordinate system and / or world coordinate system) to determine a gaze direction of a user. In a possible implementation, the headmounted device 200 includes an eye-tracking camera module 220 for each eye and a gaze direction may be based on the images of both eyes (e.g., eye vergence).

[0033] The head-mounted device 200 may further include an illuminator module 217 configured to shine light (e.g., infrared light) into the eye field-of-view 225. The light may help to illuminate the surface of the eye so that digital eye-images captured by the camera module 220 have a higher quality (e.g., higher signal-to-noise), which may help a feature detection process associated with eye tracking.

[0034] The head-mounted device 200 may further include a light sensor 299 configured to measure an ambient light level. In a possible implementation the light sensor 299 output may trigger a low-light condition when the measured ambient light is at or below a low-light threshold. In a possible implementation, the low-light condition can trigger the illuminator module 217 to illuminate the eye field-of-view 225. In an alternative implementation, the illuminator module 217 may illuminate the eye continuously while eye tracking.

[0035] The head-mounted device 200 may further include location sensors 230. The location sensors may be configured to determine a position of the head-mounted device (i.e., the user) on the planet, in a building (or other designated area), or relative to another device. For this, the head-mounted device 400 may communicate with other devices over a wireless communication link 235. For example, a user’s position may be determined within a building based on communication between the head-mounted device 200 and an access point 231 or indoor positioning unit. In another example, a user’s position may be determined on the planet based on a global positioning system (GPS) link between the head-mounted device 200 and a GPS satellite 232 (e.g., a plurality of GPS satellites). In another example, a user’s position may be determined relative to a mobile device 233 (e.g., mobile phone) based on ultra-wide band (UWB) communication or Bluetooth communication between the mobile device 233 and the location sensor 230.

[0036] The head-mounted device 200 may further include a heads-up display 240 (i.e., HUD) displayed on a portion (e.g., the entire portion) of a lens 110 (or lenses) of AR glasses. In a possible implementation, a projector positioned in a frame arm of the glasses may project light to a surface of a lens, where it is reflected to an eye of the user. In another possible implementation, an optical waveguide and a diffractive optical element may be integrated with a lens 110 to proj ect the display to the eye of the user.

[0037] The head-mounted device 200 may further include a battery 250. The battery 250 may be configured to provide energy to the subsystems, modules, and devices of the headmounted device 200 to enable their operation. The battery 250 may be rechargeable and have an operating life (e.g., lifetime) between charges. The head-mounted device 200 may include circuitry or softw are to monitor a battery level of the battery 250.

[0038] The head-mounted device 200 may further include a communication interface 260. The communication interface may be configured to communicate information digitally over a wireless communication link (e.g., WiFi, Bluetooth, etc.). For example, the headmounted device 200 may be communicatively coupled to a network 237 (i.e., the cloud) or a device (e.g., the mobile phone 238) over a wireless communication link 236. The wireless communication link 236 may allow operations of a computer-implemented method to be divided between devices (i.e., split-processing). Additionally, the wireless communication link 236 may allow' a device to communicate a condition, such as a battery level or a power mode (e.g., low-power mode). In this regard, devices communicatively coupled to the headmounted device 200 may be considered as accessories to the head-mounted device 200 and therefore each may be referred to as an accessory device (e.g., mobile phone, tablet).

[0039] The head-mounted device 200 may further include a memory 270. The memory may be anon-transitory computer readable medium (i.e., CRM). The memory may be configured to store a computer program product. The computer program can instruct a processor to perform computer implemented methods (i.e., computer programs). These computer programs (also known as modules, programs, software, softw are applications or code) include machine instructions for a programmable processor and can be implemented in a high-level procedural and / or object-oriented programming language, and / or in assembly / machine language. As used herein, the terms “machine-readable medium” or “computer-readable medium” refers to any computer program product, apparatus and / or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal.The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0040] The head-mounted device 200 may further include a processor 280. The processor may be configured to carry out instructions (e.g., software, applications, etc.) to configure the functions of the head-mounted device 200. In a possible implementation the processor may include multiple processor cores. In a possible implementation, the headmounted computing device may include multiple processors. In a possible implementation, processing for the head-mounted computing device may be carried out over a network 237.

[0041] FIG. 3 is a top view of a portion of a head-mounted device including a camera module according to a possible implementation of the present disclosure. When the headmounted device is worn, the front portion 120 off the head-mounted device may be positioned adjacent to (i.e., in front of) a face of the user, while a temple arm 130 may be positioned adjacent to a side of a head of the user. For example, the temple arm may extend from a connection point (e.g., hinge) at the front portion 120 to an ear portion 150 at an ear of the user. As shown, the front portion 120 of the head-mounted device can include a lens 110 positioned in front of an eye 301 of a user. The lens may include a mirror 111. For example, the mirror 111 may be disposed as one or more layers on a surface of the lens 110 facing the eye 301 of the user. The mirror 111 may have a fdtering quality so that light from a visible portion of the electromagnetic spectrum (i.e., visible light 312) may pass through the lens with little (e.g., < 4%) reflection.

[0042] The mirror 1 1 1 may create a virtual image of a surface of the eye that includes visible features of the eye, such as the iris 311 and the pupil 313. The virtual image of the eye in the mirror may be referred to as a reflection of the eye. The camera module 320 can receive the reflection of the eye from the mirror instead of receiving light from the eye directly. As a result an optical path between the eye 301 and the camera module 320 may be folded by the mirror 111.

[0043] Due to filtering properties of the mirror, the reflected light 314 may be in a portion of the electromagnetic spectrum that is invisible to a user. For example, the mirror 111 may be configured as a short-pass mirror with an ability to reflect wavelengths longer than a cutoff wavelength and transmit wavelengths shorter than the cut-off wavelength. The cut-off wavelength may be an infrared wavelength (e.g., 800 nm < 1 < 1000 nm) so that the reflected light 314 is infrared light (e.g., near infrared light).

[0044] The mirror 111 can also allow an illuminator module 330 to transmit light to the (outer surface of the) eye 301 via the mirror 111. For example, the illuminator module 330 may be configured to project transmitted light 315 from at least one optical source (e.g., LED, VCSEL, etc.) to the mirror 111. The transmitted light 315 may be infrared light (e.g., at the same wavelength(s) as the reflected light 314), which is invisible to a user and which is reflected by the (short-pass) mirror 111 to illuminate (the surface of the) eye 301. The illumination can increase an intensity of the reflected light 314 to make its detection by the camera module 320 easier.

[0045] The camera module 320 may be integrated with (e.g., inside) the temple arm 130 of the head-mounted device 100. As shown, the camera module 320 may be fabricated (e.g., installed) with the temple arm 130 so that a surface (e.g., receiving surface) of the camera module 320 is flush with an inner surface 135 surface of the temple arm 130. The camera module 320 may be located at a first position between the front portion 120 and an ear portion 150 of the temple arm 130 (i.e., along the temple arm).

[0046] In a possible implementation, the head-mounted device 100 further includes a metasurface illuminator module 330. As shown, the metasurface illuminator module 330 may be integrated so that a surface of the illuminator module 330 is flush with an inner surface 135 surface of the temple arm 130. The metasurface illuminator module 330 may be located al a second position between the front portion 120 and the ear portion 150 of the temple arm 130 (i.e., along the temple arm). In a possible implementation, the camera module 320 and the metasurface illuminator module 330 may be integrated together so that both are contained within a module at a position along the temple arm.

[0047] FIG. 4 is a ray -trace of light in a head-mounted device according to a possible implementation of the present disclosure. As shown, light from the mirror (i.e., corresponding to the reflection of the eye) may be received by an optical element of the camera module 320. The optical element 401 is configured to redirect and focus the light to a detector 402. The redirection may enable the detector to be positioned in a plane conducive for alignment.

[0048] Returning to FIG. 3, the detector 402 may be aligned parallel with a plane 136 defined by the inner surface 135 of the temple arm 130. As shown, the detector 402 faces the optical element but does not face the mirror 111 and does not face the eye (not shown), either of which would require more space. The optical element 401 generates a real image (e.g., real eye-image) on the detector 402.

[0049] As shown in FIG. 4, the optical element 401 generates a real image 412 on the detector. For example, reflected light 410 corresponding to a reflected image (i. e. , virtual) of an arrow 411 may be redirected and focused on the detector 402 as a real image of the arrow 412.

[0050] FIG. 5 is a side cross-sectional view of a camera module according to a possible implementation of the present disclosure. The camera module 500 includes an optical element 401 configured to redirect and focus light onto a detector 520. For example, the optical element 401 may be configured to generate a real image of an eye of a user on the detector 520, and the detector 520 is configured to capture a digital eye-image 531 based on the real image. Accordingly the detector 520 and the optical element 401 may be aligned so that their centers define an optical axis 501 that bisects each. The detector 520 and the optical element 401 may be spaced at a focal length (f) of the optical element 401. For example, the optical element 401 and the detector 520 may be supported and positioned (e.g., attached) by a housing (not shown) that mechanically spaces the detector 520 a focal length (f) apart from the optical element 401 so that the real image is in focus. The focal length and a dimension (e.g., diameter (D)) of the optical element 401 and detector 520 may be selected so that the camera module 500 has an f-number (e.g., f / # = 2) for capturing digital eye-images having a quality (e.g., noise level) suitable for eye-tracking.

[0051] The optical element 401 includes at least one metasurface to generate a real-image of the eye on the detector 520. As shown, the optical element 401 can include a first metasurface 51 1 configured to function as a first lens (i.e., first metasurface lens) and a second metasurface 512 configured to function as a second lens (i.e., second metasurface lens). The first metasurface 511 and the second metasurface 512 may be formed in a first layer and a second layer disposed on opposite sides of a substrate 510. The first metasurface may define a plane 536 of the camera module 500. The camera module 500 may be integrated with the temple arm 130 so that the plane 536 is flush with (i.e., aligned with) the inner surface 135 of the temple arm 130. In other words, the camera module 500 can face the user (i.e., the side of the user’s head) instead of facing the mirror as shown in the hypothetical camera module 530. This position / orientation of the camera module 500 with respect to the temple arm 130 may make alignment easier. For example, once the first metasurface 511 is made flush with the plane 136 no further alignment may be necessary for the camera module 500 to receive light from the mirror 111. This position / orientation of the camera module 500 flush (i.e., flat) with the inner surface 135 may also reduce a volume (in the temple arm 130)required by the camera module, as compared to the hypothetical camera module 530 pointed at the mirror 111.

[0052] The second metasurface 512 of the optical element 401 is substantially parallel with the first metasurface 511 and can face the detector 520. The first metasurface 511 and the second metasurface 512 may each be an imprinted (e.g., nanoimprint lithography) material (e.g., Si) on a surface of the substrate. The substrate may be a material (e.g., SiCh, TiCh) that is optically transparent at the wavelength(s) (e.g., 850nm, 940nm) of the reflected light 314

[0053] FIG. 6 is a side-view, ray -trace of light in the camera module of FIG. 4 according to a possible implementation of the present disclosure. As shown, the substrate 510 of the camera module 500 may be implemented as a circular disk with a small diameter (e.g., D = 2 millimeters) and thickness (e.g., 500 microns) for integration in the temple arm. The light 410 from the mirror 111 (at the lens of the head-mounted device) is redirected by the first metasurface 511 to a path along a Z-direction (i.e., optical axis) of the camera module 500. The light from the first metasurface may be substantially parallel with the optical axis (Z). The light from the first metasurface 511 can be focused onto the detector 520 by the second metasurface 512. Accordingly , the detector 520 may be positioned a focal length (f) away from the second metasurface 512. The detector 520 can be implemented as a pixel array camera, such as a CCD or a CMOS that is configured to convert the intensity of light at each pixel into a digital signal to generate a digital eye-image of the light focused by the second metasurface 512. The implementation of the camera module 500, which includes a first metasurface 511 and a second metasurface 512 may be referred to as a dual-metasurface camera module.

[0054] The first metasurface 511 of the dual-metasurface camera module may have impart a first phase shift on the light 410. The first phase shift can vary according to a first phase profile in a first transverse (i.e., Y) direction. The second metasurface 512 of the dual- metasurface camera module may have impart a second phase shift on the light 410. The second phase shift can vary according to a second phase profile in the first transverse (i.e., Y) direction. The first phase profile and the second phase profile may be constant in a second transverse (i.e., X) direction. The first phase profile may and the second phase profile may be different. For example, the first phase profile may be configured to redirect the light 410 while the second phase profile may be configured to focus the redirected light.

[0055] FIG. 7A is a graph of a phase profile (i.e., first phase profile) of the first metasurface 511 for a dual-metasurface camera module according to a possibleimplementation of the present disclosure. The first phase profile can be a gradient phase profile that linearly changes in the Y-direction (i.e., Y coordinate shown in FIG. 6). The gradient phase profile may change polarity on either side of the optical axis (i.e., Y = 0 position shown in FIG. 6) and may be higher (e.g., <j> > 0) for a portion of the first metasurface 511 that is closer to the mirror 111 and lower (e.g. , $ < 0) for a portion of the first metasurface 511 that is further from the mirror 111. The phase in a X-direction (not shown) may be constant.

[0056] FIG. 7B is a graph of a phase profile (i.e., second phase profile) of the second metasurface 512 for a dual-metasurface camera module according to a possible implementation of the present disclosure. The second phase profile can be a parabolic phase profile that has a parabolic shape in the Y-direction (i.e., Y coordinate shown in FIG. 6). The parabolic phase profile higher (e.g., $ = 0) for a portion of the second metasurface 512 that is closer to the optical axis (i.e., Y = 0 position shown in FIG. 6) and lower (e.g., $ < 0) for a portion of the second metasurface 512 at the edges of the substrate 510. The phase in a X- direction (not shown) may also have the parabolic phase profile, and the parabolic profiles in both transverse directions can generate a real image in two dimensions.

[0057] The each metasurface of the camera module may include a plurality of nanostructures to impart the phase shift. The nanostructures may cover a surface of the substrate and may include structures that are same type, that is the structures may have a geometrically similar form (e.g., square pillar, circular pillar, etc.), but which may have a range of different sizes (e g., diameters, heights, aspect ratios, etc.) and / or spacings according to their positions to produce the phase profile.

[0058] FIG. 8A is a perspective of a portion of a metasurface according to a possible implementation of the present disclosure. The portion of the metasurface show n includes a plurality of pillar nanostructures 801 raised above a surface of the substrate. Each pillar nanostructure may have a height in a first nanometer range (e.g., 400 nm < h < 600 nm) and may have a diameter in a second nanometer range (e.g., 50 nm < d < 350 nm). The structure of the pillar nanostructures may be obtained by an optimization algorithm to obtain the desired phase profile for a particular range of input light characteristics (e.g., wavelength, angles, etc ). Accordingly, the pillars, while having the same height, may vary in shape (e g., cylinder, square, diamond, etc.), diameter, and pattern (periodicity, pitch, offset, etc.).

[0059] FIG. 8B is a top view' of a portion of a metasurface according to a possible implementation of the present disclosure. The portion of the metasurface shown includes a plurality of pillar nanostructures. Each pillar nanostructure may be position at a point on arectangular grid or offset grid so that the pillar centers of the columns of pillar nanostructures are separated by a horizontal pitch distance 802 and the pillar centers of the rows of pillar nanostructures are separated by a vertical pitch distance 803. The horizontal pitch distance 802 and the vertical pitch distance 803 may be determined by the fabrication process. For example, an etching or imprinting process may have a minimum pitch that can be accurately reproduced. As these processes improve (i.e., increased accuracy at smaller sizes) it may be possible to perform the redirecting and focusing the functions of the first metasurface and second metasurface using a single metasurface.

[0060] FIG. 9 is a side-view, ray -trace of light in a single-metasurface camera module according to a possible implementation of the present disclosure. AS in FIG. 6, a camera module 900 includes substrate 910. The substrate may be a glass material (e.g., SiO2) with a first surface 911 facing the user. Other than an optical coating (e.g., antireflection coating), the first surface 911 may have no additional layer for nanostructures. Accordingly, the redirection of the light 410 shown may be due to the refraction of the light 410 at the air-glass interface of the first surface 911. The second surface 912 may include a layer in which a single metasurface is formed. The single metasurface may redirect and focus the light to generate a real-image of the eye on the detector 920. .

[0061] In the specification and / or figures, typical embodiments have been disclosed. The present disclosure is not limited to such exemplary embodiments. For example, a dual- metasurface imaging system has been discussed at length because there may be some advantages to this implementation when it comes to fabrication. That being said, it may be possible to combine the redirection and focusing functions of the dual metasurface into a single metasurface or it may be possible to improve imaging (e.g., reduce distortion and aberrations) using more than two metasurfaces. Further, it may be possible to integrate the camera module in various orientations with respect to the temple arm to enable different applications.

[0062] FIG. 10A illustrates a field-of-view for a dual-metasurface camera module aligned in a first direction according to a possible implementation of the present disclosure. In this configuration, which has been discussed thus far, the first metasurface of the camera module 1001 is flush with the inner surface 1015 of the temple arm 1030, and the camera module 1001 is aligned so that the gradient profile (see FIG. 7A) of the first metasurface is aligned in the Y-direction, as indicated by the arrow. The field-of-view 1010 of the camera module 1001, in this alignment, is directed at the mirror on the lens to capture a digital eye image for eye tracking.

[0063] FIG. 10B illustrates a field-of-view for a dual -metasurface camera module aligned in a second direction according to a possible implementation of the present disclosure. In this configuration, the first metasurface of the camera module 1001 is flush with the inner surface 1015 of the temple arm 1030, and the camera module 1001 is aligned so that the gradient profile of the first metasurface is aligned in the X-direction, as indicated by the arrow. The field-of-view 1010 of the camera module 1001, in this alignment, is directed towards the hands of a user to capture a digital hand image for hand tracking. In general, the camera module may be rotated about its Z-axis (i.e., optical axis) to redirect the field-of-view- towards different areas. The orientation may be determined at a time of fabrication and then not adjusted again.

[0064] In a possible implementation, the head-mounted device includes multiple dual- metasurface camera modules (and possibly illumination modules) integrated flush with the inner surface 1015 of the temple arm 1030 to capture images for different applications (e.g., eye tracking, hand tracking, etc.). An advantage of this approach is that, aside from their orientation, each of the multiple camera modules may be the same (i.e., based on the same design).

[0065] Some implementations may be implemented using various semiconductor processing and / or packaging techniques. Some implementations may be implemented using various types of semiconductor processing techniques associated with semiconductor substrates including, but not limited to, for example, Silicon (Si), Gallium Arsenide (GaAs), Gallium Nitride (GaN), Silicon Carbide (SiC) and / or so forth.

[0066] While certain features of the described implementations have been illustrated as described herein, many modifications, substitutions, changes and equivalents will now occur to those skilled in the art. It is, therefore, to be understood that the appended claims are intended to cover all such modifications and changes as fall within the scope of the implementations. It should be understood that they have been presented by way of example only, not limitation, and various changes in form and details may be made. Any portion of the apparatus and / or methods described herein may be combined in any combination, except mutually exclusive combinations. The implementations described herein can include various combinations and / or sub-combinations of the functions, components and / or features of the different implementations described.

[0067] It will be understood that, in the foregoing description, when an element is referred to as being on, connected to, electrically connected to, coupled to, or electrically coupled to another element, it may be directly on, connected or coupled to the other element,or one or more intervening elements may be present. In contrast, when an element is referred to as being directly on, directly connected to or directly coupled to another element, there are no intervening elements present. Although the terms directly on, directly connected to, or directly coupled to may not be used throughout the detailed description, elements that are shown as being directly on, directly connected or directly coupled can be referred to as such. The claims of the application, if any, may be amended to recite exemplary relationships described in the specification or shown in the figures.

[0068] As used in this specification, a singular form may, unless definitely indicating a particular case in terms of the context, include a plural form. Spatially relative terms (e.g., over, above, upper, under, beneath, below, lower, and so forth) are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. In some implementations, the relative terms above and below can, respectively, include vertically above and vertically below. In some implementations, the term adjacent can include laterally adjacent to or horizontally adjacent to.

Claims

WHAT IS CLAIMED IS:

1. A head-mounted device comprising: a frame that includes a temple arm coupled to a front portion so that when the frame is worn by a user, the front portion is positioned adjacent to the face of the user and the temple arm extends from the front portion towards an ear of the user; a lens supported in front of an eye of the user by the front portion of the frame; a mirror disposed on a surface of the lens to direct light corresponding to a reflection of the eye to a position along the temple arm; and a camera module integrated with the temple arm at the position along the temple arm, the camera module configured to: receive the light from the mirror corresponding to the reflection of the eye; redirect and focus the light using at least one metasurface to generate a real-image of the eye on a detector; and generate a digital eye-image using the detector.

2. The head-mounted device according to claim 1, wherein: the at least one metasurface includes a plurality of pillar nanostructures.

3. The head-mounted device according to claim 2, wherein: the plurality of pillar nanostructures includes pillars of different sizes4. The head-mounted device according to any one of claims 1 to 3, wherein the headmounted device includes: a first metasurface configured to function as a first metasurface lens with a gradient phase profile; and a second metasurface configured to function as a second metasurface lens with a parabolic phase profile, the first metasurface and the second metasurface separated by a substrate.

5. The head-mounted device according to claim 4, wherein: the first metasurface includes a first plurality of nanostructures disposed on a first side of the substrate; andthe second metasurface includes a second plurality of nanostructures disposed on a second side of the substrate, the second side opposite from the first side.

6. The head-mounted device according to claim 4 or 5, wherein the camera module includes: the detector being attached to the substrate so that the substrate and the detector are bisected by an optical axis and are separated along the optical axis by a focal length; and the substrate is positioned in the camera module so that the first metasurface faces away from the detector and the second metasurface faces the detector.

7. The head-mounted device according to claim 6, the camera module is aligned in the head-mounted device when the first metasurface of the camera module is flush with an inner surface of the temple arm facing the user, the camera module requiring no further alignment to receive the light from the mirror.

8. The head-mounted device according to any one of claims 4 to 7, wherein: the gradient phase profile redirects the light from the mirror towards the second metasurface; and the parabolic phase profile focuses the light from the first metasurface onto the detector to form the real-image of the eye.

9. The head-mounted device according to any one of the preceding claims, wherein the position is a first position and the head-mounted device further includes: a metasurface illuminator module integrated flush with the temple arm at a second position along the temple arm, the metasurface illuminator module configured to: project and redirect transmitted light from at least one optical source to the mirror, the mirror reflecting the transmitted light to illuminate the eye for the camera module.

10. The head-mounted device according to any one of the preceding claims, wherein the mirror is a short-pass mirror configured to reflect wavelengths above an infrared wavelength and not reflect wavelengths below the infrared wavelength so that the user may view visible light through the short-pass mirror.

11. The head-mounted device according to any one of the preceding claims, wherein the head-mounted device is augmented reality glasses or smart glasses.

12. The head-mounted device according to any one of the preceding claims, wherein the head-mounted device further includes a processor configured by software instructions to perform an eye-tracking process that includes: receiving the digital eye-image from the detector; detecting a feature of the eye in the digital eye-image; and determining the position of the eye based on a location of the feature in the digital eyeimage.

13. The head-mounted device according to any one of the preceding claims, wherein the detector is not facing the lens and is not facing the eye.

14. A head-mounted device, in particular a head-mounted device according to any one of claims 1 to 13, comprising: a metasurface camera module integrated flush with a temple arm of the head-mounted device at a position along the temple arm, the metasurface camera module including: a first metasurface configured to receive light from a surface of a lens of the headmounted device and redirect the light towards a detector; a second metasurface between the first metasurface and the detector configured to focus the light redirected by the first metasurface to generate a real-time image of an eye on the detector; and a detector configured to generate a digital eye-image of the light focused by the second metasurface, the light corresponding to a reflection of the eye of a user from the surface of the lens.

15. The head-mounted device according to claim 14, wherein the metasurface camera module is aligned in the head-mounted device when the first metasurface is flush with an inner surface of the temple arm.

16. A metasurface camera module to be integrated with a temple arm of a head-mounted device, the metasurface camera module comprising: a first plurality of nanostructures disposed on a first side of a substrate; a second plurality of nanostructures disposed on a second side of the substrate, the second side opposite the first side; and a detector facing the second plurality of nanostructures and separated from the substrate by a focal length along an optical axis, the optical axis bisecting the substrate and the detector.

17. The metasurface camera module according to claim 16, wherein: the first plurality of nanostructures is a first metasurface lens with a gradient phase profile; and the second plurality of nanostructures is a second metasurface lens with a parabolic phase profile.

18. The metasurface camera module according to claim 17, wherein: the gradient phase profile of the first metasurface lens is configured to redirects light from a lens of the head-mounted device towards the second metasurface lens; and the parabolic phase profile of the second metasurface lens is configured to focus light from the first metasurface lens to generate a real eye-image on the detector.

19. The metasurface camera module according to claim 17 or 18, wherein: the first metasurface lens is flat and is configured to be flush with an inner surface of the temple arm facing a user and configured to receive light corresponding to a reflection of an eye of the user from a surface of a lens of the head-mounted device.

20. The metasurface camera module according to any one of claims 17 to 19, wherein: the first metasurface lens and the second metasurface lens are configured to generate a real image of an eye of a user on the detector.

21. The metasurface camera module according to any one of claims 17 to 20, wherein the metasurface camera module is to be integrated with the temple arm of the headmounted device according to any one of claim 1 to 15.

Images