Integrated camera for eye tracking

By integrating a camera into the eyepiece of a head-mounted device and utilizing the air gap surface of the refractive layer to compensate for distortion, the problems of image distortion and high power consumption in eye-tracking technology have been solved, achieving higher sampling resolution and energy efficiency.

CN123113152APending Publication Date: 2026-11-17CTRL-LABS CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202610674195.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2025-05-16
Filing Date
2026-05-15
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing eye-tracking technologies suffer from image distortion and high power consumption when acquiring eye images, especially in head-mounted devices, which affect sampling resolution and device energy efficiency.

Method used

A camera is integrated into the eyepiece of a head-mounted device, and the distortion of the prescription surface is compensated by the air gap surface of the refractive layer, which increases the uniformity of the sampling resolution and expands the field of view of the camera, thereby reducing image distortion.

Benefits of technology

It improves the uniformity of sampling resolution for eye-tracking images, reduces computational and power requirements, and enhances the energy efficiency of the device.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123113152A_ABST
    Figure CN123113152A_ABST
Patent Text Reader

Abstract

This invention relates to an integrated camera for eye tracking. An eyepiece for a head-mounted device includes a refractive layer and a camera. The refractive layer includes a prescription surface on the eye side of the eyepiece. The camera is at least partially encapsulated within the refractive layer. The camera is configured to image a fit-eye region. The air gap surface of the refractive layer is configured to at least partially compensate for distortions in the prescription surface for the fit-eye image to be acquired by the camera.
Need to check novelty before this filing date? Find Prior Art

Description

Cross-references to related applications

[0001] This application claims priority and benefit to U.S. nonprovisional patent application No. 19 / 211,041, filed May 16, 2025, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates generally to optical devices, and more particularly to eye-tracking cameras. Background Technology

[0003] Eye-tracking technology enables head-mounted devices and other devices to interact with users based on their eye movements or orientation. Example eye-tracking techniques involve acquiring images of the eyes and then processing these images to determine eye movement or position. To reduce power consumption and improve accuracy, it is desirable to acquire eye-tracked images with low image distortion. Summary of the Invention

[0004] According to one aspect of this disclosure, a head-mounted device is provided, comprising: an eyepiece including a refractive layer having a prescription curvature on the eye side of the eyepiece; and a camera at least partially encapsulated in the refractive layer, wherein the camera is configured to image an eyebox region, and wherein an air gap surface of the refractive layer is configured to at least partially offset distortion of the prescription curvature to increase the uniformity of sampling resolution of the eyebox image acquired by the camera.

[0005] According to another aspect of this disclosure, an eyepiece for a head-mounted device is provided, the eyepiece comprising: a refractive layer having a prescription surface on the eye side of the eyepiece; and a camera at least partially encapsulated in the refractive layer, wherein the camera is configured to image an eye-fitting frame region, and wherein the air gap surface of the refractive layer is configured to at least partially compensate for distortion of the prescription surface for an eye-fitting frame image to be acquired by the camera.

[0006] According to another aspect of this disclosure, a head-mounted device is provided, comprising: an eyepiece including a refractive layer having a focusing surface on the eye side of the eyepiece; a camera at least partially encapsulated in the refractive layer; and a frame configured to fix the eyepiece, wherein the camera is configured to image an eye-frame region through a cutout in the frame and through the refractive layer, and wherein the focusing surface of the eyepiece extends the field-of-view (FOV) of the camera. Attached Figure Description

[0007] Non-limiting and non-exhaustive embodiments of the invention are described with reference to the following drawings, wherein, unless otherwise stated, the same reference numerals in the various views always denote the same parts.

[0008] Figure 1 A head-mounted device that may include an eye-tracking camera is shown according to various aspects of this disclosure.

[0009] Figure 2 An example functional diagram of an eyepiece fixed by a frame according to various aspects of this disclosure is shown.

[0010] Figure 3 A camera, at least partially encapsulated in a refractive layer according to various aspects of this disclosure, is shown.

[0011] Figure 4 The various aspects shown in this disclosure are as follows Figure 3 An enlarged view showing the air gap voids and air gap surface of the refractive layer.

[0012] Figure 5 A camera including an air gap setting element is shown according to various aspects of this disclosure.

[0013] Figure 6 The optical structure of an eye-tracking camera integrated with a base refractive layer, according to various aspects of this disclosure, is shown.

[0014] Figure 7 A focusing and refractive layer molded on an air gap setting element according to various aspects of this disclosure is shown.

[0015] Figure 8 An example functional diagram of an eyepiece fixed by a frame having a cutout for a camera is shown according to various aspects of this disclosure.

[0016] Figure 9A and Figure 9B Example cutouts in the frame of a head-mounted device according to various aspects of this disclosure are shown.

[0017] Figure 10 An example functional diagram of a frame-fixed eyepiece according to various aspects of this disclosure is shown, wherein the focusing surface of the refractive optics includes a first surface that transitions to a second surface that is planar.

[0018] Figure 11 An example functional diagram of a frame-fixed eyepiece according to various aspects of this disclosure is shown, wherein the focusing surface of the refractive optics includes a first surface that transitions to a non-planar second surface.

[0019] Figure 12An example functional diagram of an eyepiece including a display waveguide according to various aspects of this disclosure is shown, the display waveguide including a cutout in the display waveguide.

[0020] Figure 13 An example functional diagram of an eyepiece according to various aspects of this disclosure is shown, wherein the camera is not encapsulated and the prism is included in a refractive optics. Detailed Implementation

[0021] This document describes embodiments of an integrated camera for eye tracking. Numerous specific details are set forth in the following description to provide a thorough understanding of the various embodiments. However, those skilled in the art will recognize that the techniques described herein can be implemented without one or more of these specific details, or using other methods, components, materials, etc. In other instances, well-known structures, materials, or operations have not been shown or described in detail to avoid obscuring certain aspects.

[0022] Throughout this specification, references to "an embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment of the invention. Therefore, the appearance of the phrase "in one embodiment" or "in one embodiment" throughout this specification does not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0023] In some embodiments of this disclosure, the term "near eye" may be defined as including an element configured to be placed within 50 millimeters (mm) of a user's eye when using a near eye device. Therefore, a "near eye optical element" or "near eye system" will include one or more elements configured to be placed within 50 mm of a user's eye.

[0024] In various aspects of this disclosure, visible light can be defined as having a wavelength range of about 380 nanometers (nm) to 700 nm. Invisible light can be defined as light having wavelengths outside the visible light range, such as ultraviolet light and infrared light. Infrared light having a wavelength range of about 700 nm to 1 mm includes near-infrared light. In various aspects of this disclosure, near-infrared light can be defined as having a wavelength range of about 700 nm to 1.6 µm.

[0025] In various aspects of this disclosure, the term "transparent" may be defined as having a light transmittance of greater than 90%. In some aspects, the term "transparent" may be defined as a material having a visible light transmittance of greater than 90%.

[0026] It is desirable for eye-tracking cameras to fit the form factor of wearable devices. Examples of head-mounted devices that can include eye-tracking cameras include smart glasses, augmented reality (AR) glasses, mixed reality (MR) headsets, and virtual reality (VR) headsets. It is helpful for eye-tracking cameras to have a field of view that is not obstructed by hair or eyelashes, and reducing image distortion in eye-tracked images captured by eye-tracking cameras helps reduce computational and power requirements.

[0027] In various embodiments of this disclosure, an eye-tracking camera is integrated with an eyepiece, which is secured by the frame of a head-mounted device. The camera can be partially encapsulated by a refractive layer of the eyepiece, which includes a focusing surface. The eye-tracking camera can image through the focusing surface of the refractive layer to acquire an image of the user's eye within the eye-fitting frame region. To reduce distortion in the eye-fitting frame image, the air gap surface of the refractive layer can at least partially compensate for the distortion caused by the focusing surface of the refractive layer. Distortion compensation can include increasing the uniformity of the sampling resolution of the eye-fitting frame image acquired by the camera.

[0028] In various embodiments of this disclosure, the eye-tracking camera is integrated with the refractive layer of an eyepiece, which is fixed by the frame of a head-mounted device. The eye-tracking camera is configured to image the eye-frame area through a cutout in the frame and via the refractive layer of the eyepiece. The focusing surface of the refractive layer can extend the field of view (FOV) of the eye-tracking camera. Combined with Figures 1 to 13 These and other embodiments will be described in more detail.

[0029] Figure 1 A head-mounted device 100, which may include an eye-tracking camera, is illustrated according to various aspects of this disclosure. The head-mounted device 100 includes a frame 114 coupled to arms 111A and 111B. In some embodiments, the frame 114 may be mechanically hinged to the arms 111A and 111B. Eyepieces 121A and 121B are secured by the frame 114. Eyepieces 121A and 121B may include prescription surfaces tailored to a specific user of the head-mounted device 100. The illustrated head-mounted device 100 is configured to be worn on or around the head of the wearer of the head-mounted device 100.

[0030] exist Figure 1In the illustrated head-mounted device 100, each eyepiece 121A / 121B includes a waveguide 150A / 150B for guiding image light generated by the display 130A / 130B to the eye-fitting frame area for viewing by the user of the head-mounted device 100. The display 130A / 130B may include, for example, a beam-scanning display or a liquid crystal on silicon (LCOS) display for guiding display light to the wearer of the head-mounted device 100 to present a virtual image. A head-mounted device including a display can be considered a head-mounted display (HMD).

[0031] Eyepieces 121A and 121B can be transparent or semi-transparent to the user to facilitate augmented or mixed reality, allowing the user to view scene light from their surroundings while also receiving display light directed to the user's eyes via, for example, waveguides 150A and 150B. Eyepieces 121A and 121B may include two or more optical layers for different functions, such as display, eye tracking, and optical power. In some embodiments, display light from displays 130A or 130B is directed to only one eye of the wearer of the head-mounted device 100. In one embodiment, displays 130A and 130B are used to direct display light into waveguides 150A and 150B, respectively. Embodiments of this disclosure can also be used for head-mounted devices (e.g., smart glasses) that do not necessarily include displays but are configured to be worn on or around the wearer's head.

[0032] Frame 114 and arms 111A, 111B may include supporting hardware for the head-mounted device 100 (e.g., processing logic 107), wired and / or wireless data interfaces for transmitting and receiving data, a graphics processor, and one or more memories for storing data and computer-executable instructions. Processing logic 107 may include circuitry, logic, instructions stored in a machine-readable storage medium, application-specific integrated circuit (ASIC) circuitry, field-programmable gate array (FPGA) circuitry, and / or one or more processors. In one embodiment, the head-mounted device 100 may be configured to receive wired power. In one embodiment, the head-mounted device 100 is configured to be powered by one or more batteries. In one embodiment, the head-mounted device 100 may be configured to receive wired data, including video data, via a wired communication channel. In one embodiment, the head-mounted device 100 is configured to receive wireless data, including video data, via a wireless communication channel. Processing logic 107 may be communicatively coupled to network 180 to provide data to network 180 and / or access data within network 180. The communication channel between the processing logic 107 and the network 180 can be wired or wireless.

[0033] exist Figure 1 In the illustrated embodiment, the head-mounted device 100 includes example eye-tracking cameras 147 and 148. Cameras 147 and 148 are configured to image the eye-fitting region. Cameras 147 and 148 illustrate an example placement of the eye-tracking camera integrated with eyepiece 121B. Camera 147 illustrates an example nasal region positioning of the eye-tracking camera, while camera 148 illustrates an example temporal region positioning of the eye-tracking camera. Other locations are also possible. Although not specifically shown, the eye-tracking camera may also be integrated with eyepiece 121A to generate an eye-fitting image of the right eye of the wearer of the head-mounted device 100.

[0034] In some embodiments, the illumination module 160 may illuminate the eye-fitting region with near-infrared illumination light to assist cameras 147 and / or 148 in imaging the eye-fitting region for eye-tracking purposes. In some embodiments, cameras 147 and / or 148 may include lens assemblies configured to focus image light onto a complementary metal-oxide-semiconductor (CMOS) image sensor. A near-infrared filter that receives narrowband near-infrared wavelengths may be placed on the image sensor to make it sensitive to narrowband near-infrared wavelengths while rejecting visible light and wavelengths outside this narrowband. Near-infrared illuminators (not shown), such as near-infrared light-emitting diodes (LEDs) or lasers, that emit narrowband wavelengths may be included in the illumination module 160 to illuminate the eye-fitting region with narrowband near-infrared wavelengths.

[0035] Figure 2 An example functional diagram of an eyepiece 221 fixed by a frame 214 according to various aspects of this disclosure is shown. The example eyepiece 221 includes a refractive optics 230, a display waveguide 240, and a lens layer 250. Figure 2 In the illustrated embodiment, the display waveguide 240 is disposed between the lens layer 250 and the refractive optics 230. In this embodiment, the display waveguide 240 is spaced apart from both the lens layer 250 and the refractive optics 230. In some embodiments, one or more of the refractive optics 230, the display waveguide 240, and the lens layer 250 are combined. The refractive optics 230, the display waveguide 240, and the lens layer 250 may be combined at the edge (not specifically shown) of the eyepiece 221, while being spaced apart at the center of the eyepiece 221.

[0036] The refractive optics 230 and lens layer 250 may be made of a refractive material. The refractive optics 230 includes a focusing surface 231, which may be a prescription surface specific to a particular wearer of the head-mounted device. The focusing surface 231 may be configured to focus received display light 293 from the display waveguide 240 onto the eye-fitting frame region 290. The display light 293 may include a virtual image generated by the display module 245 and guided into the waveguide 240 to be presented to the eye-fitting frame region 290. For example, the display module 245 may include features described with respect to displays 130A, 130B. In some embodiments, the display module 245 may be at least partially included in the temporal region of the frame 214.

[0037] In some embodiments, the optical power imparted by the curved surface 251 of the lens layer 250, combined with the focusing surface 231 of the refractive optics 230, focuses scene light 299 onto the eye-fitting frame region 290. Scene light 299 can be reflected or scattered from the external environment. For example, a light source (e.g., sunlight or electric lighting) can illuminate objects in the external environment, and the objects reflect and / or scatter scene light 299. In some embodiments, the optical power of the focusing surface 231 compensates for the optical power imparted by the curved surface 251.

[0038] Figure 3 A camera 360, at least partially encapsulated in a refractive layer 330 according to various aspects of this disclosure, is shown. The refractive layer 330 includes a focusing refractive layer 336 and a base refractive layer 337. The focusing refractive layer 336 includes a focusing surface 372, which may be a prescription focusing surface of an optical prescription specific to a particular wearer of a head-mounted device. The base refractive layer 337 is coupled to the focusing refractive layer 336. In some embodiments, the base refractive layer 337 may be coupled to the focusing refractive layer 336 using a refractive index-matched optical adhesive. The base refractive layer 337 may include a planar side 373 opposite to the focusing surface 372 of the focusing refractive layer 336.

[0039] An example eye-tracking camera 360 includes a lens assembly 363 configured to focus image light onto an image sensor 367. The lens assembly 363 may have multiple optical elements, such as refractive lenses, diffractive optics, and / or filters, for focusing and / or filtering image light arriving at the pixels of the image sensor 367. Filters included in the lens assembly 363 may include bandpass near-infrared filters. The image sensor 367 may be electrically coupled to a circuit board 361, which provides traces for transmitting power and data to the image sensor 367. In some embodiments, the image sensor 367 may be a CMOS image sensor. The camera 360 is configured to image an eye 303 in an eye-fitting frame region 301. Notably, the camera 360 is configured to image the eye-fitting frame region 301 via a focusing refractive layer 336, thus the focusing surface 372 affects the field of view (FOV) 369 of the camera 360. The focusing surface 372 may distort the eye-fitting frame image captured by the eye-tracking camera 360.

[0040] Figure 4 The various aspects shown in this disclosure are as follows Figure 3 An enlarged view showing the air gap 353 and air gap surface 371 of the refractive layer 330. Figure 4In this configuration, the air gap surface 371 extends along the boundary of the air gap 353 formed in the focusing refractive layer 336. The air gap surface 371 of the refractive layer 330 is configured to at least partially compensate for the distortion of the focusing surface 372 to increase the uniformity of the sampling resolution of the eye-fitting frame image acquired by the camera 360. The focusing surface 372 may be a prescription surface. Increasing the uniformity of the sampling resolution of the eye-fitting frame image may be particularly important when the shape of the focusing surface 372 reduces the resolution of the eye-fitting frame image on one side of the image while increasing the resolution on the opposite side. This distortion then requires increased image processing computational power to correct the distortion caused by the focusing surface 372. In some embodiments, the air gap 353 may be only slightly wider than the lens assembly 363. In some embodiments, the air gap 353 may be smaller than the width of the lens assembly 363. The air gap 353 and the air gap surface 371 are located on the side of the focusing refractive layer 336 opposite to the focusing surface 372.

[0041] In some embodiments, the air gap surface 371 compensates for the optical power of the focusing surface 372. In some embodiments, the focusing surface 372 imparts negative optical power, and the air gap surface 371 imparts positive optical power. In some embodiments, the air gap surface 371 has the same slope and curvature as the focusing surface 372.

[0042] In some embodiments, the air gap surface 371 is approximately planar and serves as a flat wedge to guide image light to the camera 360. In some embodiments, the focusing surface 372 is a cylindrical focusing surface, and the air gap surface 371 is configured to counteract the cylindrical power of the cylindrical focusing surface 372. In some embodiments, the focusing surface 372 is a spherical focusing surface, and the air gap surface 371 is configured to counteract the spherical power of the spherical focusing surface 372.

[0043] Figure 5 A camera 560 including an air gap setting element 580 according to various aspects of this disclosure is shown. The camera 560 includes an image sensor 367 and a lens assembly 363 configured to focus image light onto the imaging plane of the image sensor 367. The air gap setting element 580 is coupled to the lens assembly 363. In some embodiments, the air gap setting element 580 is adhered to the lens assembly 363. The air gap setting element 580 may be made of a transparent material. The air gap setting element 580 may be made of a transparent material having the same or approximately the same refractive index as the focusing refractive layer that will contact or adhere to the air gap setting element 580.

[0044] Air gap setting element 580 provides air gap 553 and defines air gap surface 571. Air gap surface 571 can be planar, spherical, aspherical, cylindrical, or free-form. Air gap surface 571 can have the features described with respect to air gap surface 371.

[0045] In some implementations, various air gap setting elements can be matched to various optical formulations associated with different focusing surfaces 772. For example, ten different air gap setting elements may correspond to ten different focusing surfaces defined by ten different molds. In this way, the air gap surface 571 of the air gap setting element can be matched to a specific focusing surface 772 of a formulation to compensate for distortion of the focusing surface, thereby increasing the uniformity of the sampling resolution of the eye-fitting frame image acquired by the eye-tracking camera.

[0046] Figure 6 An optical structure 699 according to various aspects of this disclosure is shown, which includes an eye-tracking camera 560 integrated with a base refractive layer 337. In some embodiments, the eye-tracking camera 560 may be encapsulated within the base refractive layer 337.

[0047] Figure 7 A focusing refractive layer 736 molded on an air gap setting element 580 according to various aspects of this disclosure is shown. In some embodiments of this disclosure, the focusing refractive layer 736 is overmolded onto an optical structure 699 to form the optical structure 799. In this embodiment, the dimensions of the focusing refractive layer 736 are determined by a mold, which may be injected with a moldable refractive material to form the focusing refractive layer 736. The focusing surface 772 may also be determined by the boundaries of the mold, while the air gap setting element 580 defines an air gap surface 571 because the moldable refractive material is provided on the air gap surface 571 and subsequently cured. The air gap surface 571 may include the features and characteristics described with respect to the air gap surface 371.

[0048] In some embodiments, the air gap surface 571 compensates for the optical power of the focusing surface 772. In some embodiments, the focusing surface 772 imparts negative optical power, and the air gap surface 571 imparts positive optical power. In some embodiments, the air gap surface 571 has the same slope and curvature as the focusing surface 772.

[0049] In some embodiments, the air gap surface 571 is approximately planar and serves as a flat wedge to guide image light to the camera 360. In some embodiments, the focusing surface 772 is a cylindrical focusing surface, and the air gap surface 571 is configured to counteract the cylindrical power of the cylindrical focusing surface 772. In some embodiments, the focusing surface 772 is a spherical focusing surface, and the air gap surface 571 is configured to counteract the spherical power of the spherical focusing surface 772.

[0050] Figure 8 An example functional diagram of an eyepiece 821 fixed by a frame 814, which has a cutout for a camera 860, is shown according to various aspects of this disclosure. The example eyepiece 821 includes refractive optics 830, a display waveguide 840, and a lens layer 250. Figure 8 In this configuration, a display waveguide 840 is disposed between a lens layer 250 and a refractive optics device 830. Both the refractive optics device 830 and the lens layer 250 may be made of a refractive material. The refractive optics device 830 includes a focusing surface 831 on its eye-side 896, which may be a prescription surface specific to a particular wearer of the head-mounted device. The focusing surface 831 may be configured to focus received display light 293 from the display waveguide 840 onto the eye-fitting frame region 290. The display light 293 may include a virtual image generated by a display module (not specifically shown) and guided into the waveguide 840 to be presented to the eye-fitting frame region 290. Figure 8 The nasal region of frame 814 can be shown, and the display module can be positioned in the temporal region of frame 814.

[0051] In some embodiments, the optical power imparted by the curved surface 251 of the lens layer 250, combined with the focusing surface 831 of the refractive optics 830, focuses scene light 299 onto the eye-fitting frame region 290. Scene light 299 can be reflected or scattered from the external environment. In some embodiments, the optical power of the focusing surface 831 compensates for the optical power imparted by the curved surface 251.

[0052] Figure 8 The frame 814 is shown extending beyond the camera 860 and also partially extending over the refractive optics 830. The dashed lines shown represent the portion of the frame 814 that extends over the refractive optics 830. The camera 860 is configured to pass through a cutout in the frame 814 ( Figure 9A and Figure 9B (The example cut shown in the figure) and the eye-fitting frame region 290 is imaged by the refractive optics 830. The focusing surface 831 of the refractive optics 830 helps to extend the FOV 869 of the camera 860 to include most of the eye-fitting frame region 290 and focus the eye-fitting frame image light 291 onto the camera 860.

[0053] Figure 9A Example cutout 933 is shown in framework 914 according to various aspects of this disclosure. Figure 9A A portion of the head-mounted device 900 is shown from the user's perspective. In other words, Figure 9AThe eye side of frame 914 is shown, not the world-facing side. The head-mounted device 900 includes a frame 914 that holds an eyepiece 921 in place. Frame 914 includes a bridge of the nose region 916, and a cutout 933 is located in or near the bridge of the nose region 916. Therefore, camera 860 can image the eye region through the cutout 933 and the refractive layer of the eyepiece 921. The cutout 933 is a gap in frame 914 that is also surrounded by frame 914. Although the cutout 933 is shown near the nose region of frame 914, the cutout 933 could be located in the temporal region of the eyeglass frame 914 or elsewhere on frame 914.

[0054] Figure 9B An example cutout 934 is shown in framework 915 according to various aspects of this disclosure. Figure 9B A portion of the head-mounted device 901 is shown from the user's perspective. The head-mounted device 901 includes a frame 915 that holds an eyepiece 922 in place. The frame 915 includes a bridge of the nose region 917, and a cutout 934 is located in or near the bridge of the nose region 917. Therefore, a camera 860 can image the eye socket region through the cutout 934 and the refractive layer of the eyepiece 922. The cutout 934 is a notch in the frame 915, but is not completely enclosed by the frame 915. Although the cutout 934 is shown near the nose region of the frame 915, the cutout 934 could be located in the temporal region of the eyeglass frame 915 or elsewhere on the eyeglass frame 915.

[0055] Figure 10 An example functional diagram of an eyepiece 1021 fixed by a frame 1014 according to various aspects of this disclosure is shown, wherein the focusing surface of the refractive optics 1030 includes a first surface 1031 transitioning to a second surface 1032 that forms a plane. The example eyepiece 1021 includes the refractive optics 1030, a display waveguide 840, and a lens layer 250. Figure 10In the display, waveguide 840 is disposed between lens layer 250 and refractive optics 1030. Both refractive optics 1030 and lens layer 250 may be made of refractive material. Refractive optics 1030 includes a focusing surface on the eye side 1096 of the head-mounted device, comprising a first surface 1031 and a second surface 1032. The first surface 1031 may be a prescription surface specific to a particular wearer of the head-mounted device. The first surface 1031 of the focusing surface can be configured to focus received display light 293 from display waveguide 840 onto the eye-fitting frame region 290. The planar second surface 1032 of the focusing surface of refractive optics 1030 is configured to extend the field of view 1069 of camera 860 to image a wider eye-fitting frame region 290. Utilizing the planar second surface 1032 simplifies manufacturing processes and reduces costs.

[0056] The camera 860 can be integrated with the refractive optics 1030 to reduce the shape features of the head-mounted device while providing the camera 860 with a suitable viewing angle for imaging the eye frame region 290. In some embodiments, the camera 860 is at least partially encapsulated with the refractive optics 1030.

[0057] In some embodiments, the optical power imparted by the curved surface 251 of the lens layer 250, combined with the curvature of the first surface 1031 of the refractive optics 1030, focuses scene light 299 onto the eye-fitting frame region 290. In some embodiments, the optical power of the first surface 1031 of the focusing surface compensates for the optical power imparted by the curved surface 251.

[0058] For clarity, Figure 10 The portion of frame 1014 (not shown) extends above the refractive optics 1030. However, camera 860 is configured to pass through a cutout in frame 1014 ( Figure 9A and Figure 9B (Example cut shown in the figure) and imaging of the eye frame region 290 by means of refractive optics 1030.

[0059] Figure 11 An example functional diagram of an eyepiece 1121 fixed by a frame 1114 according to various aspects of this disclosure is shown, wherein the focusing surface of the refractive optics 1130 includes a first surface 1131 that transitions to a non-planar second surface 1132. The example eyepiece 1121 includes the refractive optics 1130, a display waveguide 840, and a lens layer 250. Figure 11In the display, waveguide 840 is disposed between lens layer 250 and refractive optics 1130. Both refractive optics 1130 and lens layer 250 may be made of refractive material. Refractive optics 1130 includes a focusing surface on the eye side 1196 of the head-mounted device, comprising a first surface 1131 and a second surface 1132. The first surface 1131 may be a prescription surface specific to a particular wearer of the head-mounted device. The first surface 1131 of the focusing surface may be configured to focus received display light 293 from display waveguide 840 onto the eye-fitting frame region 290. The second surface 1132 of the focusing surface of refractive optics 1130 is configured to extend the field of view 1169 of camera 860 to image a wider eye-fitting frame region 290. The second surface 1132 is non-planar. The second surface 1132 may be spherical, aspherical, cylindrical, or freeform.

[0060] The camera 860 can be integrated with the refractive optics 1130 to reduce the shape features of the head-mounted device while providing the camera 860 with a suitable viewing angle to image the eye frame region 290. In some embodiments, the camera 860 is at least partially encapsulated with the refractive optics 1130.

[0061] In some embodiments, the optical power imparted by the curved surface 251 of the lens layer 250, combined with the curvature of the first surface 1131 of the refractive optics 1130, focuses scene light 299 onto the eye-fitting frame region 290. In some embodiments, the optical power of the first surface 1131 of the focusing surface compensates for the optical power imparted by the curved surface 251.

[0062] For clarity, Figure 11 The portion of frame 1114 not shown extends above the refractive lens 1130. However, camera 860 is configured to pass through a cutout in frame 1114 ( Figure 9A and Figure 9B (Example cut shown in the figure) and imaging of the eye frame region 290 by means of refractive optics 1130.

[0063] Figure 12 An example functional diagram of an eyepiece 1221 including a display waveguide 1240 according to various aspects of this disclosure is shown. The display waveguide 1240 includes a cutout 1243 in the display waveguide. The example eyepiece 1221 includes a refractive optics 1230, a display waveguide 1240, and a lens layer 250. Figure 12In this configuration, a display waveguide 1240 is disposed between a lens layer 250 and a refractive optics device 1230. Both the refractive optics device 1230 and the lens layer 250 may be made of a refractive material. The refractive optics device 1230 includes a focusing surface on its eye side 1296, comprising a first surface 1231 and a second surface 1232. The first surface 1231 may be a prescription surface specific to a particular wearer of the head-mounted device. The first surface 1231 of the focusing surface may be configured to focus received display light 293 from the display waveguide 840 onto the eye-fitting frame region 290. The second surface 1232 of the focusing surface of the refractive optics device 1230 is configured to extend the field of view 1269 of the camera 1260 to image a wider eye-fitting frame region 290.

[0064] exist Figure 12 In the refracting optics 1230, a refractive extension layer 1239 is included. The refractive extension layer 1239 can be used to encapsulate the camera 1260 so as to fix the camera 1260 at a specific position relative to the refractive optics 1230.

[0065] Figure 12 The image also shows that the camera 1260 is positioned within a cutout 1243 in the display waveguide 1240. The cutout in the display waveguide 1240 is likely necessary to accommodate the occupied area of ​​the positioning camera 1260.

[0066] In some embodiments, the optical power imparted by the curved surface 251 of the lens layer 250, combined with the curvature of the first surface 1231 of the refractive optics 1230, focuses scene light 299 onto the eye-fitting frame region 290. In some embodiments, the optical power of the first surface 1231 of the focusing surface compensates for the optical power imparted by the curved surface 251.

[0067] For clarity, Figure 12 The portion of frame 1214 (not shown) extends above the refractive optics 1230. However, camera 1260 is configured to pass through a cutout in frame 1214 ( Figure 9A and Figure 9B (Example cut shown in the figure) and imaging of the eye frame region 290 by means of refractive optics 1230.

[0068] Figure 13 An example functional diagram of an eyepiece 1321 according to various aspects of this disclosure is shown, wherein the camera 1360 is not packaged, and the prism 1339 is included in the refractive optics 1330. The example eyepiece 1321 includes the refractive optics 1330, a display waveguide 1340, and a lens layer 1350. Figure 13In this configuration, a waveguide 1340 is disposed between a lens layer 1350 and a refractive optics device 1330. Both the refractive optics device 1330 and the lens layer 1350 may be made of a refractive material. The refractive optics device 1330 includes a focusing surface on its eye side 1396, comprising a first surface 1331 and a second surface 1332. The first surface 1331 may be a prescription surface specific to a particular wearer of the head-mounted device. The first surface 1331 of the focusing surface may be configured to focus received display light 293 from the display waveguide 1340 onto the eye-fitting frame region 290. The second surface 1332 of the focusing surface of the refractive optics device 1330 is configured to extend the field of view (FOV) 1369 of the camera 1360 to image a wider eye-fitting frame region 290.

[0069] exist Figure 13 In this embodiment, prism 1339 is included within refractive optics 1330. In some embodiments, prism 1339 may be a material continuous with refractive optics 1330. In some embodiments, prism 1339 is a feature in a mold in which refractive optics 1330 and prism 1339 are manufactured in the same injection molding process. In some embodiments, prism 1339 may be adhered to refractive optics using an adhesive with a matching refractive index. Prism 1339 is disposed between display waveguide 1340 and the camera side of refractive optics 1330. Prism 1339 may also be used to shape and expand the field of view (FOV) 1369 of camera 1360. A second surface 1332 may be combined with prism 1339 to shape and expand the FOV 1369 of camera 1360.

[0070] To accommodate the prism 1339 and the unencapsulated camera 1360, cutouts may be required in the waveguide 1340. Furthermore, the lens layer 1350 can be shortened compared to the lens layer 1250 to provide space for the camera 1360.

[0071] In some embodiments, the optical power imparted by the curved surface 1351 of the lens layer 1350, combined with the curvature of the first surface 1331 of the refractive optics 1330, focuses scene light 299 onto the eye-fitting frame region 290. In some embodiments, the optical power of the focusing surface 1331 compensates for the optical power imparted by the curved surface 1351.

[0072] For clarity, Figure 13 The portion of frame 1314 (not shown) extends above the refractive optics 1330. However, camera 1360 is configured to pass through a cutout in frame 1314 ( Figure 9A and Figure 9B (Example cut shown in the figure) and imaging of the eye frame region 290 by means of refractive optics 1330.

[0073] Embodiments of the present invention may include artificial reality systems or be implemented in combination with artificial reality systems. Artificial reality is a form of reality that has been modulated in some way before being presented to a user. This form of reality may include, for example, virtual reality (VR), augmented reality (AR), mixed reality (MR), hybrid reality, or some combination and / or derivative thereof. Artificial reality content may include fully generated content or generated content combined with acquired (e.g., real-world) content. Artificial reality content may include video, audio, haptic feedback, or some combination thereof, any one of which may be presented in a single channel or in multiple channels (e.g., stereoscopic video that produces a three-dimensional effect for the viewer). Furthermore, in some embodiments, artificial reality may also be associated with applications, products, accessories, services, or some combination thereof for, for example, creating content in artificial reality and / or otherwise using it in artificial reality (e.g., performing activities in artificial reality). Artificial reality systems that deliver artificial reality content can be implemented on a variety of platforms, including head-mounted displays (HMDs) connected to a host computer system, stand-alone HMDs, mobile devices or computing systems, or any other hardware platform capable of delivering artificial reality content to one or more viewers.

[0074] The term "processing logic" (e.g., processing logic 107) in this disclosure may include one or more processors, one or more microprocessors, one or more multi-core processors, one or more application-specific integrated circuits (ASICs), and / or one or more field-programmable gate arrays (FPGAs) that perform the operations disclosed herein. In some embodiments, memory (not shown) is integrated into the processing logic to store instructions for performing operations and / or to store data. According to embodiments of this disclosure, the processing logic may also include analog or digital circuitry for performing operations.

[0075] The term "one memory" or "multiple memories" described in this disclosure may include one or more volatile or non-volatile memory architectures. The "one memory" or "multiple memories" may be removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Example memory technologies may include RAM, ROM, EEPROM, flash memory, CD-ROM, digital versatile disk (DVD), high-definition multimedia / data storage disk, or other optical storage devices, magnetic cassette, magnetic tape, disk storage devices or other magnetic storage devices, or any other non-transfer medium that can be used to store information for access by a computing device.

[0076] A network can include any network or network system, such as, but not limited to, the following: peer-to-peer network; local area network (LAN); wide area network (WAN); public network (e.g., the Internet); private network; cellular network; wireless network; wired network; a combination of wireless and wired network; and satellite network.

[0077] The communication channel may include one or more wired or wireless communications, or be routed via one or more wired or wireless communications using the following methods: IEEE 802.11 protocol, short-range wireless protocol, Serial Peripheral Interface (SPI), or Inter-Integrated Circuit (I2C). 2 C) Universal Serial Port (USB), Controller Area Network (CAN), Cellular Data Protocols (e.g., 3G, 4G, LTE, 5G), Optical Communication Networks, Internet Service Providers (ISPs), Peer-to-Peer Networks, Local Area Networks (LANs), Wide Area Networks (WANs), Public Networks (e.g., the "Internet"), Private Networks, Satellite Networks, or other networks.

[0078] Computing devices may include desktop computers, laptop computers, tablet computers, phablets, smartphones, feature phones, server computers, or other devices. Server computers may be located remotely in a data center or may be stored locally.

[0079] The processes explained above are described in terms of computer software and hardware. The described techniques can be embodied in machine-executable instructions stored on a tangible or non-transitory machine-readable storage medium, which, when executed by a machine, will cause the machine to perform the described operations. Furthermore, these processes can be embodied in hardware (e.g., application-specific integrated circuits (“ASICs”) or other hardware).

[0080] Tangible, non-transitory, machine-readable storage media include any mechanism that provides (i.e., stores) information in a form accessible by a machine (e.g., a computer, network device, personal digital assistant, manufacturing tool, any device having one or more processors, etc.). For example, machine-readable storage media include recordable / non-recordable media (e.g., read-only memory (ROM), random access memory (RAM), disk storage media, optical storage media, flash memory devices, etc.).

[0081] The above description of the embodiments of the invention illustrated (including the content described in the abstract) is not intended to be exhaustive or to limit the invention to the precise forms disclosed. While specific embodiments and examples of the invention have been described herein for illustrative purposes, those skilled in the art will recognize that various modifications are possible within the scope of the invention.

[0082] Based on the above detailed description, these modifications can be made to the present invention. The terminology used in the appended claims should not be construed as limiting the invention to the specific embodiments disclosed in the specification. Rather, the scope of the invention will be determined entirely by the appended claims, which will be interpreted according to the established principles of claim interpretation.

Claims

1. A head-mounted device, the head-mounted device comprising: An eyepiece, the eyepiece including a refractive layer having a prescription curved surface on the eye side of the eyepiece; as well as A camera, at least partially encapsulated in the refractive layer, wherein the camera is configured to image an eye-fitting region, and wherein the air gap surface of the refractive layer is configured to at least partially compensate for distortion of the prescription surface to increase the uniformity of the sampling resolution of the eye-fitting image acquired by the camera.

2. The head-mounted device according to claim 1, wherein, The air gap surface of the refractive layer compensates for the prescription optical power of the prescription surface.

3. The head-mounted device according to claim 1, wherein, The prescription surface imparts negative optical power, and the air gap surface of the refractive layer imparts positive optical power.

4. The head-mounted device according to claim 1, wherein, The air gap surface of the refractive layer has the same slope and curvature as the prescription surface.

5. The head-mounted device according to claim 1, further comprising: An air gap setting element coupled to the lens assembly of the camera, wherein the air gap setting element is defined by the refractive layer molded on the air gap setting element, defining the air gap surface.

6. The head-mounted device according to claim 1, wherein, The refractive layer includes a base refractive layer and a focusing refractive layer, the focusing refractive layer including the prescription surface, and wherein the base refractive layer is coupled to the focusing refractive layer, and the air gap surface is along the boundary of the air gap void formed in the focusing refractive layer.

7. The head-mounted device according to claim 1, wherein, The air gap surface is approximately planar and serves as a flat wedge to guide image light to the camera.

8. The head-mounted device according to claim 1, wherein, The prescription surface is a cylindrical prescription surface, and the air gap surface is configured to counteract the cylindrical optical power of the cylindrical prescription surface.

9. The head-mounted device according to claim 1, wherein, The prescription surface is a spherical prescription surface, and the air gap surface is configured to counteract the spherical optical power of the spherical prescription surface.

10. The head-mounted device according to claim 1, wherein, The prescription surface is configured to focus display light, including virtual images, onto the eye-friendly frame area.

11. An eyepiece for a head-mounted device, the eyepiece comprising: A refractive layer having a prescription curved surface on the eyepiece side; as well as A camera, at least partially encapsulated in the refractive layer, wherein the camera is configured to image an eye-fitting region, and wherein the air gap surface of the refractive layer is configured to at least partially compensate for distortion of the prescription surface for an eye-fitting image to be acquired by the camera.

12. The eyepiece according to claim 11, wherein, The air gap surface of the refractive layer compensates for the prescription optical power of the prescription surface.

13. The eyepiece according to claim 11, wherein, The prescription surface imparts negative optical power, and the air gap surface of the refractive layer imparts positive optical power.

14. The eyepiece according to claim 11, wherein, The air gap surface of the refractive layer has the same slope and curvature as the prescription surface.

15. The eyepiece according to claim 11, further comprising: Display waveguides are configured to direct display light to the eye-fitting frame region, wherein the prescription surface is configured to focus display light, including a virtual image, onto the eye-fitting frame region.

16. The eyepiece according to claim 15, further comprising: A lens layer, wherein the display waveguide is disposed between the lens layer and the refracting layer's surface. The optical power of the lens layer, combined with the prescription surface, is configured to focus scene light onto the eye-fitting frame region, the scene light being reflected or scattered from the external environment.

17. A head-mounted device, the head-mounted device comprising: An eyepiece, the eyepiece including a refractive layer having a focusing surface on the eye side of the eyepiece; A camera, said camera being at least partially encapsulated in the refractive layer; as well as A frame configured to fix the eyepiece, wherein the camera is configured to image an eye-fitting frame region through a cutout in the frame and through the refractive layer, and wherein the focusing surface of the eyepiece extends the field of view (FOV) of the camera.

18. The head-mounted device of claim 17, further comprising: The display waveguide is configured to guide display light to the eye-fitting frame region, wherein the focusing surface is configured to focus display light including a virtual image onto the eye-fitting frame region, and wherein the display waveguide includes a cutout for accommodating the camera, the camera being disposed within the cutout of the display waveguide.

19. The head-mounted device according to claim 17, wherein, The focusing surface includes a first surface that transitions to a second surface, wherein the first surface is configured to focus light onto an eye-fitting frame region, and wherein the second surface is configured to focus eye-fitting frame image light onto the camera, the second surface being configured to increase the field of view (FOV) of the camera.

20. The head-mounted device according to claim 19, wherein, The second surface is approximately plane.