Active disparity sensing for head mounted displays

Abstract

A near-eye optical assembly includes a display waveguide (106A, 106B) and an optical structure. The display waveguide is configured to receive display light and to direct the display light toward an eye (102A, 102B) of a user. The optical structure includes an input coupler (304), an optical path, and an output coupler (312). The input coupler is arranged to receive a portion of the display light propagating through the waveguide. The light path directs the portion of the display light from the input coupler to an output coupler configured to provide the received portion of the display light to a parallax sensing circuit (108).

Description

Technical Field

[0001] Aspects of the present disclosure generally relate to head mounted displays (HMDs), and in particular but not exclusively to HMDs including parallax sensing. Background Art

[0002] A head mounted display (HMD) is a display device typically worn on a user's head. HMDs can be used for various applications (such as gaming, aviation, engineering, medical, entertainment, etc.) to provide artificial reality content to the user. Artificial reality is a form of reality that has been adjusted in some way before being presented to the user, and artificial reality can include, for example, virtual reality (VR), augmented reality (AR), mixed reality (MR), hybrid reality, or some combination and / or derivative thereof. Some HMDs can be configured to allow the user to view with both eyes. For example, the user can view a 3D image or a stereoscopic image by displaying artificial reality content through both the left eye eyepiece and the right eye eyepiece of the HMD. Summary of the Invention

[0003] According to a first aspect of the present disclosure, a near-eye optical component is provided, the near-eye optical component including: a display waveguide configured to receive display light and guide the display light towards the user's eye; and an optical structure. The optical structure includes: an input coupler configured to receive a portion of the display light propagating through the display waveguide; an output coupler configured to receive the portion of the display light from the input coupler and provide the portion of the display light to a parallax sensing circuit; and an optical path disposed between the input coupler and the output coupler to guide the portion of the display light from the input coupler to the output coupler.

[0004] In some embodiments, the optical path includes a parallax waveguide disposed on a surface of the display waveguide.

[0005] In some embodiments, the optical structure further includes one or more reflectors for guiding the portion of the display light along a surface of the display waveguide.

[0006] In some embodiments, the near-eye optical component further includes an input grating for coupling the display light into the display waveguide, wherein the input grating is disposed on a side of the display waveguide opposite to the input coupler of the optical structure.

[0007] In some embodiments, the input grating and the input coupler are disposed in a lateral region of the near-eye optical component, and wherein the optical path is configured to direct the portion of the display light from the lateral region to a medial region of the near-eye optical component.

[0008] In some embodiments, the input coupler includes at least one imaging optical device.

[0009] In some embodiments, the input coupler includes an angular-sensitive input grating.

[0010] In some embodiments, the optical structure includes a plurality of photonic integrated circuits (PICs), the plurality of PICs being included in the near-eye optical component and being coupled to provide respective portions of the display light to the disparity sensing circuit.

[0011] In some embodiments, the optical path includes a disparity waveguide embedded within the display waveguide.

[0012] According to a second aspect of the present disclosure, there is provided a head-mounted display (HMD) including: a projector configured to generate display light; and a near-eye optical component. The near-eye optical component includes: a display waveguide configured to receive the display light and direct the display light towards a user's eye; and a photonic integrated circuit (PIC) coupled to the display waveguide, wherein the PIC includes: an input coupler disposed on a surface of the display waveguide to receive a portion of the display light propagating through the display waveguide; an optical path configured to direct the portion of the display light along the surface of the display waveguide; and an output coupler configured to receive the portion of the display light from the optical path and being coupled to provide the portion of the display light to the disparity sensing circuit.

[0013] In some embodiments, the optical path includes a disparity waveguide disposed on the surface of the display waveguide.

[0014] In some embodiments, the projector includes: a first region of pixels configured to generate display light within the field of view of the near-eye optical component; and a second region of one or more pixels configured to generate display light outside the field of view, wherein the portion of the display light received by the input coupler includes display light generated by the second region of one or more pixels.

[0015] According to a third aspect of the present disclosure, there is provided a head-mounted display (HMD) including: a frame; a parallax sensing circuit disposed in a bridging region of the frame; and a left-eye optical component and a right-eye optical component fixed within the frame, wherein at least one of the left-eye optical component or the right-eye optical component includes: a display waveguide configured to receive display light from a corresponding projector and guide the display light toward a user's eye; and an optical structure. The optical structure includes: an input coupler configured to receive a portion of the display light propagating through the display waveguide; an output coupler configured to receive the portion of the display light from the input coupler and provide the portion of the display light to the parallax sensing circuit; and an optical path disposed between the input coupler and the output coupler to guide the portion of the display light from the input coupler to the output coupler, wherein the parallax sensing circuit is configured to detect a parallax between the left-eye optical component and the right-eye optical component based on the portion of the display light.

[0016] In some embodiments, the optical path includes a parallax waveguide disposed on a surface of the display waveguide.

[0017] In some embodiments, the optical structure further includes one or more reflectors for guiding the portion of the display light along a surface of the display waveguide.

[0018] In some embodiments, the HMD further includes an input grating for coupling the display light into the display waveguide, wherein the input grating is disposed on a side of the display waveguide opposite to the input coupler of the optical structure.

[0019] In some embodiments, the HMD further includes at least one temple arm coupled to the frame, wherein the input grating and the input coupler are disposed in a first region near the at least one temple arm, and wherein the optical path is configured to guide the portion of the display light from the first region to a second region near the bridging region of the frame.

[0020] In some embodiments, the input coupler includes at least one imaging optical device.

[0021] In some embodiments, the input coupler includes an angle-sensitive input grating.

[0022] In some embodiments, the optical structure includes one or more photonic integrated circuits (PICs), the one or more PICs being included in at least one of the left-eye optical component or the right-eye optical component, wherein each PIC of the one or more PICs is coupled to provide a corresponding portion of the display light to the disparity sensing circuit. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Non-limiting and non-exhaustive embodiments of the present invention are described with reference to the following drawings, wherein, unless otherwise specified, like reference numerals refer to like components throughout the views.

[0024] Figure 1 An example head-mounted display (HMD) is shown.

[0025] Figure 2 An example HMD according to aspects of the present disclosure is shown.

[0026] Figure 3A An example near-eye optical component according to aspects of the present disclosure is shown.

[0027] Figure 3B Another example near-eye optical component according to aspects of the present disclosure is shown, the near-eye optical component including a disparity waveguide embedded within a display waveguide.

[0028] Figure 3C Irradiation of multiple photonic integrated circuits (PICs) according to aspects of the present disclosure is shown.

[0029] Figure 3D Irradiation of a single PIC of multiple PICs according to aspects of the present disclosure is shown.

[0030] Figure 4 Another example near-eye optical component according to aspects of the present disclosure is shown.

[0031] Figure 5 A front view of an example near-eye optical component according to aspects of the present disclosure is shown.

[0032] Figure 6A 、 Figure 6B and Figure 6C A projector used in conjunction with a near-eye optical component according to aspects of the present disclosure is shown.

[0033] Figures 7A to 7D A process of forming a near-eye optical component according to aspects of the present disclosure is shown.

[0034] Figure 8 An example computing device for active parallax sensing for an HMD in accordance with aspects of the present disclosure is shown. DETAILED DESCRIPTION

[0035] Embodiments of near-eye optical components and head-mounted displays (HMDs) with active parallax sensing are described herein. In the following description, numerous specific details are set forth to provide a thorough understanding of the embodiments. However, one of ordinary skill in the art will recognize that the techniques described herein may be practiced without one or more of these specific details, or with other methods, components, materials, etc. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring certain aspects.

[0036] The phrase "in one embodiment" or "in an embodiment" as used throughout this specification means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrase "in one embodiment" or "in an embodiment" throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0037] In some implementations of the present disclosure, the term "near-eye" may be defined to include an element that is configured to be placed within 50 mm of a user's eye when using a near-eye device. Thus, a "near-eye optical element", "near-eye optical component", or "near-eye system" will include one or more elements that are configured to be placed within 50 mm of a user's eye.

[0038] As discussed above, some HMDs may be configured to allow binocular viewing by presenting content to both a user's left and right eyes via corresponding left-eye and right-eye optical components. However, during use, the HMD may be subject to various thermal or mechanical stresses that cause one or more components of the HMD (such as the frame, lenses, optical components, etc.) to deform and / or become misaligned. Such deformations may result in a parallax of the content displayed between the two eyes. In some cases, parallax refers to spatial (or binocular) parallax, where the position of the content presented to the user is different from the position expected by the HMD. In other contexts, parallax may refer to changes in the intensity, noise, and / or distortion of the content. Such parallax may have a negative impact on the viewing of images rendered by the HMD, particularly 3D or stereoscopic images, which may inhibit the user's fusion of the left and right images and may result in a reduced sense of immersion or presence experienced.

[0039] Some HMDs may be configured to detect the aforementioned parallax. For example,Figure 1 FIG. 100 shows an example head-mounted display (HMD) 100 that includes a disparity detector 108. The HMD 100 is also shown as including a first digital projector 104A, a second digital projector 104B, a first waveguide 106A, and a second waveguide 106B. As Figure 1 shown, the digital projector 104A is configured to generate display light 110A that is directed through the first waveguide 106A to the left eye 102A. Similarly, the digital projector 104B is configured to generate display light 110B that is directed through the second waveguide 106B to the right eye 102B. The disparity detector 108 can include optics, a camera, and processing circuitry that is configured to detect binocular disparity of the HMD 100 based on a remainder 114A of the display light 110A and a remainder 114B of the display light 110B. That is, the remainder 114A includes the remaining display light that is directed through the waveguide 106A after a majority of the display light 110A has been extracted toward the eye 102A. Similarly, the remainder 114B includes the remaining display light 110B after the display light 110B has been extracted toward the eye 102B. However, the waveguides 106A and 106B can be lossy because they are designed to increase the efficiency of light entering the eyes. Thus, the photons included in the remainders 114A and 114B of the display light can be significantly fewer than the photons directly emitted from their respective digital projectors 104A and 104B. This low optical efficiency can inhibit or prevent the disparity detector 108 from accurately sensing any disparity.

[0040] In addition, the content presented to the user via the display light 110A / 110B can be sparse and inconsistent. That is, the virtual graphics presented to the user can be small and only appear periodically. Thus, the detection of any disparity using the remainders 114A and 114B of the display light can be further hindered by these factors. Even further, the waveguides 106A and 106B can be vulnerable to ambient light (e.g., ambient light 116). That is, the ambient light 116 can couple into the waveguide 106A, thereby further reducing the contrast for disparity sensing by the disparity detector 108.

[0041] Accordingly, aspects of the present disclosure include near-eye optical components and an HMD that includes one or more dedicated optical structures, such as a photonic integrated circuit (PIC), for sensing binocular disparity. As will be described below, the optical structure can be coupled to the display waveguide to provide an optical path for a portion of the display light that is separate and distinct from the optical path for displaying content to the user. Thus, compared to the example HMD 100 of Figure 1 , the efficiency of the light received by the disparity detection circuitry can be significantly increased. These and other aspects will be described in more detail below.

[0042] Figure 2 FIG. 2 illustrates an example HMD 200 in accordance with aspects of the present disclosure. Example HMDs 200 shown include a frame 202, temple arms 204A and 204B, and near-eye optical components 206A and 206B. The frame 202 is shown as including a parallax sensing circuit 203 disposed in a bridge region 205 of the frame 202. Figure 2 An exploded view of the near-eye optical component 206A is also shown. The near-eye optical component 206A is shown as including a display layer 210 and optional optical elements 212.

[0043] As Figure 2 shown, the frame 202 is coupled to the temple arms 204A and 204B for securing the HMD 200 to a user's head. Example HMD 200 may also include support hardware incorporated into the frame 202 and / or the temple arms 204A and 204B. The hardware of the HMD 200 may include any one of the following: processing logic, wired and / or wireless data interfaces for sending and receiving data, a graphics processor, and one or more memories for storing data and computer-executable instructions. In one example, the HMD 200 may be configured to receive a wired power supply and / or may be configured to be powered by one or more batteries. Additionally, the HMD 200 may be configured to receive wired data and / or wireless data including video data.

[0044] Figure 2 Shown are the near-eye optical components 206A and 206B configured to be mounted to the frame 202. The frame 202 may accommodate the near-eye optical components 206A and 206B by surrounding at least a portion of the perimeter of the near-eye optical components 206A and 206B. The near-eye optical component 206A is configured to receive visible scene light 222 at a rear side 211 of the near-eye optical component 206A and direct the visible scene light 222 toward an eye side 209. In some examples, the near-eye optical component 206A may appear transparent to a user to facilitate augmented reality or mixed reality such that the user may view visible scene light 222 from the environment while also receiving display light 224 directed to their eyes through the display layer 210. In other examples, some or all of the near-eye optical components 206A and 206B may be incorporated into a virtual reality headset, where the transparent nature of the near-eye optical components 206A and 206B allows a user to view an electronic display (e.g., a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a micro LED display, etc.) incorporated into the virtual reality headset.

[0045] As Figure 2 shown, the display layer 210 is disposed between the eye-facing side 209 and the rear side 211 of the near-eye optical assembly 206A. In some examples, the display layer 210 may include a display waveguide 216 configured to direct display light 224 to present one or more virtual graphics to the eyes of the user of the HMD 200. In some aspects, the display waveguide 216 is configured to direct display light 224 generated by an electronic display or a projector to the eyes of the user. In some embodiments, at least a portion of the electronic display is included in the frame 202 of the head-mounted device 200. The electronic display may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a micro-LED display, a micro-projector, or a liquid crystal on silicon (LCOS) display for generating the display light 224.

[0046] Figure 2 It is also shown that the display layer 210 includes an optical structure 218. In some aspects, the optical structure 218 includes one or more photonic integrated circuits (PICs). In some examples, the optical structure 218 provides an optical path (e.g., a waveguide) that is separate and distinct from the optical path provided by the display waveguide 216. For example, the optical structure 218 may be coupled to receive a portion of the display light 224 propagating through the display waveguide 216 and direct that portion of the display light along the surface of the display waveguide 216 to the disparity sensing circuit 203. In some examples, the optical structure 218 directs that portion of the display light along the surface of the display waveguide 216 from a region near the temple arm 204A to a region near the bridge region 205 (e.g., toward the location of the disparity sensing circuit 203).

[0047] In some examples, the near-eye optical assembly 206B has a similar structure, including a display waveguide and a separate optical structure for directing a portion of its display light to the disparity sensing circuit 203. The disparity sensing circuit 203 can then detect the presence and amount of binocular disparity based on the respective portions of the display light received from both the near-eye optical assemblies 206A and 206B.

[0048] Figure 2Also shown is that the near-eye optical component 206A includes an optional optical element 212. The optical element 212 may include one or more elements, such as a filter, a lens, a mirror, a prism, an attenuator, a diffractive element, a waveguide, a polarizer, etc. In some examples, the display layer 210 and / or the optional optical element 212 may have a curvature for focusing light (e.g., scene light 222) onto the user's eye. Thus, in some examples, the display layer 210 and / or the optional optical element 212 may be referred to as a lens. In some aspects, the display layer 210 and / or the optional optical element 212 have a thickness and / or curvature corresponding to the user's specifications. In other words, the display layer 210 and / or the optional optical element 212 may be prescription lenses.

[0049] Figure 3A An example near-eye optical component 300 in accordance with aspects of the present disclosure is shown. The illustrated example of the near-eye optical component 300 is shown to include a projector 302, a first input coupler 304, a PIC 305A, and a display waveguide 310. The PIC 305A is shown to include a second input coupler 308, a parallax waveguide 310, and an output coupler 312. Figure 3A The parallax sensing circuit 203 is also shown. The PIC 305A is Figure 2 a possible example of the optical structure 218. The near-eye optical component 300 is Figure 2 a possible implementation of the near-eye optical component 206A and / or the near-eye optical component 206B.

[0050] As Figure 3A shown, the projector 302 is configured to generate display light 224. In some examples, the display light 224 is visible light including color components (e.g., red, green, and blue). The first input coupler 304 is disposed between the projector 302 and the display waveguide 306 to couple the display light 224 into the display waveguide 306. In some examples, the first input coupler 304 includes one or more input gratings. As shown, the display waveguide 306 receives the display light 224 and guides the display light 224 toward the user's eye. For example, the display light 224 may propagate through the display waveguide 306 by way of total internal reflection (TIR), where the display waveguide 306 includes one or more extraction features for extracting the display light 224 from the display waveguide 306 to direct it toward the eye side 209.

[0051] The second input coupler 308 is shown to be disposed on the surface 307 of the display waveguide 306. In some aspects, the second input coupler 308 is disposed on the side of the display waveguide opposite the first input coupler 304. For example, Figure 3AIt is shown that the second input coupler 308 is disposed on the surface 307 of the rear side 211 of the display waveguide 306, while the first input coupler 304 is disposed on the eye-facing side 209 of the display waveguide 306. Accordingly, the second input coupler 308 is arranged to receive at least a portion of the display light 224 propagating through the display waveguide 306 (e.g., propagating from the eye-facing side 209 of the display waveguide 306 to the rear side 211 of the display waveguide 306).

[0052] In some examples, the second input coupler 308 includes an input grating that is configured to couple at least a portion 309 of the display light 224 into the optical path provided by the PIC 305A. In some aspects, the input grating is an angle-sensitive input grating. In Figure 3A an example, the optical path of the PIC 305A is provided by the parallax waveguide 310. The parallax waveguide 310 is configured to guide a portion 309 of the display light 224 along the surface 307 of the display waveguide 306. In some examples, the first input coupler 304 and the second input coupler 308 are disposed in the lateral region 311 of the near-eye optical assembly 300 (e.g., near the temple arm of the frame 202). The optical path of the PIC 305A is configured to guide a portion 309 of the display light 224 from the lateral region 311 to the medial region 313 (e.g., near the bridge region 205 of the frame 202). In some examples, the parallax waveguide 310 is disposed on the surface 307 of the display waveguide 306 and is configured to guide a portion 309 of the display light 224 to the output coupler 312 by way of TIR. In other examples, the parallax waveguide 310 is disposed below the surface 307 of the display waveguide 306. For example, Figure 3B it is shown a PIC 305B that includes a parallax waveguide 310 embedded within the display waveguide 306 below the surface 307.

[0053] Now returning to Figure 3A , the output coupler 312 is configured to receive a portion 309 of the display light from the optical path (e.g., the parallax waveguide 310) and provide the portion 309 to the parallax sensing circuit 203. The output coupler 312 may include one or more output gratings that are configured to couple the portion 309 out of the parallax waveguide 310 to the parallax sensing circuit 203. The parallax sensing circuit 203 may include being configured to detect the near-eye optical assembly 300 and another near-eye optical assembly ( Figure 3AAn optical device, a camera, and a processing circuit for binocular disparity between (not explicitly shown in the figure). The disparity sensing circuit 203 can detect binocular disparity based on the remainder 309 received from the remaining part of the near-eye optical component 300 and based on the remainder 315 of the display light received from another near-eye optical component. In some aspects, the disparity sensing circuit 203 is configured to detect the tip tilt between the projector 302 and the display waveguide 306. In another aspect, the disparity sensing circuit 203 can detect the tip tilt of the projector 302 and / or the display waveguide from the left-eye optical component to the right-eye optical component. In yet another aspect, the disparity sensing circuit 203 can detect the deformation of the display waveguide 306. Even further, the disparity sensing circuit 203 can be configured to detect the inherent deformation of the projector 302 itself.

[0054] Although Figure 3A The near-eye optical component 300 is shown as including a single PIC 305A, but the near-eye optical component 300 can include any number of PICs, including one or more PICs. For example, the PIC 305A can be one of the multiple PICs included in the near-eye optical component 300, and each PIC is configured to provide a corresponding portion of the display light 224 to the disparity sensing circuit 203. In some aspects, the second input coupler 308 can include at least one imaging optical device to simultaneously illuminate several PICs with a portion 309 of the display light. For example, Figure 3C Shows the illumination of multiple PICs 320A to 320I using the imaging optical device 318 according to aspects of the present disclosure.

[0055] Figure 3D Shows multiple PICs 320A to 320I of the near-eye optical component according to aspects of the present disclosure, but arranged such that only a single PIC among the multiple PICs is illuminated at a time. For example, one or more pixels of the projector can be used to illuminate the PIC, where the single illuminated PIC corresponds to the tip of the projector or to the tilt of the projector relative to the display waveguide. That is, the tip tilt and / or disparity of one or more components of the near-eye optical component can shift the position of the portion 309 of the display light on the PICs 320A to 320I.

[0056] In some examples, the near-eye optical component according to aspects of the present disclosure can include multiple PICs for each color channel. In this example, the disparity sensing circuit 203 can be configured to disambiguate the fluctuations in the display intensity based on the corresponding portions of the display received from the multiple PICs.

[0057] In another example, the near-eye optical component includes multiple PICs arranged horizontally and vertically, where the disparity sensing circuit 203 is configured to disambiguate the tip and / or tilt rotation.

[0058] In yet another example, the near-eye optical component may include PICs that include a combination of an angle-sensitive input grating and an output grating and a non-angle-sensitive input grating and an output grating on the projector side (i.e., the eye side 209) and the PIC side (i.e., the rear side 211), wherein the disparity sensing circuit 203 is configured to disambiguate between the display waveguide disparity and the projector disparity.

[0059] Figure 4 Another example near-eye optical component 400 in accordance with aspects of the present disclosure is shown. The illustrated example of the near-eye optical component 400 is shown to include a projector 302, an optical structure 405, and a display waveguide 306. The optical structure 405 is shown to include a second input coupler 402 and an output coupler 404. The optical structure 405 is Figure 2 a possible example of the optical structure 218. The near-eye optical component 400 is Figure 2 a possible implementation of the near-eye optical component 206A and / or the near-eye optical component 206B.

[0060] The second input coupler 402 is shown disposed on the surface 307 of the display waveguide 306. In some examples, the second input coupler 402 includes a reflector that is configured to direct at least a portion 309 of the display light 224 into the optical path provided by the optical structure 405. In Figure 4 the example, the second input coupler 402 is configured to direct the portion 309 along the optical path on the surface 307 of the display waveguide 306 towards the output coupler 404, which may also include a reflector. The output coupler 404 is configured to receive the portion 309 of the display light from the optical path and provide the portion 309 to the disparity sensing circuit 203.

[0061] Figure 5 A front view of an example near-eye optical component 500 in accordance with aspects of the present disclosure is shown. The illustrated example of the near-eye optical component 500 is shown to include input couplers 502A, 502B, and 502C, disparity waveguides 504A, 504B, and 504C, and output couplers 506A, 506B, and 506C. The near-eye optical component 500 is Figure 2A possible implementation of the near-eye optical component 206A and / or the near-eye optical component 206B. The parallax waveguide 504A may include one or more waveguides for guiding a portion of the display light from the corresponding one or more input couplers 502A to the corresponding one or more output couplers 506A. Similarly, the parallax waveguide 504B may include one or more waveguides for guiding a portion of the display light from the corresponding one or more input couplers 502B to the corresponding one or more output couplers 506B. The parallax waveguide 504C may include one or more waveguides for guiding a portion of the display light from the corresponding one or more input couplers 502C to the corresponding one or more output couplers 506C.

[0062] In some examples, the parallax waveguides 504A, 504B, and 504C are disposed in a peripheral region 508 of the near-eye optical component 500 adjacent to the edge 510. As mentioned above, the near-eye optical component 500 may be configured to be accommodated within a frame (e.g., Figure 2 the frame 202). Thus, in some examples, the frame may be configured to shield the peripheral region 508 from ambient light to prevent or reduce interference with the parallax waveguide.

[0063] Figure 6A , Figure 6B and Figure 6C shows a projector 602 for use with a near-eye optical component according to aspects of the present disclosure. The projector 602 is Figure 3A a possible implementation of the projector 302. In some aspects, the projector 602 includes a plurality of pixels configured to generate display light for presentation to a user. As Figure 6A shown, the projector 602 may include a first region 604 and a second region 605. The first region 604 corresponds to pixels within the field of view 603 of the HMD, while the second region 605 corresponds to pixels outside the field of view 603. In some embodiments, aspects of the present disclosure may be configured to utilize one or more pixels located in the second region 605 (i.e., outside the field of view) for parallax sensing. In some examples, one or more pixels located in the second region 605 may be configured to be always on (i.e., always emit display light) because the one or more pixels are outside the field of view. Providing always-on pixels may allow for immediate detection of parallax. Even, due to the increased efficiency of the dedicated optical path using PIC, these pixels may be at a level that is imperceptible to the user.

[0064] For example, Figure 6AIncludes using a single pixel 606A in the second region 605 for parallax detection. The display light generated by the single pixel 606A can be coupled into any of the aforementioned optical structures (e.g., PIC) for detection and processing by the parallax sensing circuit 203. Figure 6B Illustrates the use of two pixels 606A and 606B, both located in the second region 605, for parallax detection. In some examples, pixels 606A and 606B are collinear (i.e., on the same y-axis). Figure 6C Illustrates the use of four pixels 604A, 604B, 604C, and 604D, all located in the second region 605, for parallax detection. As described above, pixels 604A and 604B can be collinear on the same y-axis. Similarly, pixels 604C and 604D can be collinear, but on the orthogonal x-axis.

[0065] Figures 7A to 7D Illustrates a process for forming a near-eye optical component according to aspects of the present disclosure. In Figure 7A , a SiN layer 704 is deposited onto a substrate layer 702. In some aspects, the substrate layer 702 is a waveguide (e.g., Figure 3A 's display waveguide 306). In some examples, the SiN layer 704 has a refractive index of approximately 2.02. Next, Figure 7B Illustrates the etching of the SiN layer 704 to form parallax waveguides 706A, 706B, and 706C. In Figure 7C , a cladding layer 708 is deposited on the parallax waveguides 706A, 706B, and 706C. In some examples, the cladding layer 708 includes SiO2 and has a refractive index of approximately 1.47. Figure 7D Illustrates the etching of input gratings and / or output gratings 710A to 710C for coupling in and out for each parallax waveguide.

[0066] Figure 8 Illustrates an example computing device for active parallax sensing for an HMD according to aspects of the present disclosure. The illustrated example of the computing device 802 is shown to include a communication interface 804, one or more processors 806, hardware 808, and a memory 810. In one example, Figure 8 One or more of the components shown in Figure 2 can be incorporated into the frame 202 and / or temple arms 204A / 204B of the head-mounted device 200. In other examples, Figure 8 One or more of the components shown in

[0067] The communication interface 804 may include wireless communication components and / or wired communication components that enable the computing device 802 to send data to and receive data from other networked devices. The hardware 808 may include additional hardware interfaces, data communication, or data storage hardware. For example, the hardware interfaces may include data output devices (e.g., electronic displays, audio speakers) and one or more data input devices.

[0068] The memory 810 may be implemented using computer-readable media (e.g., computer storage media). In some aspects, computer-readable media may include volatile and / or non-volatile media, removable and / or non-removable media implemented in any method or technology for storing information (e.g., computer-readable instructions, data structures, program modules, or other data). Computer-readable media includes, but is not limited to, RAM, ROM, EEPROM, flash memory, or other storage technologies, CD-ROM, digital versatile disk (DVD), high-definition multimedia / data storage disk, or other optical storage devices, magnetic cassette, magnetic tape, disk storage devices, or other magnetic storage devices, or any other non-transitory media that can be used to store information for access by a computing device.

[0069] The processor 806 and memory 810 of the computing device 802 may implement a display module 812 and a parallax sensing module 814. The display module 812 and the parallax sensing module 814 may include routines, program instructions, objects, and / or data structures that perform specific tasks or implement specific abstract data types. The memory 810 may also include data storage devices (not shown) used by the display module 812 and / or the parallax sensing module 814.

[0070] The display module 812 may be configured to control the generation and rendering of one or more virtual graphics (e.g., text, pictures, video, or other visual information) on one or more optical components of the HMD. For example, the display module 812 may control the projector 302 to generate Figure 3A the display light 224. The parallax sensing module 814 may be configured to detect the parallax between the optical components of the HMD in response to one or more measurement results performed by a parallax sensing circuit (e.g., Figure 2 the parallax sensing circuit 203). In some examples, the parallax sensing module 814 performs parallax sensing as part of a startup routine of the HMD (e.g., when powered on). In other examples, the parallax sensing module 814 may perform parallax sensing in response to an input received from a user, such as through a software calibration feature. In yet another example, the parallax sensing module 814 may perform parallax sensing at periodic intervals when the HMD is in use.

[0071] In some examples, the parallax sensing module 814 is configured to generate a parallax signal 815 in response to detecting parallax in the HMD. The parallax signal 815 can indicate the presence and amount of the parallax. As Figure 8 shown, the display module 812 can receive the parallax signal 815, where the display module 812 is further configured to adjust the rendering of one or more virtual graphics in response to the parallax signal. For example, the display module 812 can be configured to determine an offset based on the parallax signal 815 to move or adjust the rendering position of the virtual graphics on one or both of the electronic displays. In some examples, adjusting the rendering position of the virtual graphics adjusts the position at which the virtual graphics are presented on the electronic display relative to the optical elements of the HMD. Even further, adjusting the rendering of the virtual graphics can include aligning the rendering position of the virtual graphics displayed on the left-eye optical component with the virtual graphics displayed on the right-eye optical component.

[0072] Embodiments of the present invention can include or be implemented in conjunction with an artificial reality system. Artificial reality is a form of reality that has been adjusted in some way before being presented to a user, and artificial reality can include, for example, virtual reality (VR), augmented reality (AR), mixed reality (MR), hybrid reality, or some combination and / or derivative thereof. Artificial reality content can include fully generated content or content that is generated in combination with captured (e.g., real-world) content. Artificial reality content can include video, audio, haptic feedback, or some combination thereof, and any of video, audio, haptic feedback, or some combination thereof can be presented in a single channel or multiple channels (e.g., stereoscopic video that produces a three-dimensional effect for a viewer). Additionally, in some embodiments, artificial reality can also be associated with applications, products, accessories, services, or some combination thereof that are used, for example, to create content in artificial reality and / or otherwise for use in artificial reality (e.g., to perform activities in artificial reality). An artificial reality system that provides artificial reality content can be implemented on a variety of platforms, including a head-mounted display (HMD) connected to a host computer system, a stand-alone HMD, a mobile device or computing system, or any other hardware platform capable of providing artificial reality content to one or more viewers.

[0073] The foregoing description of the embodiments shown in the present invention (including the content described in the abstract) is not intended to be exhaustive or to limit the present invention to the precise forms disclosed. While specific embodiments and examples of the present invention have been described herein for purposes of illustration, various modifications within the scope of the present invention are possible as will be recognized by those skilled in the relevant art.

[0074] In view of the foregoing detailed description, the present invention may be modified in these ways. The terms used in the appended claims should not be construed as limiting the invention to the specific embodiments disclosed in the specification. Rather, the scope of the present invention will be determined entirely by the appended claims, which will be interpreted in accordance with established principles of claim interpretation.

Claims

1. A near-eye optical component, the near-eye optical component comprising: A display waveguide configured to receive display light and direct the display light towards a user's eye; and an optical structure comprising: an input coupler configured to receive a portion of the display light propagating through the display waveguide; an output coupler configured to receive the portion of the display light from the input coupler and provide the portion of the display light to a parallax sensing circuit; and an optical path disposed between the input coupler and the output coupler to direct the portion of the display light from the input coupler to the output coupler.

2. The near-eye optical component according to claim 1, wherein, The optical path includes a parallax waveguide disposed on a surface of the display waveguide.

3. The near-eye optical component according to claim 1 or 2, wherein, The optical structure further includes one or more reflectors for guiding the portion of the display light along the surface of the display waveguide.

4. The near-eye optical component according to any one of the preceding claims, the near-eye optical component further comprising an input grating for coupling the display light into the display waveguide, wherein, The input grating is disposed on a side of the display waveguide opposite to the input coupler of the optical structure.

5. The near-eye optical component according to claim 4, wherein, The input grating and the input coupler are disposed in a lateral region of the near-eye optical component, and wherein the optical path is configured to direct the portion of the display light from the lateral region to a near-middle region of the near-eye optical component.

6. The near-eye optical component according to any one of the preceding claims, wherein, The input coupler includes at least one imaging optical device.

7. The near-eye optical component according to any one of the preceding claims, wherein, The input coupler includes an angle-sensitive input grating.

8. The near-eye optical component according to any one of the preceding claims, wherein, The optical structure includes a plurality of photon integrated circuits (PICs) included in the near-eye optical component, the plurality of PICs being coupled to provide respective portions of the display light to the parallax sensing circuit; and / or preferably, wherein the optical path includes a parallax waveguide embedded within the display waveguide.

9. A head-mounted display (HMD), the HMD comprising: A projector configured to generate display light; and a near-eye optical component comprising: a display waveguide configured to receive the display light and direct the display light towards a user's eye; and a photon integrated circuit (PIC) coupled to the display waveguide, wherein the PIC includes: an input coupler disposed on a surface of the display waveguide to receive a portion of the display light propagating through the display waveguide; an optical path configured to guide the portion of the display light along the surface of the display waveguide; and an output coupler configured to receive the portion of the display light from the optical path and coupled to provide the portion of the display light to a parallax sensing circuit.

10. The HMD according to claim 9, wherein, The optical path includes a parallax waveguide disposed on the surface of the display waveguide; and / or preferably, wherein the projector includes: a first pixel region configured to generate display light within a field of view of the near-eye optical component; and A second one or more pixel regions configured to generate display light outside the field of view, wherein the portion of the display light received by the input coupler includes display light generated by the second one or more pixel regions.

11. A head-mounted display (HMD), the HMD comprising: A frame; A parallax sensing circuit disposed in a bridging region of the frame; And A left-eye optical component and a right-eye optical component fixed within the frame, wherein at least one of the left-eye optical component or the right-eye optical component includes: A display waveguide configured to receive display light from a corresponding projector and direct the display light towards a user's eye; and An optical structure including: An input coupler configured to receive a portion of the display light propagating through the display waveguide; An output coupler configured to receive the portion of the display light from the input coupler and provide the portion of the display light to the parallax sensing circuit; and An optical path disposed between the input coupler and the output coupler to direct the portion of the display light from the input coupler to the output coupler, wherein the parallax sensing circuit is configured to detect a parallax between the left-eye optical component and the right-eye optical component based on the portion of the display light.

12. The HMD according to claim 11, wherein, The optical path includes a parallax waveguide disposed on a surface of the display waveguide; and / or preferably, wherein the optical structure further includes one or more reflectors for guiding the portion of the display light along the surface of the display waveguide.

13. The HMD according to claim 11 or 12, the HMD further comprising an input grating for coupling the display light into the display waveguide, wherein, The input grating is disposed on a side of the display waveguide opposite to the input coupler of the optical structure; and / or preferably, the HMD further includes at least one temple arm coupled to the frame, wherein the input grating and the input coupler are disposed in a first region near the at least one temple arm, and wherein the optical path is configured to direct the portion of the display light from the first region to a second region near the bridging region of the frame.

14. The HMD according to any one of claims 11 to 13, wherein, The input coupler includes at least one imaging optical device.

15. The HMD according to any one of claims 11 to 14, wherein, The input coupler includes an angle-sensitive input grating; and / or preferably, wherein the optical structure includes one or more photonic integrated circuits (PICs), the one or more PICs being included in at least one of the left-eye optical component or the right-eye optical component, wherein each of the one or more PICs is coupled to provide a corresponding portion of the display light to the parallax sensing circuit.

Images