Variable pixel density display system with mechanically-actuated image projector

Abstract

To provide a suitable variable pixel density display system.SOLUTION: A head-mounted virtual and augmented reality display system includes a light projector with one or more light-emissive type micro-displays having a first resolution and a pixel pitch. The projector outputs light that forms a frame of a virtual content having at least a portion associated with a second resolution exceeding the first resolution. A projector outputs light that forms a first sub-frame of a rendered frame at the first resolution, and a portion of the projector shifts using an actuator so that a physical position of the light outputted for an individual pixel occupies a gap between the old places of the light outputted for an individual pixel. Then, the projector outputs light that forms a second sub-frame of a rendered frame. The first and second sub-frames are outputted within the flicker fusion threshold.SELECTED DRAWING: Figure 36A

Description

[Technical Field]

[0001] (Priority Claim) This application claims priority to U.S. Provisional Application No. 62 / 911,018, filed October 4, 2019, and entitled "AUGMENTED AND VIRTUAL REALITY DISPLAY SYSTEMS WITH SHARED DISPLAY FOR LEFT AND RIGHT EYES," U.S. Provisional Application No. 62 / 800,363, filed February 1, 2019, and entitled "VIRTUAL AND AUGMENTED REALITY DISPLAY SYSTEMS WITH EMISSIVE MICRO-DISPLAYS," and U.S. Provisional Application No. 62 / 786,199, filed December 28, 2018, and entitled "LOW MOTION-TO-PHOTON LATENCY ARCHITECTURE FOR AUGMENTED AND VIRTUAL REALITY DISPLAY SYSTEMS," all of which are incorporated herein by reference in their entireties. (Incorporated by reference)

[0002] This application is incorporated by reference into the following: U.S. Patent Application No. 14 / 555,585, filed November 27, 2014, published on July 23, 2015 as U.S. Patent Publication No. 2015 / 0205126; U.S. Patent Application No. 14 / 69, filed April 18, 2015, published on October 22, 2015 as U.S. Patent Publication No. 2015 / 0302652; No. 0,401, filed March 14, 2014, issued August 16, 2016, now U.S. Patent No. 9,417,452; U.S. Patent Application No. 14 / 212,961, filed July 14, 2014, published October 29, 2015 as U.S. Patent Publication No. 2015 / 0309263; U.S. Patent Application No. 14 / 331,218, published March 1, 2018 Also, U.S. Patent Application Publication No. 2018 / 0061121, filed December 14, 2018; U.S. Patent Application Publication No. 16 / 221065, filed September 27, 2018; U.S. Patent Application Publication No. 2018 / 0275410, filed December 28, 2018; U.S. Provisional Application No. 62 / 786,199, filed December 14, 2018; U.S. Patent Application No. 16 / 221 This application incorporates in its entirety each of U.S. Provisional Application No. 62 / 702,707, filed July 24, 2018, U.S. Provisional Application No. 15 / 481,255, filed April 6, 2017, and U.S. Patent Application No. 15 / 927,808, filed April 21, 2018, and published as U.S. Patent Application Publication No. 2018 / 0275410 on September 27, 2018. (Technical field)

[0003] The present disclosure relates to display systems, and more particularly to augmented and virtual reality display systems. [Background technology]

[0004] Modern computing and display technology has facilitated the development of systems for so-called “virtual reality” or “augmented reality” experiences, in which digitally reproduced images or portions thereof are presented to a user in a manner that appears or can be perceived as real. Virtual reality, or “VR,” scenarios typically involve the presentation of digital or virtual image information without transparency to other actual real-world visual input, while augmented reality, or “AR,” scenarios typically involve the presentation of digital or virtual image information as an augmentation to the user's visualization of the real world around them. Mixed reality, or “MR,” scenarios are a type of AR scenario that typically involve virtual objects integrated into and responsive to the natural world. For example, MR scenarios may include AR image content that appears blocked by or is perceived to otherwise interact with objects in the real world.

[0005] Referring to FIG. 1 , an AR scene 10 is depicted. A user of the AR technology sees a real-world park-like setting 20 featuring people, trees, a building in the background, and a concrete platform 30. The user also perceives that they are "seeing" "virtual content," such as a robotic figure 40 standing on the real-world platform 30 and a flying, cartoon-like avatar character 50 that appears to be an anthropomorphic bumblebee. These elements 50, 40 are "virtual" in that they do not exist in the real world. The human visual perception system is complex, making it difficult to produce AR technology that facilitates a comfortable, natural-feeling, and rich presentation of virtual image elements among other virtual or real-world image elements. Summary of the Invention [Means for solving the problem]

[0006] According to some embodiments, a head-mounted display system includes a support structure configured to mount on a user's head, an optical projection system supported by the support structure, an eyepiece, and one or more processors. The optical projection system includes a microdisplay including an array of light emitters associated with a first resolution, the array of light emitters configured to output light forming a frame of virtual content, projection optics, and one or more actuators. The eyepiece is supported by the support structure and configured to receive light from the optical projection system and direct the received light toward the user. The one or more processors are configured to receive a rendered frame of the virtual content, the rendered frame comprising at least a portion associated with a second resolution, the second resolution being higher than the first resolution. The one or more processors are further configured to cause an emissive microdisplay projector to output light forming a first sub-frame of the rendered frame, the first sub-frame and the rendered frame being substantially the same size. The one or more processors are further configured to shift, via the one or more actuators, one or more movable parts of the optical projection system to adjust a position associated with the light emitter light output from the optical projection system, causing the optical projection system to output light that forms a second sub-frame of the rendered frame.

[0007] According to some other embodiments, a method implemented by a head-mounted display system of one or more processors includes providing a rendered frame of virtual content, the rendered frame comprising at least a portion associated with a second resolution. An emissive microdisplay projector is caused to output light forming a first subframe of the rendered frame, the first subframe having a first resolution less than the second resolution, the emissive microdisplay projector comprising an array of light emitters associated with the first resolution and having a pixel pitch. The emissive microdisplay projector is shifted via one or more actuators to adjust a geometric position associated with the light output by the emissive microdisplay projector, the geometric position being adjusted by a distance less than the pixel pitch. The emissive microdisplay projector is caused to output light forming a second subframe of the rendered frame, the second subframe having the first resolution.

[0008] According to yet another embodiment, a system includes one or more processors and one or more computer storage media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations. The operations include generating rendered frames of virtual content for display as augmented reality content via an emissive microdisplay projector system of the system, the rendered frames being associated with a second resolution, the emissive microdisplay projector including one or more light emitter arrays configured to output light forming the virtual content associated with a first, lower resolution. The rendered frames of virtual content are divided into a plurality of subframes, each subframe including a subset of the pixels included in the rendered frame. The light is output sequentially via the emissive microdisplay projector system, the light forming the plurality of subframes, the emissive microdisplay projector system being shifted according to a movement pattern for each subframe via one or more actuators, the emissive microdisplay projector system being shifted along one or more axes in a plane parallel to a plane of an output pupil of the projector system.

[0009] According to some other embodiments, a method implemented by a head-mounted display system of one or more processors includes generating rendered frames of virtual content for display as the virtual content via an emissive microdisplay projector system of the head-mounted display system, the rendered frames being associated with a second resolution, the emissive microdisplay projector comprising an emitter configured to output light forming the virtual content associated with a first, lower resolution. The rendered frames of the virtual content are divided into a plurality of subframes, each subframe including a subset of the pixels included in the rendered frame. The light is output sequentially via the emissive microdisplay projector system, the light forming the plurality of subframes, the emissive microdisplay projector system being shifted according to a movement pattern along one or more axes via one or more actuators for each subframe, the emissive microdisplay projector system being shifted along one or more axes in a plane parallel to a plane of an output pupil of the projector system.

[0010] Some additional examples are provided below.

[0011] Example 1 A head-mounted display system includes: a support structure configured to mount on a user's head; a microdisplay supported by the support structure and comprising an array of light emitters associated with a first resolution, the array of light emitters configured to output light that forms frames of virtual content; an optical projection system including projection optics and one or more actuators; an eyepiece supported by the support structure and configured to receive light from the optical projection system and direct the received light to a user; and one or more processors configured to receive rendered frames of the virtual content and direct the rendered frames. the head-mounted display system comprising: one or more processors configured to: cause an emissive microdisplay projector to output light that forms a first sub-frame of a rendered frame, the first sub-frame and the rendered frame being substantially the same size; and one or more processors configured to: shift, via one or more actuators, one or more movable parts of the optical projection system to adjust a position associated with light emitter light output from the optical projection system; and cause the optical projection system to output light that forms a second sub-frame of the rendered frame.

[0012] Example 2 2. The head-mounted display of claim 1, wherein the portion associated with the second resolution is associated with a foveal region of the user's eye.

[0013] Example 3 3. The head-mounted display of example 2, wherein the one or more processors are configured to determine that the light forming part is within a threshold angular distance of the user's fovea.

[0014] Example 4 3. A head-mounted display as described in Example 2, wherein the one or more processors are configured to cause the light emitter to update the emitted light that forms the portion for the second sub-frame and to not cause the light emitter to update the emitted light that forms the portion of the rendered frame outside the portion for the first sub-frame.

[0015] Example 5 2. The head-mounted display system of example 1, wherein each emissive microdisplay array has an associated emitter size, the emitter size being less than the pixel pitch.

[0016] Example 6 6. The head-mounted display of example 5, wherein the total number of subframes of a rendered frame is determined based on a size associated with a pixel pitch and an emitter size.

[0017] Example 7 A head-mounted display as described in Example 6, wherein the one or more processors are configured to cause the light projection system to continuously output light that forms the total number of sub-frames.

[0018] Example 8 8. A head-mounted display as described in Example 7, wherein the one or more processors are configured to time-multiplex the rendered frames by causing one or more actuators to shift a portion of the optical projection system on a sub-frame basis.

[0019] Example 9 A head-mounted display as described in Example 8, wherein the one or more processors are configured to cause one or more actuators to shift portions of the optical projection system associated with the array of optical emitters such that geometric positions are tiled within the regions between individual emitters.

[0020] Example 10 2. The head-mounted display of claim 1, wherein the one or more processors are configured to cause the one or more actuators to shift a portion of the optical projection system according to a movement pattern, the movement pattern being a continuous movement pattern.

[0021] Example 11 2. The head-mounted display of example 1, wherein the first sub-frame and the second sub-frame each comprise pixels associated with a distinct portion of the frame being rendered.

[0022] Example 12 2. The head-mounted display of example 1, wherein the optical projection system comprises a plurality of arrays of optical emitters.

[0023] Example 13 13. The head-mounted display of example 12, further comprising an X-cube prism, wherein each of the arrays of light emitters faces a different side of the X-cube prism.

[0024] Example 14 13. A head-mounted display as described in Example 12, wherein each array of light emitters is configured to direct light into a dedicated associated projection optics.

[0025] Example 15 13. The head-mounted display of example 12, wherein the array of light emitters is mounted in a common back plane.

[0026] Example 16 2. The head-mounted display of example 1, wherein the one or more actuators are configured to deflect the projection optics.

[0027] Example 17 2. The head-mounted display of example 1, wherein the one or more actuators are piezoelectric motors.

[0028] Example 18 2. The head-mounted display of example 1, wherein the one or more actuators deflect the emissive microdisplay projector along two axes.

[0029] Example 19 2. The head-mounted display of example 1, wherein the light emitter comprises a light-emitting diode.

[0030] Example 20 2. The head-mounted display of example 1, wherein the array of light emitters is configured to emit light of a plurality of primary colors.

[0031] Example 21 21. A head-mounted display as described in Example 20, wherein each light emitter comprises a stack of constituent light generators, each constituent light generator emitting light of a different color.

[0032] Example 22 A head-mounted display as described in Example 1, wherein the eyepiece lens comprises a waveguide assembly comprising one or more waveguides, each waveguide comprising an internal coupling optical element configured to internally couple light from the microdisplay into the waveguide, and an external coupling optical element configured to externally couple the internally coupled light out of the waveguide.

[0033] Example 23 10. A method implemented by a head-mounted display system of one or more processors, the method comprising: providing a rendered frame of virtual content, the rendered frame comprising at least a portion associated with a second resolution; causing an emissive microdisplay projector to output light forming a first sub-frame of the rendered frame, the first sub-frame having a first resolution less than the second resolution, the emissive microdisplay projector comprising an array of light emitters associated with the first resolution and having a pixel pitch; shifting the emissive microdisplay projector via one or more actuators to adjust a geometric position associated with the light output by the emissive microdisplay projector, the geometric position being adjusted by a distance less than the pixel pitch; and causing the emissive microdisplay projector to output light forming a second sub-frame of the rendered frame, the second sub-frame having the first resolution.

[0034] Example 24 10. A system comprising: one or more processors; and one or more computer storage media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including: generating a rendered frame of virtual content for display as augmented reality content via an emissive microdisplay projector system of the system, wherein the rendered frame is associated with a second resolution and the emissive microdisplay projector comprises one or more light emitter arrays configured to output light forming the virtual content associated with a first, lower resolution; dividing the rendered frame of the virtual content into a plurality of sub-frames, each sub-frame comprising a subset of the pixels included in the rendered frame; and sequentially outputting light forming the plurality of sub-frames via the emissive microdisplay projector system, wherein the emissive microdisplay projector system is shifted according to a movement pattern for each sub-frame via one or more actuators, wherein the emissive microdisplay projector system is shifted along one or more axes in a plane parallel to a plane of an output pupil of the projector system.

[0035] Example 25 25. The system of Example 24, wherein the one or more processors are configured to cause one or more actuators to shift emissive microdisplay projectors associated with the emissive microdisplay array such that their geometric positions are tiled within the individual inter-emitter regions.

[0036] Example 26 26. The system of example 25, wherein the one or more processors are configured to cause the one or more actuators to displace the light emitter array along one or more axes.

[0037] Example 27 26. The system of Example 25, wherein the microdisplay projector system includes a projection optical system, and wherein the one or more processors are configured to cause one or more actuators to displace the projection optical system along one or more axes, and the projection optical system is configured to output light to a user of the system.

[0038] Example 28 10. A method implemented by a head-mounted display system of one or more processors, the method comprising: generating a rendered frame of virtual content for display as the virtual content via an emissive microdisplay projector system of the head-mounted display system, the rendered frame being associated with a second resolution, the emissive microdisplay projector comprising an emitter configured to output light forming the virtual content associated with a first, lower resolution; dividing the rendered frame of the virtual content into a plurality of sub-frames, each sub-frame comprising a subset of pixels included in the rendered frame; and sequentially outputting the light via the emissive microdisplay projector system, the light forming the plurality of sub-frames, the emissive microdisplay projector system being shifted according to a movement pattern for each sub-frame along one or more axes via one or more actuators, the emissive microdisplay projector system being shifted along the one or more axes in a plane parallel to a plane of an output pupil of the projector system.

[0039] Example 28 29. The method of example 28, wherein the emissive microdisplay projector system is associated with an array of light emitters, the geometric locations of which are shifted so as to be tiled within the areas between the individual emitters.

[0040] Example 29 30. The method of example 29, wherein the one or more actuators deflect the light emitter array along one or more axes.

[0041] Example 30 30. The method of example 29, wherein the one or more actuators displace the projection optics of the micro LED projector system along one or more axes, and the projection optics is configured to output light to a user of the head-mounted display system. The present invention provides, for example, the following. (Item 1) 1. A head-mounted display system, comprising: a support structure configured to mount on the head of a user; an optical projection system supported by the support structure, the optical projection system comprising: a microdisplay comprising an array of light emitters associated with a first resolution, the array of light emitters configured to output light forming a frame of virtual content; a projection optical system; One or more actuators an optical projection system comprising: an eyepiece supported by the support structure, the eyepiece configured to receive light from the optical projection system and direct the received light to the user; one or more processors, said one or more processors comprising: receiving a rendered frame of virtual content, the rendered frame comprising at least a portion associated with a second resolution, the second resolution being higher than the first resolution; causing the emissive microdisplay projector to output light that forms a first sub-frame of the rendered frame, the first sub-frame and the rendered frame being substantially the same size; displacing one or more movable parts of the optical projection system via the one or more actuators to adjust a position associated with an optical emitter light output from the optical projection system; causing the light projection system to output light forming a second sub-frame of the rendered frame; one or more processors configured to perform A head-mounted display system comprising: (Item 2) Item 1. A head-mounted display as described in item 1, wherein the portion associated with the second resolution is associated with the foveal region of the user's eye. (Item 3) 3. The head-mounted display of item 2, wherein the one or more processors are configured to determine that the light forming the portion is within a threshold angular distance of the user's fovea. (Item 4) the one or more processors: causing a light emitter to update the emitted light forming the portion for the second sub-frame; For the first sub-frame, causing the light emitter not to update the emitted light forming a portion of the rendered frame outside of said portion. Item 3. The head-mounted display according to item 2, configured as follows: (Item 5) Item 1. The head-mounted display system of item 1, wherein each emissive microdisplay array has an associated emitter size, the emitter size being less than the pixel pitch. (Item 6) Item 6. A head-mounted display as described in item 5, wherein the total number of subframes of the rendered frame is determined based on a size associated with the pixel pitch and the emitter size. (Item 7) 7. The head-mounted display of claim 6, wherein the one or more processors are configured to cause the light projection system to continuously output light that forms the total number of sub-frames. (Item 8) 8. The head-mounted display of item 7, wherein the one or more processors are configured to time-multiplex the rendered frames by causing the one or more actuators to shift a portion of the optical projection system on a sub-frame basis. (Item 9) 9. The head-mounted display of item 8, wherein the one or more processors are configured to cause the one or more actuators to shift portions of the optical projection system such that geometric locations associated with the array of light emitters are tiled within individual inter-emitter regions. (Item 10) Item 1. A head-mounted display as described in item 1, wherein the one or more processors are configured to cause the one or more actuators to shift a portion of the optical projection system according to a movement pattern, the movement pattern being a continuous movement pattern. (Item 11) Item 1. A head-mounted display as described in item 1, wherein the first sub-frame and the second sub-frame each comprise pixels associated with a distinct portion of the rendered frame. (Item 12) Item 1. A head-mounted display as described in item 1, wherein the light projection system comprises a plurality of arrays of the light emitters. (Item 13) Item 13. A head-mounted display as described in item 12, further comprising an X-cube prism, each of the arrays of light emitters facing a different side of the X-cube prism. (Item 14) Item 13. A head-mounted display as described in item 12, wherein each of the arrays of light emitters is configured to direct light into a dedicated associated projection optical system. (Item 15) Item 13. A head-mounted display as described in item 12, wherein the array of light emitters is mounted in a common rear plane. (Item 16) Item 1. A head-mounted display as described in item 1, wherein the one or more actuators are configured to displace the projection optical system. (Item 17) Item 1. A head-mounted display as described in item 1, wherein the one or more actuators are piezoelectric motors. (Item 18) Item 10. The head-mounted display of item 1, wherein the one or more actuators deflect the emissive microdisplay projector along two axes. (Item 19) Item 1. The head-mounted display of item 1, wherein the light emitter comprises a light-emitting diode. (Item 20) Item 1. A head-mounted display as described in item 1, wherein the array of light emitters is configured to emit light of multiple primary colors. (Item 21) 21. A head-mounted display as described in item 20, wherein each light emitter comprises a stack of constituent light generators, each constituent light generator emitting light of a different color. (Item 22) The eyepiece includes a waveguide assembly including one or more waveguides, each waveguide having: an incoupling optical element configured to incoupling light from the microdisplay into the waveguide; an outcoupling optical element configured to outcouple the incoupling light out of the waveguide; Item 1. A head-mounted display comprising: (Item 23) 1. A method implemented by a head mounted display system of one or more processors, the method comprising: providing a rendered frame of virtual content, the rendered frame having at least a portion associated with a second resolution; causing an emissive microdisplay projector to output light forming a first sub-frame of the rendered frame, the first sub-frame having a first resolution less than the second resolution, the emissive microdisplay projector comprising an array of light emitters having a pixel pitch associated with the first resolution; deflecting the emissive microdisplay projector via one or more actuators to adjust a geometric position associated with light output by the emissive microdisplay projector, the geometric position being adjusted a distance less than the pixel pitch; causing the emissive microdisplay projector to output light forming a second sub-frame of the rendered frame, the second sub-frame having the first resolution; A method comprising: (Item 24) 1. A system comprising: one or more processors; one or more computer storage media having instructions stored thereon that, when executed by the one or more processors, cause the one or more processors to: generating rendered frames of virtual content for display as augmented reality content via an emissive microdisplay projector system of the system, the rendered frames associated with a second resolution, the emissive microdisplay projector comprising one or more light emitter arrays configured to output light forming virtual content associated with a first, lower resolution; Dividing a rendered frame of the virtual content into a plurality of sub-frames, each sub-frame comprising a subset of pixels included in the rendered frame; sequentially outputting light via the emissive microdisplay projector system, the light forming the plurality of sub-frames, the emissive microdisplay projector system being shifted for each sub-frame according to a movement pattern via one or more actuators, the emissive microdisplay projector system being shifted along one or more axes in a plane parallel to a plane of an output pupil of the projector system; one or more computer storage media for performing operations including: A system comprising: (Item 25) Item 25. The system of item 24, wherein the one or more processors are configured to cause the one or more actuators to shift the emissive microdisplay projectors such that geometric locations associated with the emissive microdisplay array are tiled within individual inter-emitter regions. (Item 26) 26. The system of claim 25, wherein the one or more processors are configured to cause the one or more actuators to deflect the light emitter array along the one or more axes. (Item 27) Item 26. The system of item 25, wherein the microdisplay projector system comprises a projection optical system, the one or more processors are configured to cause the one or more actuators to displace the projection optical system along the one or more axes, and the projection optical system is configured to output light to a user of the system. (Item 28) 1. A method implemented by a head mounted display system of one or more processors, the method comprising: generating rendered frames of virtual content for display as virtual content via an emissive microdisplay projector system of the head-mounted display system, the rendered frames associated with a second resolution, the emissive microdisplay projector comprising an emitter configured to output light forming virtual content associated with a first, lower resolution; Dividing a rendered frame of the virtual content into a plurality of sub-frames, each sub-frame comprising a subset of pixels included in the rendered frame; continuously outputting light via the emissive microdisplay projector system, the light forming a plurality of subframes, the emissive microdisplay projector system being shifted along one or more axes via one or more actuators according to a movement pattern for each subframe, the emissive microdisplay projector system being shifted along one or more axes in a plane parallel to a plane of an output pupil of the projector system; A method comprising: (Item 29) Item 29. The method of item 28, wherein the emissive microdisplay projector system is shifted such that the geometric locations associated with the light emitter array are tiled within the area between the individual emitters. (Item 30) 30. The method of claim 29, wherein the one or more actuators deflect the light emitter array along the one or more axes. (Item 31) 30. The method of claim 29, wherein the one or more actuators displace projection optics of the micro LED projector system along the one or more axes, and the projection optics is configured to output light to a user of the head-mounted display system. [Brief explanation of the drawings]

[0042] [Figure 1] FIG. 1 illustrates a user's view of an augmented reality (AR) device.

[0043] [Figure 2] FIG. 2 illustrates a conventional display system for simulating a three-dimensional image for a user.

[0044] [Figure 3] 3A-3C illustrate the relationship between the radius of curvature and the radius of focus.

[0045] [Figure 4A] Figure 4A illustrates a representation of the accommodation-vergence response of the human visual system.

[0046] [Figure 4B] FIG. 4B illustrates an example of different accommodation and convergence states of a pair of a user's eyes.

[0047] [Figure 4C] FIG. 4C illustrates an example of a top-down view representation of a user viewing content through a display system.

[0048] [Figure 4D] FIG. 4D illustrates another example of a top-down view representation of a user viewing content through a display system.

[0049] [Figure 5] FIG. 5 illustrates aspects of an approach for simulating three-dimensional images by correcting for wavefront divergence.

[0050] [Figure 6] FIG. 6 illustrates an embodiment of a waveguide stack for outputting image information to a user.

[0051] [Figure 7] FIG. 7 illustrates an example of an output beam output by a waveguide.

[0052] [Figure 8] FIG. 8 illustrates an example of a stacked eyepiece, where each depth plane contains an image formed using multiple different primary colors.

[0053] [Figure 9A] FIG. 9A illustrates a cross-sectional side view of an example of a set of stacked waveguides, each including an internal coupling optical element.

[0054] [Figure 9B] FIG. 9B illustrates a perspective view of the multiple stacked waveguide embodiment of FIG. 9A.

[0055] [Figure 9C] FIG. 9C illustrates a top-down plan view of the multiple stacked waveguide embodiment of FIGS. 9A and 9B.

[0056] [Figure 9D] FIG. 9D illustrates a top-down plan view of another embodiment of multiple stacked waveguides.

[0057] [Figure 9E] FIG. 9E illustrates an example of a wearable display system.

[0058] [Figure 10] FIG. 10 illustrates an example of a wearable display system with a light projection system having a spatial light modulator and a separate light source.

[0059] [Figure 11A] FIG. 11A illustrates an example of a wearable display system with a light projection system having multiple emissive microdisplays.

[0060] [Figure 11B] FIG. 11B illustrates an example of an emissive microdisplay with an array of light emitters.

[0061] [Figure 12] FIG. 12 illustrates another example of a wearable display system with a light projection system having multiple emissive microdisplays and associated light redirecting structures.

[0062] [Figure 13A] FIG. 13A illustrates an example of a side view of a wearable display system with an optical projection system having multiple emissive microdisplays and an eyepiece having a waveguide with overlapping, laterally shifted, optical incoupling optical elements.

[0063] [Figure 13B] FIG. 13B illustrates another example of a wearable display system with a light projection system having multiple emissive microdisplays configured to direct light to a single light inter-coupling area of ​​the eyepiece.

[0064] [Figure 14] FIG. 14 illustrates an example of a wearable display system with a single emissive microdisplay.

[0065] [Figure 15] FIG. 15 illustrates a side view of an embodiment of an eyepiece having a stack of waveguides with overlapping interconnecting optical elements.

[0066] [Figure 16] FIG. 16 illustrates a side view of an example of a stack of waveguides with color filters to reduce ghosting or crosstalk between the waveguides.

[0067] [Figure 17] FIG. 17 illustrates an example of a top-down view of the eyepiece of FIGS.

[0068] [Figure 18] FIG. 18 illustrates another embodiment of a top-down view of the eyepiece of FIGS.

[0069] [Figure 19A] FIG. 19A illustrates a side view of an example eyepiece having a stack of waveguides with overlapping, laterally offset, internal coupling optical elements.

[0070] [Figure 19B] FIG. 19B illustrates a side view of an embodiment of the eyepiece of FIG. 19A with a color filter to reduce ghosting or crosstalk between waveguides.

[0071] [Figure 20A] FIG. 20A illustrates an example of a top-down view of the eyepiece of FIGS. 19A and 19B.

[0072] [Figure 20B] FIG. 20B illustrates another example of a top-down view of the eyepiece of FIGS. 19A and 19B.

[0073] [Figure 21] FIG. 21 illustrates a side view of an embodiment of rebouncing in a waveguide.

[0074] [Figure 22A] 22A-22C illustrate examples of top and bottom views of an eyepiece having an internal coupling optical element configured to reduce re-bounce. [Figure 22B] 22A-22C illustrate examples of top and bottom views of an eyepiece having an internal coupling optical element configured to reduce re-bounce. [Figure 22C] 22A-22C illustrate examples of top and bottom views of an eyepiece having an internal coupling optical element configured to reduce re-bounce.

[0075] [Figure 23A] 23A-23C illustrate additional examples of top and bottom views of eyepieces having internal coupling optical elements configured to reduce re-bounce. [Figure 23B] 23A-23C illustrate additional examples of top and bottom views of eyepieces having internal coupling optical elements configured to reduce re-bounce. [Figure 23C] 23A-23C illustrate additional examples of top and bottom views of eyepieces having internal coupling optical elements configured to reduce re-bounce.

[0076] [Figure 24A] FIG. 24A illustrates an example of the angular emission profile of light emitted by individual light emitters of an emissive microdisplay and captured by projection optics.

[0077] [Figure 24B] FIG. 24B illustrates an example of narrowing the angular emission profile using an array of optical collimators.

[0078] [Figure 25A] FIG. 25A illustrates an example of a side view of an array of tapered reflective wells for directing light into projection optics.

[0079] [Figure 25B] FIG. 25B illustrates an example of a side view of an asymmetric tapered reflective well.

[0080] [Figure 26] 26A-26C illustrate examples of differences in light paths for light emitters at different positions relative to the centerline of the superstrate lens.

[0081] [Figure 27] FIG. 27 illustrates an example of a side view of an individual light emitter of an emissive microdisplay with an overlying nanolens array.

[0082] [Figure 28] FIG. 28 is a perspective view of the emissive microdisplay embodiment of FIG.

[0083] [Figure 29] FIG. 29 illustrates an example of a wearable display system involving the full-color emissive microdisplay of FIG.

[0084] [Figure 30A] FIG. 30A illustrates an example of a wearable display system with an emissive microdisplay and an associated array of light collimators.

[0085] [Figure 30B] FIG. 30B illustrates an example of an optical projection system with multiple emissive microdisplays, each with an associated array of optical collimators.

[0086] [Figure 30C] FIG. 30C illustrates an example of a wearable display system with multiple emissive microdisplays, each with an associated array of light collimators.

[0087] [Figure 31] 31A and 31B illustrate an example of a waveguide assembly having a variable focus element for varying the wavefront divergence of the light relative to the viewer.

[0088] [Figure 32A] FIG. 32A illustrates an example of an emissive microdisplay having an array of light emitters separated by a gap.

[0089] [Figure 32B]FIG. 32B illustrates an example of how the emissive microdisplay of FIG. 32A can be configured to emulate a higher fill factor microdisplay through time multiplexing and repositioning of the array or associated optics.

[0090] [Figure 32C] FIG. 32C illustrates an example of a foveated image formed by an emissive microdisplay such as the emissive microdisplay of FIG. 32A.

[0091] [Figure 32D] FIG. 32D illustrates an example of an emissive microdisplay, such as the emissive microdisplay of FIG. 32A, configured to form a foveated image with three or more levels of resolution within the image.

[0092] [Figure 33] FIG. 33 illustrates another example of a foveated image provided by an emissive microdisplay, such as the emissive microdisplay of FIG. 32A.

[0093] [Figure 34] FIG. 34 illustrates various exemplary paths of movement of a portion of an emissive microdisplay to shift the position of a displayed pixel.

[0094] [Figure 35] Figures 35A and 35B illustrate how displacement of the light emitter and projection optics can change the position of the displayed pixel.

[0095] [Figure 36A] FIG. 36A illustrates an example of a wearable display system having an optical projection system with actuators coupled to the projection optics.

[0096] [Figure 36B]FIG. 36B illustrates an example of a wearable display system having a light projection system with multiple actuators, each coupled to a different microdisplay.

[0097] [Figure 37A] FIG. 37A illustrates an example of a wearable display system having an optical projection system with an eyepiece having a single waveguide.

[0098] [Figure 37B] FIG. 37B illustrates an example of a wearable display system having a light projection system in which a single array of light emitters outputs light of different primary colors using actuators coupled to the projection optics.

[0099] [Figure 37C] FIG. 37C illustrates an embodiment of a wearable display system similar to that of FIG. 37B, except for the attachment of the actuators to the microdisplay rather than the projection optics.

[0100] [Figure 38A] FIG. 38A illustrates an example of a wearable display system having a light projection system that directs light of different primary colors to an eyepiece without the use of an optical combiner and combines the different colored light using actuators coupled to the projection optics.

[0101] [Figure 38B] FIG. 38B illustrates another example of a wearable display system having a light projection system that directs light of different primary colors to the eyepiece without the use of an optical combiner and combines the different colored light using actuators coupled to the projection optics.

[0102] [Figure 38C]FIG. 38C illustrates an example of a wearable display system similar to that of FIG. 38A, except that the actuators are attached to the individual microdisplays rather than to the projection optics.

[0103] [Figure 38D] FIG. 38D illustrates an embodiment of a wearable display system similar to that of FIG. 38B, except for the attachment of the actuators to the integrated microdisplay structure rather than the projection optics.

[0104] [Figure 39] FIG. 39 illustrates a flowchart of an exemplary process for outputting sub-frames of a rendered frame of virtual content. DETAILED DESCRIPTION OF THE INVENTION

[0105] An augmented reality (AR) or virtual reality (VR) system may display virtual content to a user or viewer. This content may be displayed on a head-mounted display, for example, as part of eyewear, that projects image information to the user's eyes. Additionally, if the system is an AR system, the display may also transmit light from the surrounding environment to the user's eyes, allowing a view of the surrounding environment. As used herein, it should be understood that a "head-mounted" or "head-mountable" display is a display that can be mounted on the user's head or viewer.

[0106] To improve the usability of an AR or VR system (also referred to simply as a "display system"), it may be beneficial to reduce the size, weight, and / or power consumption of the display system. As an example, a user may be more likely to use the display system if the size and general prominence of the display system are reduced. As another example, a user may be more likely to use the display system if the weight placed on the user's head is reduced. Similarly, reduced power consumption can enable the use of smaller batteries, reduce heat generated by the display system, etc. Various embodiments described herein facilitate such benefits, including reducing the size of portions of the display system.

[0107] As described herein, the light (also referred to herein as image light) that forms the virtual content may be generated by one or more display technologies. For example, the light may be generated by a light projection system included within a display system. This light may then be routed via an optical system to a user of the display system for output as virtual content. The virtual content may be represented as image pixels contained within rendered frames that are presented sequentially to the user. To achieve high-quality (e.g., lifelike) virtual content, the display system may render and then output frames of the virtual content at sufficient resolution (e.g., above a threshold resolution). Thus, the image pixels may be close enough together to achieve sufficient resolution.

[0108] However, it should be understood that design constraints associated with a display system may limit the ability to achieve such proximity within image pixels, and therefore the resolution. For example, to miniaturize a display system, the display system may be required to have a reduced display size (e.g., projector size). An exemplary display may include a liquid crystal on silicon (LCoS) display. To output image light that forms the virtual content, the LCoS display may be required to utilize a separate lighting module including one or more light emitters. In this example, the LCoS panel may impose a spatially varying modulation on the generated light to form the virtual content. However, to reduce the size associated with an LCoS panel while maintaining high resolution, the pixel pitch associated with the LCoS panel may need to be reduced. Pixel pitch, as described herein, may refer to the physical distance on the display between similar locations on similar elements of the display that form image pixels. Due to the physical constraints on the small pixel pitch combined with the need for a separate lighting module, an LCoS display may be larger than desired in some applications.

[0109] Some embodiments disclosed herein advantageously include an emissive microdisplay, such as a microLED display. In some embodiments, the microdisplay is a microOLED display. Display systems utilizing emissive microdisplays may avoid the added bulk of lighting modules. In addition, emissive microdisplays may advantageously facilitate the presentation of images with a small pixel pitch. As described, exemplary display systems may utilize one or more emissive microdisplays to achieve reduced size, weight, and power consumption, among other advantages.

[0110] Emissive microdisplays have several advantages for use in wearable display systems. As an example, the power consumption of emissive microdisplays generally varies with image content, such that dark or sparse content generally requires less power to display. Because AR environments are often sparse because it may be desirable for a user to be able to see their surroundings, emissive microdisplays may have average power consumption that is lower than that of other display technologies that use spatial light modulators to modulate light from light sources. In contrast, other display technologies may utilize substantial power even for dark, sparse, or "all-off" virtual content. As another example, emissive microdisplays may provide significantly higher frame rates (which may enable the use of partial-resolution arrays) and provide a low level of visually apparent motion artifacts (e.g., motion blur). As another example, emissive microdisplays may not require the type of polarization optics required by LCoS displays. Thus, emissive microdisplays may avoid the optical losses present in polarization optics.

[0111] While arrays of light emitters such as micro-LEDs may offer substantial size, weight, and / or power savings, current light emitters do not provide a sufficiently small pixel pitch to enable high-resolution virtual content within a small display system form factor. As a non-limiting example, some micro-LED-based microdisplays may enable a pixel pitch of about 2 to about 3 microns. Even with such pixel pitches, to provide the desired number of pixels, micro-LED displays may still be undesirably large for use in wearable display systems, particularly because a goal for such systems may be to have a form factor and size similar to that of eyeglasses.

[0112] As will be described in further detail, optical projection systems including emissive microdisplays may achieve effective small pixel pitches through rapid physical adjustments or displacements to portions of the optical projection system. For example, the emissive microdisplay may be physically adjusted in position or displaced along one or more axes. As another example, an optical element (e.g., projection optics) may be physically adjusted in position or displaced along one or more axes.

[0113] As described herein, the size associated with a light emitter, such as a micro-LED, may be referred to as emitter size. Emitter size may refer to the dimension of the light emitter along a particular axis (e.g., a lateral axis). Emitter size may also refer to the dimension of the light emitter along two axes (e.g., lateral and longitudinal axes). Similarly, pixel pitch may refer to the distance between similar points on immediately adjacent light emitters along a particular axis (e.g., a lateral axis), with each different axis having its own pixel pitch. For example, in some embodiments, light emitters may be spaced closer together along a first axis than along a second axis (e.g., an orthogonal axis). An example of an array of light emitters is described in further detail herein and illustrated in FIG. 32A.

[0114] It should be understood that the size of a light emitter may be less than the gap separating immediately adjacent light emitters. For example, it may be difficult to form an emissive microdisplay with light emitters above a threshold density due to physical and electrical constraints. Exemplary constraints may include current crowding, substrate droop, etc. Thus, there may be a substantial gap or space between adjacent light emitters. The gap between two light emitters is referred to herein as the inter-emitter region. The inter-emitter region, an example of which is illustrated in FIG. 32A, may therefore delineate the area (e.g., maximum area) of an emissive microdisplay that includes a single light emitter. The size of the inter-emitter region may therefore limit the range over which an emissive microdisplay can achieve a certain high density or high resolution.

[0115] Advantageously, the ability to operate light emitters, such as micro-LEDs, at high speeds may enable time-multiplexed presentation of images using the same one or more emissive microdisplays. For example, the geometric position of the light emitter relative to the projection optics may be shifted so that the same light emitter is enabled to present different pixels of an image at different times. In some embodiments, a rendered frame of virtual content may be presented as a series of sub-frames in rapid succession via a time-multiplexing scheme. In this example, each sub-frame may be associated with a specific physical position of the light emitter relative to the projection optics. It should be understood that the geometric position can thus be varied by changing the locations of the light emitter and the projection optics relative to each other (e.g., by changing the physical position of the light emitter while holding the projection optics stationary, by changing the physical position of the projection optics while holding the light emitter stationary, or by changing the physical positions of both the light emitter and the projection optics). As described in more detail below, the geometric position may be adjusted (e.g., via one or more actuators) to cause the light emitter to tile the areas between individual emitters. Thus, emissive microdisplays can advantageously achieve high-resolution virtual content output.

[0116] Thus, an optical projection system may be configured to project each full-resolution frame of virtual content by projecting one or more partial-resolution sub-frames. For example, one or more partial-resolution sub-frames may be projected. The sub-frames may be projected in rapid succession and may be offset from one another (e.g., by less than a full pixel pitch along one or more axes over which the sub-frames are translated). For example, an emissive microdisplay included within the projector may be physically displaced along one or more axes. As described above, an emissive microdisplay may include light emitters, such as micro-LEDs, having a pixel pitch. This pixel pitch may therefore inform the resolution at which the emissive microdisplay can output frames of virtual content. The light emitters may be adjusted in position, for example, by less than the pixel pitch, to effectively reduce the functional pixel pitch and the gap between light emitters, thus increasing the resolution for the same size display. As an example, a display may include light emitters separated by a pixel pitch of 2.5 microns, and each light emitter may have an emitter size of 0.833 microns. In some embodiments, the light emitters may be adjusted in position a number of times based on the number of times the light emitters may be translated in parallel to different (e.g., non-overlapping) positions within the inter-emitter area. In this example, the inter-emitter area is 6.25 microns. 2 The exemplary light emitter may be adjusted in position three times along a first axis and three times along a second orthogonal axis. Thus, the exemplary light emitter may effectively assume nine positions within the inter-emitter region. For one or more of the nine positions, a particular sub-frame of the same rendered frame of virtual content may be presented. Thus, the successively presented sub-frames may be perceived as a high-resolution frame of virtual content. In effect, the light emitter may form an image with an apparent pixel density that is higher than the physical density of the light emitter.

[0117] In some embodiments, the user's visual system may merge the sub-frames together so that the user perceives a full-resolution frame. For example, the pixels of the sub-frames may be blended to form a full-resolution frame. Preferably, the sub-frames may be displayed sequentially at a frame rate higher than the flicker fusion threshold of the human visual system. As an example, the flicker fusion threshold may be 60 Hz, which is considered fast enough that most users do not perceive the sub-frames as being displayed at different times. In some embodiments, the different sub-frames are displayed sequentially at a rate equal to or higher than the flicker fusion threshold (e.g., equal to or higher than 60 Hz).

[0118] As a result, an emissive microdisplay can be configured to have fewer light emitters than the number of image pixels contained in each full-resolution rendered frame of virtual content. For example, a full-resolution image may include 2,000 x 2,000 pixels, while the emissive microdisplay may be an array of only 1,000 x 1,000 elements. The use of lower-resolution emissive microdisplays may be particularly beneficial for wearable systems, such as the display systems described herein. As an example, lower-resolution displays may be smaller, lighter, and / or consume less power than higher-resolution displays.

[0119] While the above describes steps for moving or adjusting the position of an emissive microdisplay (e.g., including a microLED array), it should be understood that the position of the projection optics may alternatively or additionally be adjusted. For example, as will be described in further detail below with respect to Figures 35A-35B, the projection optics may route light generated via the emissive microdisplay to a user of the display system. As an example, the projection optics may route light to an input incoupling optical element (e.g., an incoupling grating) of an eyepiece configured to receive and direct light encoded with image information (image light) to a user. Thus, instead of physically translating the emissive microdisplay, the projection optics may be translated along one or more axes. During translation, the projection optics may change the geometric position of each array along one or more axes prior to outputting light for incoupling via the incoupling grating. As described herein, an optical projection system may include one or more emissive microdisplays, projection optics, etc. Thus, the light output of an optical projection system may be adjusted by physical translation of parts of the system (eg, by changing the location of the pixels presented by the optical projection system).

[0120] It should be understood that some portions of a rendered frame of virtual content may be more visually apparent to a user than other portions. For example, a user may have high visual acuity with respect to portions of the virtual content that lie on the user's fovea (referred to herein as "foveal portions"). To determine the location of these foveal portions, the display system may determine a fixation point at which the user is fixating. Portions of the virtual content that are within a threshold angular distance of this fixation point may be identified as being on the user's fovea. As will be described, the display system may be configured to increase the resolution associated with the foveal portion. The resolution of the remaining portions may be increased very little or not at all.

[0121] As an example, an emissive microdisplay may be configured to update pixels included within the foveal portion at a rate greater than that for pixels included within other portions. As described above, the geometric positions of the light emitters may be translated or adjusted to tile the inter-emitter regions of the array. Optionally, light emitters utilized to output light forming pixels including the foveal region may be updated for a relatively high percentage of different geometric positions (e.g., every different geometric position), while light emitters for forming pixels away from the foveal region may be updated for a lower percentage of different geometric positions (e.g., these light emitters may be “off” or may simply present the same information as in their previous positions). Light emitters utilized to output light forming pixels included within other regions may be updated less frequently. For example, these light emitters may be updated twice or only once for a given full-resolution rendered frame of virtual content. For example, the light emitter for the foveal region may be updated at every four different geometric locations within the inter-emitter region, while the light emitter corresponding to the peripheral portion of the image may only be updated at every other geometric location.

[0122] As a result, a rendered frame formed from rapidly displayed or projected sub-frames may have an effective resolution that varies across the rendered frame. Using foveated imaging, the effective resolution of an emissive microdisplay can be made higher in a foveated region (e.g., a region of interest, a region on which a user focuses, a region designated by a user, a region designated by a designer, etc.) and lower in other regions (e.g., outside the region of interest). Configuring an emissive microdisplay to provide a foveated image may further help conserve resources, for example, by eliminating and / or reducing the processing and power load associated with displaying or projecting less interesting regions (e.g., regions less likely to attract a user's attention). Exemplary Display Systems with Emissive Microdisplays

[0123] Advantageously, display systems utilizing emissive microdisplays as described herein may enable low weight and compact form factors, and may also provide high frame rates and low motion blur. Preferably, the microdisplay is an emissive microdisplay, which offers the advantages of high brightness and high pixel density. In some embodiments, the emissive microdisplay is a microLED display. In some other embodiments, the emissive microdisplay is a microOLED display. In some embodiments, the emissive microdisplay comprises an array of light emitters having a pitch of less than 10 μm, less than 8 μm, less than 6 μm, less than 5 μm, or less than 2 μm, e.g., including 1-5 μm, and an emitter size of 2 μm or less, 1.7 μm or less, or 1.3 μm or less. In some embodiments, the emitter size is within a range having an upper size limit as noted above and a lower size limit of 1 μm. In some embodiments, the emitter size to pitch ratio is 1:1 to 1:5, 1:2 to 1:4, or 1:2 to 1:3, which can have the advantage of individual control of the emitters and efficient use of light emitted by the eyepiece, as discussed further herein.

[0124] In some embodiments, multiple emissive microdisplays may be utilized to form images for a head-mounted display system. The light containing the image information to form these images may be referred to as image light. It should be understood that the image light may vary in, for example, wavelength, intensity, polarization, etc. The emissive microdisplays output the image light to an eyepiece, which then relays the light to the user's eye.

[0125] In some embodiments, multiple emissive microdisplays may be utilized and positioned on different sides of an optical combiner, such as an X-cube prism or dichroic X-cube. The X-cube prism receives light rays from different microdisplays on different faces of the cube and outputs the light rays from the same face of the cube. The output light may be directed toward projection optics configured to converge or focus the image light onto an eyepiece.

[0126] In some embodiments, the multiple emissive microdisplays comprise monochrome microdisplays configured to output light of a single primary color. Combining the various primary colors forms a full-color image. In some other embodiments, one or more of the emissive microdisplays may have subpixels configured to emit light of two or more, but not all, of the primary colors utilized by the display system. For example, a single emissive microdisplay may have subpixels that emit blue and green light, while separate emissive microdisplays on different faces of the X-cube may have pixels configured to emit red light. In some embodiments, the multiple microdisplays are full-color displays, each comprising a pixel formed from multiple subpixels configured to emit light of, for example, different primary colors. Advantageously, combining the light of multiple full-color microdisplays may increase display brightness and dynamic range.

[0127] It should be understood that an emissive microdisplay may comprise an array of light emitters. The light emitters may emit light with a Lambertian angular emission profile. Unfortunately, such an angular emission profile may result in "wasted" light, as only a small portion of the emitted light may ultimately be incident on the eyepiece. In some embodiments, a light collimator may be utilized to narrow the angular emission profile of light emitted by a light emitter. As used herein, a light collimator is an optical structure that narrows the angular emission profile of incident light. That is, a light collimator receives light from an associated light emitter with a relatively wide initial angular emission profile and outputs the light with an angular emission profile narrower than the wide initial angular emission profile. In some embodiments, the rays of light exiting the light collimator are more parallel than the rays of light received by the light collimator before being transmitted through and exiting the collimator. Examples of light collimators include microlenses, nanolenses, reflective wells, metasurfaces, and liquid crystal gratings. In some embodiments, the light collimators may be configured to steer and ultimately focus the light onto different laterally offset light combining optical elements. In some embodiments, each light emitter has a dedicated light collimator. The light collimator is preferably positioned directly adjacent to or in contact with the light emitter and captures a large percentage of the light emitted by the associated light emitter.

[0128] In some embodiments, a single emissive microdisplay may be utilized to direct light to the eyepiece. For example, the single emissive microdisplay may be a full-color display comprising light emitters that emit light of different primary colors. In some embodiments, the light emitters may form localized groups within a common area, each group comprising a light emitter that emits light of each primary color. In such embodiments, each group of light emitters may share a common microlens. Advantageously, light of different colors from different light emitters follows different paths through the microlens, which may manifest as light of different primary colors incident on different internal coupling optical elements of the eyepiece, as discussed herein.

[0129] In some embodiments, a full-color microdisplay may comprise repeating groups of light emitters of the same primary color. For example, the microdisplay may include rows of light emitters, with the light emitters in each individual row configured to emit light of the same color. Thus, different rows may emit light of different primary colors. In addition, the microdisplay may have an associated array of light collimators configured to direct light to desired locations on the eyepiece, e.g., to associated internal coupling optical elements. Advantageously, although the individual light emitters of such a full-color microdisplay may not be positioned to form a high-quality full-color image as viewed directly on the microdisplay, the lens array appropriately steers light from the light emitters to the eyepiece, which combines the monochrome images formed by the different color light emitters, thereby forming a high-quality full-color image.

[0130] In some embodiments, the eyepiece, which receives image light from the microdisplay, may include a waveguide assembly. The area of ​​the waveguide of the waveguide assembly onto which the image light is incident may include an in-coupling optical element that in-couples the incident image light so that the light propagates through the waveguide by total internal reflection (TIR). In some embodiments, the waveguide assembly may include a stack of waveguides, each with an associated in-coupling optical element. Different in-coupling optical elements may be configured to in-couple light of different colors, such that different waveguides may be configured to propagate light of different colors therein. A waveguide may include an out-coupling optical element that out-couples light propagating therein so that the out-coupled light propagates toward the user's eye. In some other embodiments, the waveguide assembly may include a single waveguide with an associated in-coupling optical element configured to in-couple light of different primary colors.

[0131] In some embodiments, the in-coupling optical elements are offset laterally as viewed by the projection optics. Different in-coupling optical elements may be configured to in-couple light of different colors. Preferably, image light of different colors follows different paths to the eyepiece and therefore impinges on different corresponding in-coupling optical elements.

[0132] In some other embodiments, other types of eyepieces or optics for relaying image light to the user's eye may be utilized. For example, as discussed herein, the eyepiece may include one or more waveguides that propagate image light therethrough by TIR. As another example, the eyepiece may include a basin mirror combiner that includes a semi-transparent mirror that both directs image light to the viewer and allows for a view of the surrounding environment.

[0133] In some embodiments, the eyepiece may be configured to selectively output light with different amounts of wavefront divergence to provide multiple virtual depth planes (also referred to herein simply as "depth planes") of virtual content perceived to be at different distances away from the user. For example, the eyepiece may include multiple waveguides, each having an outcoupling optical element with different refractive power for outputting light with different amounts of wavefront divergence. In some other embodiments, a variable-focus element may be provided between the eyepiece and the user's eye. The variable-focus element may be configured to dynamically change refractive power to provide a desired wavefront divergence for particular virtual content. In some embodiments, as an alternative to, or in addition to, a waveguide optical structure to provide refractive power, the display system may also include multiple lenses that provide, or in addition to, refractive power.

[0134] In addition to the compact form factor and high frame rate discussed above, emissive microdisplays, according to some embodiments, may offer one or more of the following advantages. For example, microdisplays may offer a significantly smaller pixel pitch and higher pixel density. Microdisplays may also offer high brightness and efficiency. For example, light emitters in emissive microdisplays may only consume power to emit light when needed to provide content at a certain brightness. This contrasts with other display technologies, where a light source may illuminate an entire panel of pixels, regardless of whether some of those pixels are dark. Furthermore, it should be appreciated that the human visual system integrates received light over time, and light emitters in emissive microdisplays, such as microLEDs, advantageously have a high duty cycle (e.g., light emitters in a microdisplay include a short activation period to ramp from an "off" to a full "on" state, and a corresponding short period to ramp down from an "on" state to an "off" state, allowing the light emitters to emit light at an on level for a large percentage of each cycle). As a result, less power may be used to generate an image with a given perceived brightness compared to conventional display technologies with lower duty cycles. In some embodiments, the duty cycle may be 70% or greater, 80% or greater, or 90% or greater. In some embodiments, the duty cycle may be approximately 99%. Additionally, as described herein, microdisplays may facilitate significantly higher frame rates, which may provide advantages including reducing misalignment between a user's head position and the displayed content.

[0135] Reference is now made to the drawings in which like reference numerals refer to like parts throughout. Unless otherwise indicated, the drawings are schematic and are not necessarily drawn to scale.

[0136] FIG. 2 illustrates a conventional display system for simulating a three-dimensional image for a user. It should be understood that when a user's eyes are spaced apart and viewing a real object in space, each eye may have a slightly different view of the object, forming an image of the object at a different location on each eye's retina. This may be referred to as binocular disparity and may be utilized by the human visual system to provide the perception of depth. Conventional display systems simulate binocular disparity by presenting two distinct images 190, 200, one for each eye 210, 220, with slightly different views of the same virtual object, corresponding to the view of the virtual object as it would appear by each eye as if the virtual object were a real object at a desired depth. These images provide binocular cues that the user's visual system may interpret to derive the perception of depth.

[0137] Continuing with reference to FIG. 2 , images 190 and 200 are spaced apart from eyes 210 and 220 by a distance 230 on the z-axis. The z-axis is parallel to the optical axis of the viewer when the eye is fixating on an object at optical infinity directly in front of the viewer. Images 190 and 200 are flat and at a fixed distance from eyes 210 and 220. Based on slightly different views of the virtual object in the images presented to eyes 210 and 220, respectively, the eyes may necessarily rotate so that the image of the object falls on a corresponding point on each eye's retina, maintaining single binocular vision. This rotation may cause the gaze of each eye 210 and 220 to converge on a point in space where the virtual object is perceived to reside. As a result, providing three-dimensional images traditionally involves manipulating the convergence and divergence of the user's eyes 210 and 220 and providing binocular cues that the human visual system interprets to provide the perception of depth.

[0138] However, creating a realistic and comfortable perception of depth is challenging. It should be understood that light from an object at different distances from the eye has a wavefront with different amounts of divergence. Figures 3A-3C illustrate the relationship between distance and light ray divergence. The distance between the object and the eye 210 is represented in the order of decreasing distances R1, R2, and R3. As shown in Figures 3A-3C, light rays become more divergent as the distance to the object decreases. Conversely, as the distance increases, light rays become more collimated. In other words, the light field generated by a point (an object or portion of an object) can be said to have a spherical wavefront curvature that is a function of the distance the point is from the user's eye. The curvature increases as the distance between the object and the eye 210 decreases. While only a single eye 210 is illustrated in Figures 3A-3C and various other figures herein for clarity of illustration, the discussion regarding the eye 210 may apply to both eyes 210 and 220 of the viewer.

[0139] Continuing with reference to Figures 3A-3C, light from an object that a viewer's eye is fixating may have different wavefront divergences. Due to the different wavefront divergences, the light may be focused differently by the eye's lens, which in turn may require the lens to assume a different shape to form a focused image on the eye's retina. If a focused image is not formed on the retina, the resulting retinal blur acts as an accommodative cue, causing the shape of the eye's lens to change until a focused image is formed on the retina. For example, the accommodative cue may trigger relaxation or contraction of the ciliary muscles surrounding the eye's lens, thereby modulating the force applied to the lower ligament that holds the lens, thus changing the shape of the eye's lens and forming a focused image of the fixated object on the eye's retina (e.g., fovea) until retinal blur of the fixated image is eliminated or minimized. The process by which the eye's lens changes shape can be referred to as accommodation, and the shape of the eye's lens required to form a focused image of a fixated object on the eye's retina (e.g., the fovea) can be referred to as the state of accommodation.

[0140] Referring now to Figure 4A, a representation of the accommodation-vergence response of the human visual system is illustrated. Eye movement to fixate an object causes the eye to receive light from the object, which forms an image on each of the eye's retinas. The presence of retinal blur in the image formed on the retina can provide a cue for accommodation, and the relative location of the image on the retina can provide a cue for vergence. Accommodative cues cause accommodation, resulting in the eye's lens adopting a specific accommodation state in which a focused image of the object is formed on the eye's retina (e.g., the fovea). Conversely, vergence cues cause vergence movements (eye rotations) so that the images formed on each retina of each eye are at corresponding retinal points, maintaining single binocular vision. In these positions, the eyes can be said to adopt a specific vergence state. Continuing with reference to FIG. 4A , accommodation can be understood as the process by which the eyes achieve a particular accommodation state, and convergence can be understood as the process by which the eyes achieve a particular convergence state. As shown in FIG. 4A , the accommodation and convergence states of the eyes can change when the user fixates on a different object. For example, the accommodated state can change when the user fixates on a new object at a different depth on the z-axis.

[0141] Without being limited by theory, it is believed that a viewer of an object may perceive the object as "three-dimensional" due to a combination of vergence and accommodation. As described above, vergence movement of the two eyes relative to one another (e.g., eye rotation such that the pupils move toward or away from one another, converging the eyes' lines of sight and fixating on an object) is closely coupled to accommodation of the eye's lenses. Under normal conditions, a change in focus of the eye's lenses to change focus from one object to another at a different distance will automatically produce a corresponding change in vergence to the same distance, a relationship known as the "accommodation-vergence reflex." Similarly, a change in vergence will, under normal conditions, induce a corresponding change in lens shape.

[0142] 4B, an example of different accommodation and convergence states of the eyes is illustrated. Paired eye 222a fixates an object at optical infinity, while paired eye 222b fixates an object 221 at less than optical infinity. Notably, the convergence states of each pair of eyes are different: paired eye 222a points straight ahead, while paired eye 222 converges on object 221. The accommodation states of the eyes forming each pair of eyes 222a and 222b are also different, as represented by the different shapes of lenses 210a, 220a.

[0143] Unfortunately, many users of conventional "3-D" display systems may find such systems uncomfortable or may not perceive any depth perception due to a mismatch between accommodation and convergence states in these displays. As described above, many stereoscopic or "3-D" display systems display a scene by providing a slightly different image to each eye. Such systems are uncomfortable for many viewers because, among other things, they simply provide different presentations of a scene, causing changes in the eyes' convergence states without corresponding changes in the eyes' accommodation states. Rather, images are presented by the display at a fixed distance from the eyes so that the eyes view all image information in a single accommodation state. Such an arrangement counters the "accommodation-vergence-divergence reflex" by causing changes in the convergence states without a corresponding change in the accommodation state. This mismatch is believed to cause viewer discomfort. Display systems that offer better alignment between accommodation and convergence-divergence movements may create a more realistic and comfortable simulation of three-dimensional images.

[0144] Without being limited by theory, it is believed that the human eye is typically capable of interpreting a finite number of depth planes to provide depth perception. As a result, a highly realistic simulation of perceived depth may be achieved by providing the eye with different presentations of images corresponding to each of these limited number of depth planes. In some embodiments, the different presentations may provide both vergence cues and matching cues for accommodation, thereby providing physiologically correct accommodation-vergence divergence matching.

[0145] 4B , two depth planes 240 are illustrated, corresponding to different distances in space from the eyes 210, 220. For a given depth plane 240, vergence-divergence cues may be provided by displaying appropriately different perspective images for each eye 210, 220. Additionally, for a given depth plane 240, the light forming the image provided to each eye 210, 220 may have a wavefront divergence corresponding to the light field generated by a point at the distance of that depth plane 240.

[0146] In the illustrated embodiment, the distance along the z-axis of depth plane 240 containing point 221 is 1 meter. As used herein, distance or depth along the z-axis may be measured with a zero point located at the exit pupil of the user's eye. Thus, depth plane 240 located at a depth of 1 meter corresponds to a distance of 1 meter away from the exit pupil of the user's eye on the optical axis of those eyes when those eyes are pointed toward optical infinity. As an approximation, the depth or distance along the z-axis may be measured from a display (e.g., the surface of a waveguide) in front of the user's eye, and a value for the distance between the device and the exit pupil of the user's eye may be added. That value may be referred to as pupil distance and may correspond to the distance between the exit pupil of the user's eye and a display worn by the user in front of the eye. In practice, the value for pupil distance may be a normalized value generally used for all viewers. For example, pupil distance may be assumed to be 20 mm, and the depth plane at a depth of 1 meter may be at a distance of 980 mm in front of the display.

[0147] 4C and 4D, examples of matched accommodation-vergence-divergence distances and mismatched accommodation-vergence-divergence distances are illustrated, respectively. As illustrated in FIG. 4C, the display system may provide an image of a virtual object to each eye 210, 220. The image may cause the eyes 210, 220 to assume a convergence-divergence state in which the eyes converge on point 15 on the depth plane 240. In addition, the image may be formed by light having a wavefront curvature corresponding to the real object on that depth plane 240. As a result, the eyes 210, 220 assume an accommodation state in which the image is focused on the retinas of those eyes. Thus, the user may perceive the virtual object as being at point 15 on the depth plane 240.

[0148] It should be understood that the accommodation and convergence states of the eyes 210, 220 are each associated with a particular distance on the z-axis. For example, an object at a particular distance from the eyes 210, 220 will cause those eyes to assume a particular accommodation state based on the distance of the object. The distance associated with a particular accommodation state is referred to as the accommodation distance A. d Similarly, a particular convergence-divergence distance V associated with the eyes in a particular convergence-divergence state or position relative to one another may be referred to as d However, if the accommodation distance and the convergence distance are consistent, the relationship between accommodation and convergence can be said to be physiologically correct. This is considered the most comfortable scenario for the viewer.

[0149] However, in a stereoscopic display, the accommodation distance and the vergence distance may not always be aligned. For example, as illustrated in FIG. 4D , images displayed to the eyes 210, 220 may be displayed with a wavefront divergence corresponding to the depth plane 240, and the eyes 210, 220 may be in a particular accommodation state in which points 15a, 15b on that depth plane are focused. However, the images displayed to the eyes 210, 220 may provide convergence cues that cause the eyes 210, 220 to converge on a point 15 that is not located on the depth plane 240. As a result, in some embodiments, the accommodation distance corresponds to the distance from the exit pupils of the eyes 210, 220 to the depth plane 240, while the vergence distance corresponds to the greater distance from the exit pupils of the eyes 210, 220 to point 15. The accommodation distance is different from the vergence distance. As a result, there is an accommodation-vergence-divergence mismatch. Such a mismatch is considered undesirable and can cause discomfort to the user. The mismatch can be caused by distance (e.g., V d -A d ) and can be characterized in terms of diopters.

[0150] It should be understood that in some embodiments, a reference point other than the exit pupil of the eye 210, 220 may be used to determine the distance for determining accommodation-vergence mismatch, so long as the same reference point is used for accommodation distance and vergence distance. For example, the distance may be measured from the cornea to the depth plane, from the retina to the depth plane, from the eyepiece (e.g., a waveguide in a display device) to the depth plane, etc.

[0151] Without being limited by theory, it is believed that a user may still perceive an accommodation-vergence-divergence mismatch of up to about 0.25 diopters, up to about 0.33 diopters, and up to about 0.5 diopters as physiologically correct, without the mismatch itself causing significant discomfort. In some embodiments, a display system disclosed herein (e.g., display system 250, FIG. 6 ) presents a viewer with an accommodation-vergence-divergence mismatch of about 0.5 diopters or less. In some other embodiments, the accommodation-vergence-divergence mismatch of images provided by the display system is about 0.33 diopters or less. In still other embodiments, the accommodation-vergence-divergence mismatch of images provided by the display system is about 0.25 diopters or less, including about 0.1 diopters or less.

[0152] FIG. 5 illustrates aspects of an approach for simulating a three-dimensional image by modifying wavefront divergence. The display system includes a waveguide 270 configured to receive light 770 encoded with image information and output the light to a user's eye 210. The waveguide 270 may output light 650 with a defined amount of wavefront divergence corresponding to the wavefront divergence of a light field generated by a point on a desired depth plane 240. In some embodiments, the same amount of wavefront divergence is provided for all objects presented on that depth plane. In addition, the user's other eye will be illustrated as being provided with image information from a similar waveguide.

[0153] In some embodiments, a single waveguide may be configured to output light with a set wavefront divergence corresponding to a single or limited number of depth planes, and / or the waveguide may be configured to output light of a limited range of wavelengths. As a result, in some embodiments, multiple or stacked waveguides may be utilized to provide different wavefront divergences for different depth planes and / or to output light of different ranges of wavelengths. As used herein, it should be understood that a depth plane may be a plane or may follow the contour of a curved surface.

[0154] 6 illustrates an example of a waveguide stack for outputting image information to a user. Display system 250 includes a stack of waveguides or stacked waveguide assembly 260 that can be utilized to provide a three-dimensional perception to the eye / brain using multiple waveguides 270, 280, 290, 300, 310. It should be understood that display system 250 may be considered a light field display in some embodiments. Additionally, waveguide assembly 260 may also be referred to as an eyepiece.

[0155] In some embodiments, display system 250 may be configured to provide a substantially continuous cue for convergence and multiple discrete cues for accommodation. The cues for convergence may be provided by displaying different images to each of the user's eyes, and the cues for accommodation may be provided by outputting light forming images with selectable discrete amounts of wavefront divergence. In other words, display system 250 may be configured to output light with variable levels of wavefront divergence. In some embodiments, each discrete level of wavefront divergence corresponds to a particular depth plane and may be provided by a particular one of waveguides 270, 280, 290, 300, and 310.

[0156] 6, the waveguide assembly 260 may also include multiple features 320, 330, 340, 350 between the waveguides. In some embodiments, the features 320, 330, 340, 350 may be one or more lenses.

[0157] The multiple lenses 320, 330, 340, 350 may be configured to transmit image information to the eye using various levels of wavefront curvature or beam divergence. Each waveguide level may be associated with a particular depth plane and configured to output image information corresponding to that depth plane. Image injection devices 360, 370, 380, 390, 400 may act as light sources for the waveguides and may be utilized to inject image information into the waveguides 270, 280, 290, 300, 310, each configured to disperse incident light across each individual waveguide for output toward the eye 210, as described herein. Light exits output surfaces 410, 420, 430, 440, 450 of image injection devices 360, 370, 380, 390, 400 and is injected into corresponding input surfaces 460, 470, 480, 490, 500 of waveguides 270, 280, 290, 300, 310. In some embodiments, each input surface 460, 470, 480, 490, 500 may be an edge of the corresponding waveguide or a portion of a major surface of the corresponding waveguide (i.e., one of the waveguide surfaces that directly faces the world 510 or the viewer's eye 210). In some embodiments, a single beam of light (e.g., a collimated beam) may be injected into each waveguide, outputting an entire field of cloned collimated beams that are directed toward the eye 210 at a particular angle (and divergence) corresponding to the depth plane associated with the particular waveguide. In some embodiments, a single one of the image injection devices 360, 370, 380, 390, 400 may be associated with and inject light into multiple (eg, three) waveguides 270, 280, 290, 300, 310.

[0158] In some embodiments, each of the image input devices 360, 370, 380, 390, 400 is a discrete display that generates image information for input into a corresponding waveguide 270, 280, 290, 300, 310. In some other embodiments, the image input devices 360, 370, 380, 390, 400 are the output of a single multiplexed display that may, for example, send image information to each of the image input devices 360, 370, 380, 390, 400 via one or more optical conduits (such as fiber optic cables). It should be understood that the image information provided by the image input devices 360, 370, 380, 390, 400 may include light of different wavelengths or colors (e.g., different primary colors, as discussed herein).

[0159] In some embodiments, light injected into the waveguides 270, 280, 290, 300, 310 is provided by an optical projection system 520, which includes an optical module 530, which may include a light emitter such as a light-emitting diode (LED). Light from the optical module 530 may be directed and modified by an optical modulator 540, e.g., a spatial light modulator, via a beam splitter 550. The optical modulator 540 may be configured to vary the perceived intensity of the light injected into the waveguides 270, 280, 290, 300, 310 and encode the light with image information. Examples of spatial light modulators include liquid crystal displays (LCDs), including liquid crystal on silicon (LCOS) displays. In some other embodiments, the spatial light modulator may be a MEMS device, such as a digital light processing (DLP) device. It should be understood that image injection devices 360, 370, 380, 390, 400 are illustrated diagrammatically, and in some embodiments, these image injection devices may represent different light paths and locations within a common projection system configured to output light into associated ones of waveguides 270, 280, 290, 300, 310. In some embodiments, the waveguides of waveguide assembly 260 may function as ideal lenses, relaying light injected into the waveguides to the user's eye. In this concept, the object may be a spatial light modulator 540, and the image may be an image on a depth plane.

[0160] In some embodiments, the display system 250 may be a scanning fiber display comprising one or more scanning fibers configured to project light in various patterns (e.g., raster scan, spiral scan, Lissajous pattern, etc.) into one or more waveguides 270, 280, 290, 300, 310 and ultimately to the viewer's eye 210. In some embodiments, the illustrated image injection devices 360, 370, 380, 390, 400 may diagrammatically represent a single scanning fiber or a bundle of scanning fibers configured to inject light into one or more waveguides 270, 280, 290, 300, 310. In some other embodiments, the illustrated image injection devices 360, 370, 380, 390, 400 may diagrammatically represent multiple scanning fibers or multiple bundles of scanning fibers, each configured to inject light into an associated one of the waveguides 270, 280, 290, 300, 310. It should be understood that one or more optical fibers may be configured to transmit light from the optical module 530 to one or more waveguides 270, 280, 290, 300, 310. It should be understood that one or more intervening optical structures may be provided between the scanning fiber or fibers and one or more waveguides 270, 280, 290, 300, 310, for example, to redirect light exiting the scanning fiber into one or more waveguides 270, 280, 290, 300, 310.

[0161] Controller 560 controls the operation of one or more of stacked waveguide assemblies 260, including the operation of image input devices 360, 370, 380, 390, 400, light source 530, and light module 540. In some embodiments, controller 560 is part of local data processing module 140. Controller 560 includes programming (e.g., instructions in a non-transitory medium) that coordinates the timing and provision of image information to waveguides 270, 280, 290, 300, 310, for example, according to any of the various schemes disclosed herein. In some embodiments, the controller may be a single integrated device or a distributed system connected by wired or wireless communication channels. Controller 560 may, in some embodiments, be part of processing module 140 or 150 (FIG. 9E).

[0162] Continuing with reference to FIG. 6 , the waveguides 270, 280, 290, 300, and 310 may be configured to propagate light within each individual waveguide by total internal reflection (TIR). Each of the waveguides 270, 280, 290, 300, and 310 may be planar or have another shape (e.g., curved) with major top and bottom surfaces and edges extending between the major top and bottom surfaces. In the illustrated configuration, the waveguides 270, 280, 290, 300, and 310 may each include outcoupling optical elements 570, 580, 590, 600, and 610 configured to extract light from the waveguide by redirecting light propagating within each individual waveguide out of the waveguide and outputting image information to the eye 210. The extracted light may also be referred to as outcoupling light, and the outcoupling optical elements may also be referred to as light extraction optical elements. The extracted beam of light may be output by the waveguide at a location where light propagating within the waveguide strikes the light extraction optical element. The outcoupling optical element 570, 580, 590, 600, 610 may be, for example, a grating including diffractive optical features as discussed further herein. While shown disposed on the bottom major surface of the waveguides 270, 280, 290, 300, 310 for ease of explanation and clarity of drawing, in some embodiments, the outcoupling optical element 570, 580, 590, 600, 610 may be disposed on the top and / or bottom major surfaces and / or directly within the volume of the waveguides 270, 280, 290, 300, 310, as discussed further herein. In some embodiments, the outcoupling optical elements 570, 580, 590, 600, 610 may be formed within a layer of material that is attached to a transparent substrate and forms the waveguides 270, 280, 290, 300, 310. In some other embodiments, the waveguides 270, 280, 290, 300, 310 may be a monolithic piece of material, and the outcoupling optical elements 570, 580, 590, 600, 610 may be formed on and / or within that piece of material.

[0163] Continuing with reference to FIG. 6 , as discussed herein, each waveguide 270, 280, 290, 300, 310 is configured to output light and form an image corresponding to a particular depth plane. For example, the waveguide 270 closest to the eye may be configured to deliver collimated light (injected into such waveguide 270) to the eye 210. The collimated light may represent an optical infinity focal plane. The next upper waveguide 280 may be configured to send collimated light that passes through a first lens 350 (e.g., a negative lens) before reaching the eye 210. Such first lens 350 may be configured to generate a slight convex wavefront curvature so that the eye / brain interprets light emerging from the next upper waveguide 280 as emerging from a first focal plane closer inward from optical infinity toward the eye 210. Similarly, the third upper waveguide 290 passes its output light through both the first 350 and second 340 lenses before reaching the eye 210. The combined refractive power of the first 350 and second 340 lenses may be configured to produce another, increasing amount of wavefront curvature such that the eye / brain interprets the light emerging from the third waveguide 290 as originating from a second focal plane that is closer inward from optical infinity towards the person than was the light from the next upper waveguide 280.

[0164] The other waveguide layers 300, 310 and lenses 330, 320 are similarly configured, with the highest waveguide 310 in the stack sending its output through all of the lenses between it and the eye for a collective focal power representing the focal plane closest to the person. To compensate for the stack of lenses 320, 330, 340, 350 when viewing / interpreting light originating from the world 510 on the other side of the stacked waveguide assembly 260, a compensating lens layer 620 may be placed on top of the stack to compensate for the collective power of the lower lens stacks 320, 330, 340, 350. Such a configuration provides as many perceived focal planes as there are available waveguide / lens pairs. Both the waveguide outcoupling optical elements and the focusing sides of the lenses may be static (i.e., not dynamic or electro-active). In some alternative embodiments, one or both may be dynamic using electro-active features.

[0165] In some embodiments, two or more of the waveguides 270, 280, 290, 300, 310 may have the same associated depth plane. For example, multiple waveguides 270, 280, 290, 300, 310 may be configured to output images set at the same depth plane, or multiple subsets of waveguides 270, 280, 290, 300, 310 may be configured to output images set at the same depth planes, with one set per depth plane. This may provide the advantage of forming tiled images to provide an extended field of view at those depth planes.

[0166] Continuing with reference to FIG. 6 , the outcoupling optical elements 570, 580, 590, 600, 610 may be configured to redirect light from their respective waveguides and output the light with an appropriate amount of divergence or collimation for the particular depth plane associated with the waveguide. As a result, waveguides with different associated depth planes may have different configurations of the outcoupling optical elements 570, 580, 590, 600, 610, which output light with different amounts of divergence depending on the associated depth plane. In some embodiments, the light-extracting optical elements 570, 580, 590, 600, 610 may be volume or surface features that can be configured to output light at specific angles. For example, the light-extracting optical elements 570, 580, 590, 600, 610 may be volume holograms, surface holograms, and / or diffraction gratings. In some embodiments, features 320, 330, 340, 350 may not be lenses. Rather, they may simply be spacers (e.g., cladding layers and / or structures to form air gaps).

[0167] In some embodiments, the outcoupling optical elements 570, 580, 590, 600, 610 are diffractive features or "diffractive optical elements" (also referred to herein as "DOEs") that form a diffraction pattern. Preferably, the DOEs have a sufficiently low diffraction efficiency so that only a portion of the light in the beam is deflected toward the eye 210 at each intersection of the DOE, while the remainder continues traveling through the waveguide via TIR. The light carrying the image information is thus split into several related output beams that exit the waveguide at various locations, resulting in a very uniform pattern of output emission toward the eye 210 for this particular collimated beam bouncing within the waveguide.

[0168] In some embodiments, one or more DOEs may be switchable between an "on" state in which they actively diffract and an "off" state in which they do not significantly diffract. For example, a switchable DOE may comprise a layer of polymer-dispersed liquid crystal in which microdroplets comprise a diffractive pattern in a host medium, and the refractive index of the microdroplets may be switched to substantially match the refractive index of the host material (in which case the pattern does not significantly diffract incident light), or the microdroplets may be switched to a refractive index that does not match that of the host medium (in which case the pattern actively diffracts incident light).

[0169] In some embodiments, a camera assembly 630 (e.g., a digital camera, including visible and infrared light cameras) may be provided to capture images of the eye 210 and / or tissue surrounding the eye 210, for example, to detect user input and / or monitor the physiological state of the user. As used herein, a camera may be any image capture device. In some embodiments, the camera assembly 630 may include an image capture device and a light source that projects light (e.g., infrared light) onto the eye, which may then be reflected by the eye and detected by the image capture device. In some embodiments, the camera assembly 630 may be mounted to a frame or support structure 80 ( FIG. 9E ) and may be in electrical communication with processing modules 140 and / or 150, which may process image information from the camera assembly 630. In some embodiments, one camera assembly 630 may be utilized per eye, monitoring each eye separately.

[0170] Camera assembly 630, in some embodiments, may observe user movements, such as the user's eye movements. As an example, camera assembly 630 may capture images of eye 210 and determine the size, position, and / or orientation of the eye's pupil (or some other structure of eye 210). Camera assembly 630 may, if desired, obtain images (processed by processing circuitry of the type described herein) that are used to determine the direction the user is looking (e.g., eye pose or gaze direction). In some embodiments, camera assembly 630 may include multiple cameras, at least one of which may be utilized for each eye and independently determine the eye pose or gaze direction of each eye separately. Camera assembly 630, in some embodiments, in combination with processing circuitry such as controller 560 or local data processing module 140, may determine eye pose or gaze direction based on flashes (e.g., reflections) of light (e.g., infrared light) reflected from a light source contained within camera assembly 630.

[0171] 7, an example of an output beam output by a waveguide is shown. While one waveguide is illustrated, it should be understood that other waveguides in waveguide assembly 260 (FIG. 6) may function similarly, and that waveguide assembly 260 includes multiple waveguides. Light 640 is launched into waveguide 270 at input surface 460 of waveguide 270 and propagates within waveguide 270 by TIR. At the point where light 640 impinges on DOE 570, a portion of the light exits the waveguide as output beam 650. Output beam 650 is illustrated as being approximately parallel, but may be redirected to propagate to eye 210 at an angle (e.g., forming a diverging output beam), as discussed herein and depending on the depth plane associated with waveguide 270. It should be understood that a nearly collimated exit beam may refer to a waveguide with outcoupling optics that outcouples light to form an image that appears to be set at a depth plane at a long distance (e.g., optical infinity) from the eye 210. Other waveguides or other sets of outcoupling optics may output a more divergent exit beam pattern, which would require the eye 210 to accommodate to a closer distance to focus on the retina and would be interpreted by the brain as light from a distance closer to the eye 210 than optical infinity.

[0172] In some embodiments, a full-color image may be formed at each depth plane by overlaying an image in each of the primary colors, for example, three or more primary colors. FIG. 8 illustrates an example of a stacked waveguide assembly, with each depth plane including an image formed using multiple different primary colors. The illustrated embodiment shows depth planes 240a-240f, but more or fewer depths are also contemplated. Each depth plane may have three or more primary color images associated with it, including a first image in a first color G, a second image in a second color R, and a third image in a third color B. Different depth planes are indicated in the diagram by different numbers for diopters (dpt) following the letters G, R, and B. By way of example only, the number following each of these letters indicates the diopter (1 / m), i.e., the inverse distance of the depth plane from the viewer, and each box in the diagram represents an individual primary color image. In some embodiments, the exact locations of the depth planes for different primary colors may be varied to account for differences in the eye's focusing of light of different wavelengths. For example, different primary color images for a given depth plane may be placed on depth planes corresponding to different distances from the user. Such an arrangement may increase visual acuity and user comfort and / or reduce chromatic aberrations.

[0173] In some embodiments, light for each primary color may be output by a single dedicated waveguide, such that each depth plane may have multiple waveguides associated with it. In such embodiments, each box in the diagram containing the letter G, R, or B may be understood to represent an individual waveguide, and three waveguides may be provided per depth plane, resulting in three primary color images per depth plane. While the waveguides associated with each depth plane are shown adjacent to each other in this drawing for ease of illustration, it should be understood that in a physical device, the waveguides may all be arranged in a stack with one waveguide per level. In some other embodiments, multiple primary colors may be output by the same waveguide, such that, for example, only a single waveguide may be provided per depth plane.

[0174] 8, in some embodiments, G is green, R is red, and B is blue. In some other embodiments, other colors associated with other wavelengths of light, including magenta and cyan, may be used in addition to or replace one or more of red, green, or blue.

[0175] It should be understood that references throughout this disclosure to a given color of light are understood to encompass light of one or more wavelengths within the range of wavelengths of light that are perceived by a viewer as that given color. For example, red light may include one or more wavelengths of light that are within the range of about 620-780 nm, green light may include one or more wavelengths of light that are within the range of about 492-577 nm, and blue light may include one or more wavelengths of light that are within the range of about 435-493 nm.

[0176] In some embodiments, light source 530 (FIG. 6) may be configured to emit light at one or more wavelengths outside the range of a viewer's visual perception, e.g., infrared and / or ultraviolet wavelengths. Additionally, the waveguide incoupling, outcoupling, and other light redirecting structures of display 250 may be configured to direct and emit this light from the display toward the user's eye 210, e.g., for imaging and / or user stimulation applications.

[0177] Referring now to FIG. 9A , in some embodiments, light impinging on a waveguide may need to be redirected to incoupling the light into the waveguide. An incoupling optical element may be used to redirect and incoupling the light into its corresponding waveguide. FIG. 9A illustrates a cross-sectional side view of an example of a plurality or set 660 of stacked waveguides, each including an incoupling optical element. The waveguides may each be configured to output light of one or more different wavelengths or one or more different wavelength ranges. Stack 660 may correspond to stack 260 ( FIG. 6 ), and the illustrated waveguides of stack 660 may correspond to a portion of the plurality of waveguides 270, 280, 290, 300, 310, although it should be understood that light from one or more of image injection devices 360, 370, 380, 390, 400 is injected into the waveguide from a location requiring the light to be redirected for incoupling.

[0178] The illustrated set 660 of stacked waveguides includes waveguides 670, 680, and 690. Each waveguide includes an associated internal coupling optical element (which may also be referred to as the light input area on the waveguide), for example, internal coupling optical element 700 is disposed on a major surface (e.g., the top major surface) of waveguide 670, internal coupling optical element 710 is disposed on a major surface (e.g., the top major surface) of waveguide 680, and internal coupling optical element 720 is disposed on a major surface (e.g., the top major surface) of waveguide 690. In some embodiments, one or more of internal coupling optical elements 700, 710, 720 may be disposed on the bottom major surface of an individual waveguide 670, 680, 690 (particularly, one or more internal coupling optical elements are reflective polarizing optical elements). As shown, the internal coupling optical elements 700, 710, 720 may be disposed on the upper major surface of the respective waveguide 670, 680, 690 (or on top of the next lower waveguide), and in particular, the internal coupling optical elements are transmissive turning optical elements. In some embodiments, the internal coupling optical elements 700, 710, 720 may be disposed within the body of the respective waveguide 670, 680, 690. In some embodiments, as discussed herein, the internal coupling optical elements 700, 710, 720 are wavelength selective, such that they selectively redirect one or more wavelengths of light while transmitting other wavelengths of light. While illustrated on one side or corner of the respective waveguide 670, 680, 690, it should be understood that the internal coupling optical elements 700, 710, 720 may be disposed within other areas of the respective waveguide 670, 680, 690 in some embodiments.

[0179] As shown, the in-coupling optical elements 700, 710, 720 may be laterally offset from one another, as seen in the illustrated head-on view, in the direction of light propagating through these in-coupling optical elements. In some embodiments, each in-coupling optical element may be offset to receive light without that light passing through another in-coupling optical element. For example, each in-coupling optical element 700, 710, 720 may be configured to receive light from different image input devices 360, 370, 380, 390, and 400, as shown in FIG. 6 , and may be separated (e.g., laterally spaced) from the other in-coupling optical elements 700, 710, 720 so as to receive substantially no light from others of the in-coupling optical elements 700, 710, 720.

[0180] Each waveguide also includes an associated optically dispersive element, for example, optically dispersive element 730 is disposed on a major surface (e.g., the top major surface) of waveguide 670, optically dispersive element 740 is disposed on a major surface (e.g., the top major surface) of waveguide 680, and optically dispersive element 750 is disposed on a major surface (e.g., the top major surface) of waveguide 690. In some other embodiments, optically dispersive elements 730, 740, 750 may be disposed on the bottom major surfaces of associated waveguides 670, 680, 690, respectively. In some other embodiments, optically dispersive elements 730, 740, 750 may be disposed on both the top and bottom major surfaces of associated waveguides 670, 680, 690, respectively, or optically dispersive elements 730, 740, 750 may be disposed on different ones of the top and bottom major surfaces in different associated waveguides 670, 680, 690, respectively.

[0181] Waveguides 670, 680, 690 may be spaced apart and separated, for example, by gas, liquid, and / or solid layers of material. For example, as shown, layer 760a may separate waveguides 670 and 680, and layer 760b may separate waveguides 680 and 690. In some embodiments, layers 760a and 760b are formed from a low refractive index material (i.e., a material having a lower refractive index than the material forming the immediately adjacent ones of waveguides 670, 680, 690). Preferably, the refractive index of the material forming layers 760a, 760b is 0.05 or more or 0.10 or less than the refractive index of the material forming waveguides 670, 680, 690. Advantageously, the lower refractive index layers 760a, 760b may function as cladding layers that promote total internal reflection (TIR) ​​of light through the waveguides 670, 680, 690 (e.g., TIR between the top and bottom major surfaces of each waveguide). In some embodiments, the layers 760a, 760b are formed from air. Although not shown, it should be understood that the top and bottom of the illustrated set of waveguides 660 may include immediate cladding layers.

[0182] Preferably, for ease of manufacturing and other considerations, the materials forming waveguides 670, 680, 690 are similar or the same, and the materials forming layers 760a, 760b are similar or the same. In some embodiments, the materials forming waveguides 670, 680, 690 may differ between one or more waveguides, and / or the materials forming layers 760a, 760b may differ while still maintaining the various refractive index relationships discussed above.

[0183] 9A, light rays 770, 780, 790 enter the set of waveguides 660. It should be understood that light rays 770, 780, 790 may be injected into the waveguides 670, 680, 690 by one or more image injection devices 360, 370, 380, 390, 400 (FIG. 6).

[0184] In some embodiments, light rays 770, 780, 790 have different properties, e.g., different wavelengths or different wavelength ranges, which may correspond to different colors. Each of the incoupling optical elements 700, 710, 720 deflects incident light such that the light propagates through a respective one of the waveguides 670, 680, 690 by TIR. In some embodiments, each of the incoupling optical elements 700, 710, 720 selectively deflects one or more particular wavelengths of light while transmitting other wavelengths to the underlying waveguide and associated incoupling optical element.

[0185] For example, in-coupling optical element 700 may be configured to deflect light ray 770 having a first wavelength or wavelength range while transmitting light rays 780 and 790 having different second and third wavelengths or wavelength ranges, respectively. Transmitted light ray 780 impinges on and is deflected by in-coupling optical element 710, which is configured to deflect light of the second wavelength or wavelength range. Light ray 790 is deflected by in-coupling optical element 720, which is configured to selectively deflect light of the third wavelength or wavelength range.

[0186] 9A , the deflected light rays 770, 780, 790 are deflected to propagate through the corresponding waveguides 670, 680, 690. That is, the in-coupling optical element 700, 710, 720 of each waveguide deflects the light into its corresponding waveguide 670, 680, 690, in-coupling the light into the corresponding waveguide. The light rays 770, 780, 790 are deflected at an angle that causes the light to propagate through the respective waveguides 670, 680, 690 by TIR. The light rays 770, 780, 790 propagate through the respective waveguides 670, 680, 690 by TIR until they impinge on the waveguide's corresponding optical dispersive element 730, 740, 750.

[0187] 9B, a perspective view of the multiple stacked waveguide embodiment of FIG. 9A is illustrated. As described above, in-coupled light rays 770, 780, 790 are deflected by in-coupling optical elements 700, 710, 720, respectively, and then propagate by TIR within waveguides 670, 680, 690, respectively. Light rays 770, 780, 790 then impinge on optically dispersive elements 730, 740, 750, respectively. Optically dispersive elements 730, 740, 750 deflect light rays 770, 780, 790 to propagate toward out-coupling optical elements 800, 810, 820, respectively.

[0188] In some embodiments, the optically dispersive elements 730, 740, 750 are orthogonal pupil expanders (OPEs). In some embodiments, the OPEs deflect or disperse light into the out-coupling optical elements 800, 810, 820, and in some embodiments, may also increase the beam or spot size of this light as it propagates into the out-coupling optical elements. In some embodiments, the optically dispersive elements 730, 740, 750 may be omitted, and the in-coupling optical elements 700, 710, 720 may be configured to deflect light directly into the out-coupling optical elements 800, 810, 820. For example, with reference to FIG. 9A , the optically dispersive elements 730, 740, 750 may be replaced with the out-coupling optical elements 800, 810, 820, respectively. In some embodiments, the outcoupling optical elements 800, 810, 820 are exit pupils (EPs) or exit pupil expanders (EPEs) that direct light toward the viewer's eye 210 ( FIG. 7 ). It should be understood that an OPE may be configured to increase the size of the eyebox in at least one axis, and that the EPE may increase the eyebox in an axis that intersects the axis of the OPE, e.g., orthogonal to the axis of the OPE. For example, each OPE may be configured to redirect a portion of the light striking the OPE to an EPE of the same waveguide, while allowing the remaining portion of the light to continue propagating down the waveguide. Upon striking the OPE, again, another portion of the remaining light is redirected to the EPE, and the remainder of that portion continues to propagate further down the waveguide, etc. Similarly, upon striking the EPE, a portion of the impinging light is directed out of the waveguide toward the user, and the remaining portion of that light continues to propagate through the waveguide until it again strikes an EP, at which point another portion of the impinging light is directed out of the waveguide, etc. As a result, a single beam of internally coupled light may be "replicated" each time a portion of that light is redirected by an OPE or EPE, thereby forming a cloned beam field of light, as shown in Figure 6. In some embodiments, the OPE and / or EPE may be configured to modify the size of the beam of light.

[0189] 9A and 9B, in some embodiments, a waveguide set 660 includes, for each primary color, waveguides 670, 680, 690, in-coupling optical elements 700, 710, 720, optically dispersive elements (e.g., OPEs) 730, 740, 750, and out-coupling optical elements (e.g., EPs) 800, 810, 820. The waveguides 670, 680, 690 may be stacked with an air gap / cladding layer between each one. The in-coupling optical elements 700, 710, 720 redirect or deflect incident light into that waveguide (with different in-coupling optical elements receiving light of different wavelengths). The light then propagates at an angle that will result in TIR within the individual waveguides 670, 680, 690. In the example shown, light ray 770 (e.g., blue light) is deflected by the first in-coupling optical element 700 in the manner described above, then continues bouncing down the waveguide, interacting with the optically dispersive element (e.g., OPE) 730 and then the out-coupling optical element (e.g., EP) 800. Light rays 780 and 790 (e.g., green and red light, respectively) pass through the waveguide 670, with light ray 780 impinging on and being deflected by the in-coupling optical element 710. Light ray 780 will then, via TIR, bounce down the waveguide 680, to its optically dispersive element (e.g., OPE) 740 and then the out-coupling optical element (e.g., EP) 810. Finally, light ray 790 (e.g., red light) passes through the waveguide 690 and impinges on the optically in-coupling optical element 720 of the waveguide 690. The light in-coupling optical element 720 deflects the light ray 790 so that it propagates by TIR to the light dispersive element (e.g., OPE) 750 and then by TIR to the out-coupling optical element (e.g., EP) 820. The out-coupling optical element 820 then finally out-couples the light ray 790 to a viewer, who also receives out-coupled light from the other waveguides 670, 680.

[0190] FIG. 9C illustrates a top-down plan view of the multiple stacked waveguide embodiment of FIGS. 9A and 9B. It should be understood that this top-down view may also be referred to as a head-on view, as viewed in the direction of light propagation toward the in-coupling optical elements 800, 810, 820; i.e., the top-down view is a view of the waveguides with image light incident normal to the page. As shown, the waveguides 670, 680, 690 may be vertically aligned, along with each waveguide's associated optically dispersive element 730, 740, 750 and associated out-coupling optical elements 800, 810, 820. However, as discussed herein, the in-coupling optical elements 700, 710, 720 are not vertically aligned. Rather, the in-coupling optical elements are preferably non-overlapping (e.g., laterally spaced apart, as seen in the top-down view). As discussed further herein, this non-overlapping spatial arrangement facilitates the injection of light from different sources into different waveguides on a one-to-one basis, thereby allowing a specific light source to be uniquely coupled to a specific waveguide. In some embodiments, arrangements including non-overlapping, spatially separated in-coupling optical elements may be referred to as shifted pupil systems, and the in-coupling optical elements in these arrangements may correspond to sub-pupils.

[0191] It should be understood that spatially overlapping areas may have lateral overlap of 70% or more, 80% or more, or 90% or more of their areas, as seen in the top-bottom views. On the other hand, laterally shifted areas may have less than 30% overlap, less than 20% overlap, or less than 10% overlap, as seen in the top-bottom views. In some embodiments, the laterally shifted areas have no overlap.

[0192] FIG. 9D illustrates a top-down plan view of another example of multiple stacked waveguides. As shown, the waveguides 670, 680, and 690 may be vertically aligned. However, compared to the configuration of FIG. 9C , the separate optically dispersive elements 730, 740, and 750 and associated outcoupling optical elements 800, 810, and 820 are omitted. Instead, the optically dispersive elements and outcoupling optical elements are effectively superimposed, occupying the same area, as seen in the top-down view. In some embodiments, an optically dispersive element (e.g., an OPE) may be disposed on one major surface of the waveguides 670, 680, and 690, and an outcoupling optical element (e.g., an EPE) may be disposed on the other major surface of those waveguides. Thus, each waveguide 670, 680, and 690 may have superimposed optically dispersive and outcoupling optical elements, collectively referred to as combined OPE / EPEs 1281, 1282, and 1283, respectively. Further details regarding such combined OPE / EPEs can be found in U.S. Patent Application No. 16 / 221,359, filed December 14, 2018, the entire disclosure of which is incorporated herein by reference. Internal coupling optical elements 700, 710, and 720 internally couple and direct light to combined OPE / EPEs 1281, 1282, and 1283, respectively. In some embodiments, as shown, internal coupling optical elements 700, 710, and 720 may be laterally offset (e.g., they are laterally spaced apart, as seen in the top and bottom views shown) when having an offset pupil spatial arrangement. Similar to the configuration of FIG. 9C , this laterally offset spatial arrangement facilitates the injection of light of different wavelengths into different waveguides (e.g., from different light sources) on a one-to-one basis.

[0193] 9E illustrates an example of a wearable display system 60 into which the various waveguide and associated systems disclosed herein may be integrated. In some embodiments, the display system 60 is the system 250 of FIG. 6, which diagrammatically illustrates some portions of the system 60 in greater detail. For example, the waveguide assembly 260 of FIG. 6 may be part of the display 70.

[0194] Continuing with reference to FIG. 9E , display system 60 includes display 70 and various mechanical and electronic modules and systems to support the functionality of display 70. Display 70 may be coupled to frame 80, which is wearable by a display system user or viewer 90 and configured to position display 70 directly in front of the user's 90's eyes. Display 70, in some embodiments, may be considered eyewear. Display 70 may include one or more waveguides, such as waveguide 270, configured to relay internally coupled image light and output the image light to the user's 90's eyes. In some embodiments, speaker 100 is coupled to frame 80 and configured to be positioned adjacent to the user's 90's ear canal (in some embodiments, another speaker, not shown, may optionally be positioned adjacent the user's other ear canal to provide stereo / shapeable sound control). Display system 60 may also include one or more microphones 110 or other devices to detect sound. In some embodiments, the microphone may be configured to allow a user to provide input or commands to the system 60 (e.g., selecting voice menu commands, natural language questions, etc.) and / or to enable audio communication with other persons (e.g., other users of similar display systems). The microphone may also be configured as an ambient sensor to collect audio data (e.g., sounds from the user and / or the environment). In some embodiments, the display system 60 may further include one or more outwardly directed environmental sensors 112 configured to detect objects, stimuli, people, animals, places, or other aspects of the world around the user. For example, the environmental sensors 112 may include one or more cameras, which may be positioned, for example, facing outward, to capture images similar to at least a portion of the user 90's normal field of view. In some embodiments, the display system may also include an ambient sensor 120a, which may be separate from the frame 80 and mounted on the user 90's body (e.g., the user's 90's head, torso, limbs, etc.).The ambient sensor 120a, in some embodiments, may be configured to obtain data characterizing a physiological state of the user 90. For example, the sensor 120a may be an electrode.

[0195] 9E , display 70 is operably coupled to local data processing module 140 by a communication link 130, such as wired or wireless connectivity, which may be mounted in a variety of configurations, such as fixedly attached to frame 80, fixedly attached to a helmet or hat worn by the user, integrated into headphones, or otherwise removably attached to user 90 (e.g., in a backpack-style configuration, in a belt-linked configuration). Similarly, sensor 120a may be operably coupled to local data processing module 140 by a communication link 120b, such as wired or wireless connectivity. Local processing and data module 140 may comprise a hardware processor and digital memory, such as non-volatile memory (e.g., flash memory or a hard disk drive), both of which may be utilized to aid in processing, caching, and storing data. Optionally, local processing and data module 140 may include one or more central processing units (CPUs), graphics processing units (GPUs), dedicated processing hardware, etc. The data may include a) data captured from sensors (such as image capture devices (such as cameras), microphones, inertial measurement units, accelerometers, compasses, GPS units, wireless devices, gyroscopes, and / or other sensors disclosed herein (e.g., which may be operatively coupled to frame 80 or otherwise attached to user 90)) and / or b) data obtained and / or processed using remote processing module 150 and / or remote data repository 160 (including data related to virtual content), possibly for processing or retrieval and subsequent passage to display 70. Local processing and data module 140 may be operatively coupled to remote processing module 150 and remote data repository 160 by communication links 170, 180, such as via wired or wireless communication links, such that these remote modules 150, 160 are operatively coupled to each other and available as resources to local processing and data module 140.In some embodiments, local processing and data module 140 may include one or more of an image capture device, a microphone, an inertial measurement unit, an accelerometer, a compass, a GPS unit, a wireless device, and / or a gyroscope. In some other embodiments, one or more of these sensors may be mounted to frame 80 or may be a freestanding structure that communicates with local processing and data module 140 by a wired or wireless communication path.

[0196] 9E , in some embodiments, remote processing module 150 may comprise one or more processors configured to analyze and process data and / or image information, including, for example, one or more central processing units (CPUs), graphics processing units (GPUs), special-purpose processing hardware, etc. In some embodiments, remote data repository 160 may comprise a digital data storage facility, which may be available through the Internet or other networking configuration in a “cloud” resource configuration. In some embodiments, remote data repository 160 may include one or more remote servers, which provide information, e.g., information for generating virtual content, to local processing and data module 140 and / or remote processing module 150. In some embodiments, all data is stored and all computations are performed in the local processing and data module, allowing for fully autonomous use from the remote module. Optionally, an external system (e.g., one or more processors, one or more computer systems), including a CPU, GPU, etc., may perform at least a portion of the processing (e.g., generating image information, processing data) and provide information to and receive information from modules 140, 150, 160, e.g., via a wireless or wired connection.

[0197] FIG. 10 illustrates an example of a wearable display system with a light projection system 910 having a spatial light modulator 930 and a separate light source 940. The light source 940 may comprise one or more light emitters and illuminate the spatial light modulator (SLM) 930. A lens structure 960 may be used to focus light from the light source 940 onto the SLM 930. A beam splitter (e.g., a polarizing beam splitter (PBS)) 950 reflects the light from the light source 940 onto the spatial light modulator 930, which reflects and modulates the light. The reflected modulated light, also referred to as image light, then propagates through the beam splitter 950 to the eyepiece 920. Another lens structure, projection optics 970, may be utilized to converge or focus the image light onto the eyepiece 920. The eyepiece 920 may include one or more waveguides or waveguides that relay the modulated light to the eye 210.

[0198] As described herein, a separate light source 940 and associated lens structure 960 may undesirably add weight and size to a wearable display system, which may reduce the comfort of the display system, especially for a user wearing the display system for extended periods of time.

[0199] Additionally, light source 940, in conjunction with SLM 930, may consume energy inefficiently. For example, light source 940 may illuminate the entire SLM 930. SLM 930 then selectively reflects the light toward eyepiece 920. Thus, not all of the light produced by light source 940 may be used to form an image. Some of this light, for example, light corresponding to dark areas of the image, is not reflected back to eyepiece 920. As a result, light source 940 utilizes energy to generate light to illuminate the entire SLM 930, but only a certain percentage of this light may be needed to form some images.

[0200] Furthermore, as described herein, in some cases, the SLM 930 may modulate light and selectively reflect incident light using micromirrors or liquid crystal molecules that modify the amount of light reflected from an underlying mirror. As a result, such devices require physical movement of optical elements (e.g., micromirrors or liquid crystal molecules, such as in an LCoS or DLP panel, respectively) to modulate light from the light source 940. The physical movement required to modulate light and encode it with image information, e.g., corresponding to a pixel, may occur relatively slowly compared to, for example, the ability to turn an LED or OLED “on” or “off.” This relatively slow movement may limit the frame rate of the display system and may be visible, for example, as motion blur, color breakup, and / or a presented image that is misaligned with the user's head pose or changes in that pose.

[0201] Advantageously, wearable displays utilizing emissive microdisplays as disclosed herein may facilitate wearable display systems with relatively low weight and bulk, high energy efficiency, and high frame rates, with low motion blur and low latency from motion to image rendering. Low blur and low latency from motion to image rendering are further discussed in U.S. Provisional Application No. 62 / 786,199, filed December 28, 2018, the entire disclosure of which is incorporated herein by reference. Additionally, compared to scanning fiber displays, emissive microdisplays may avoid artifacts caused by the use of coherent light sources.

[0202] 11A, an example of a wearable display system is illustrated with an optical projection system 1010 having multiple emissive microdisplays 1030a, 1030b, 1030c. Light from the microdisplays 1030a, 1030b, 1030c is combined by an optical combiner 1050 and directed toward an eyepiece 1020, which relays the light to the user's eye 210. Projection optics 1070 may be provided between the optical combiner 1050 and the eyepiece 1020. In some embodiments, the eyepiece 1020 may be a waveguide assembly comprising one or more waveguides. In some embodiments, the optical projection system 1010 and the eyepiece 1020 may be supported (e.g., mounted) on a frame 80 (FIG. 9E).

[0203] In some embodiments, the microdisplays 1030a, 1030b, 1030c may be monochrome microdisplays, each outputting light of a different primary color to provide a monochrome image that is combined to form a full-color image, as discussed herein.

[0204] In some other embodiments, microdisplays 1030a, 1030b, and 1030c may each be a full-color display configured to output light of all primary colors. For example, microdisplays 1030a, 1030b, and 1030c each include red, green, and blue light emitters. Microdisplays 1030a, 1030b, and 1030c may be identical and display the same image. However, utilizing multiple microdisplays may provide advantages for increasing the brightness and brightness dynamic range of the image by combining light from the multiple microdisplays to form a single image. In some embodiments, two or more (e.g., three) microdisplays may be utilized, and optical combiner 1050 is configured to combine light from all of these microdisplays.

[0205] The microdisplay may comprise an array of light emitters. Examples of light emitters include organic light-emitting diodes (OLEDs) and micro light-emitting diodes (microLEDs). It should be understood that OLEDs utilize organic materials to emit light, while microLEDs utilize inorganic materials to emit light. Advantageously, some microLEDs offer higher brightness and higher efficiency (in terms of lux / W) than OLEDs. In some embodiments, the microdisplay is preferably an emissive microLED display.

[0206] 11A , the microdisplays 1030a, 1030b, and 1030c may be configured to emit image light 1032a, 1032b, and 1032c, respectively. If the microdisplays are monochrome microdisplays, the image light 1032a, 1032b, and 1032c may each be a different primary color. The optical combiner 1050 receives the image light 1032a, 1032b, and 1032c and effectively combines the light so that the light generally propagates in the same direction, e.g., toward the projection optics 1070. In some embodiments, the optical combiner 1050 may be a dichroic X-cube prism having reflective interior surfaces that redirect the image light 1032a, 1032b, and 1032c to the projection optics 1070. It should be appreciated that the projection optics 1070 may be a lens structure comprising one or more lenses that converge or focus the image light onto the eyepiece 1020. The eyepiece 1020 then relays the image light 1032a, 1032b, 1032c to the eye 210.

[0207] In some embodiments, the eyepiece 1020 may comprise multiple stacked waveguides 1020a, 1020b, 1020c, each with a separate internal coupling optical element 1022a, 1022b, 1022c. In some embodiments, the number of waveguides is proportional to the number of primary colors provided by the microdisplays 1030a, 1030b, 1030c. For example, if there are three primary colors, the number of waveguides in the eyepiece 1020 may include a set of three waveguides or multiple sets of three waveguides each. In some embodiments, each set may output light with a wavefront divergence corresponding to a particular depth plane, as discussed herein. It should be understood that waveguides 1020a, 1020b, 1020c and in-coupling optical elements 1022a, 1022b, 1022c may correspond to waveguides 670, 680, 690 and in-coupling optical elements 700, 710, 720, respectively, of Figures 9A-9C. As viewed from the projection optics 1070, in-coupling optical elements 1022a, 1022b, 1022c may be offset laterally such that they, at least in part, do not overlap as seen in such views.

[0208] As shown, the various internal coupling optical elements disclosed herein (e.g., internal coupling optical elements 1022a, 1022b, 1022c) may be disposed on a major surface of an associated waveguide (e.g., waveguides 1020a, 1020b, 1020c, respectively). In addition, and as also shown, the major surface on which a given internal coupling optical element is disposed may be the back surface of the waveguide. In such a configuration, the internal coupling optical element may be a reflective light redirecting element, which internally couples light through the associated waveguide by reflecting the light at an angle that supports TIR. In certain other configurations, the internal coupling optical element may be disposed on the front surface of the waveguide (closer to the projection optics 1070 than the back surface). In such a configuration, the internal coupling optical element may be a transmissive light redirecting element, which internally couples light by changing the propagation direction of the light as it is transmitted through the internal coupling optical element. It should be understood that any of the incoupling optical elements disclosed herein may be reflective or transmissive incoupling optical elements.

[0209] 11A , image light 1032a, 1032b, 1032c from different ones of the microdisplays 1030a, 1030b, 1030c may follow different paths to the eyepiece 1020 such that they impinge on different ones of the incoupling optical elements 1022a, 1022b, 1022c. If the image light 1032a, 1032b, 1032c includes light of different primary colors, the associated incoupling optical elements 1022a, 1022b, 1022c may be configured to selectively incoupling light of different wavelengths, for example, as described above with respect to the incoupling optical elements 700, 710, 720 of FIGS. 9A-9C .

[0210] 11A , the optical combiner 1050 may be configured to redirect the image light 1032a, 1032b, 1032c emitted by the microdisplays 1030a, 1030b, 1030c so that the image light propagates along different optical paths to impinge on the appropriate associated one of the internal combining optical elements 1022a, 1022b, 1022c. Thus, the optical combiner 1050 combines the image light 1032a, 1032b, 1032c in the sense that the image light is output from a common face of the optical combiner 1050, but the light exits the optical combiner in slightly different directions. For example, the reflective interior surfaces 1052, 1054 of the X-cube prism may each be angled to direct the image light 1032a, 1032b, 1032c along different paths to the eyepiece 1020. As a result, the image light 1032a, 1032b, 1032c may be incident on different associated ones of the in-coupling optical elements 1022a, 1022b, 1022c. In some embodiments, the microdisplays 1030a, 1030b, 1030c may be appropriately angled relative to the reflective interior surfaces 1052, 1054 of the X-cube prism to provide the desired light path to the in-coupling optical elements 1022a, 1022b, 1022c. For example, the face of one or more of the microdisplays 1030a, 1030b, 1030c may be angled to match the face of the optical combiner 1050 so that image light emitted by the microdisplay is incident at the appropriate angle on the reflective interior surfaces 1052, 1054 and propagates toward the associated in-coupling optical element 1022a, 1022b, or 1022c. It should be understood that in addition to a cube, optical combiner 1050 may take the form of various other polyhedrons. For example, optical combiner 1050 may be in the shape of a right-angled prism having at least two faces rather than a square.

[0211] 11A , in some embodiments, the monochrome microdisplay 1030b directly facing the output surface 1051 may advantageously output green light. It should be understood that the reflective surfaces 1052, 1054 may have optical losses when reflecting light from the microdisplay. Additionally, the human eye is most sensitive to the color green. As a result, the monochrome microdisplay 1030b facing the output surface 1051 preferably outputs green light so that the green light can travel directly through the optical combiner 1050 without having to be reflected to be output from the optical combiner 1050. However, it will be understood that the green monochrome microdisplay may face another surface of the optical combiner 1050 in some other embodiments.

[0212] As discussed herein, the perception of a full-color image by a user can be achieved in some embodiments using time-division multiplexing. For example, different ones of the emissive microdisplays 1030a, 1030b, and 1030c can be activated at different times to generate different primary color images. In such embodiments, the different primary color images forming a single full-color image can be displayed sequentially quickly enough that the human visual system does not perceive the primary color images as being displayed at different times. That is, the different primary color images forming a single full-color image can all be displayed within a duration short enough that the user perceives the primary color images as being presented simultaneously, rather than being separated in time. For example, it should be understood that the human visual system can have a flicker fusion threshold. The flicker fusion threshold can be understood as the duration beyond which the human visual system is unable to distinguish between images presented at different times. Images presented within that duration are fused, or combined, and as a result, can be perceived by the user as being presented simultaneously. Flickering images with a temporal gap between images outside that duration will not combine, and image flicker will be perceptible. In some embodiments, the duration is 1 / 60 seconds or less, which corresponds to a frame rate of 60 Hz or greater. Preferably, image frames for any individual eye are provided to the user at a frame rate equal to or greater than the duration of the user's flicker fusion threshold. For example, the frame rate per left or right eyepiece may be 60 Hz or greater, or 120 Hz or greater, and as a result, the frame rate provided by the optical projection system 1010 may, in some embodiments, be 120 Hz or greater, or 240 Hz or greater.

[0213] It should be appreciated that time division multiplexing may advantageously reduce the computational load on a processor (e.g., a graphics processor) utilized to form the displayed image. In some other embodiments, such as when sufficient computational resources are available, all primary color images forming a full color image may be displayed simultaneously by microdisplays 1030a, 1030b, 1030c.

[0214] As discussed herein, each of the microdisplays 1030a, 1030b, 1030c may include an array of light emitters. Figure 11B illustrates an example of an array 1042 of light emitters 1044. If the associated microdisplay is a monochrome microdisplay, the light emitters 1044 may all be configured to emit light of the same color.

[0215] If the associated microdisplay is a full-color microdisplay, different ones of the light emitters 1044 may be configured to emit light of different colors. In such an embodiment, the light emitters 1044 may be considered subpixels and may be arranged in groups, with each group having at least one light emitter configured to emit light of each primary color. For example, if the primary colors are red, green, and blue, each group may have at least one red subpixel, at least one green subpixel, and at least one blue subpixel.

[0216] Although the light emitters 1044 are shown arranged in a grid pattern for ease of illustration, it will be understood that the light emitters 1044 may have other regularly repeating spatial arrangements. For example, the number of light emitters of different primary colors may vary, the size of the light emitters may vary, the shape of the light emitters and / or the shape created by a group of light emitters may vary, etc.

[0217] 11B , it should be understood that the micro-emitters 1044 emit light. Additionally, manufacturing constraints, such as lithography or other patterning and processing limitations, and / or electrical considerations may limit the proximity with which neighboring light emitters 1044 are spaced. As a result, there may be an area 1045 surrounding a light emitter 1044 where it is impractical to form other light emitters 1044. This area 1045 forms an inter-emitter region between the light emitters 1044. In some embodiments, taking into account the area 1045, the light emitters have a pitch of, for example, less than 10 μm, less than 8 μm, less than 6 μm, or less than 5 μm, and greater than 1 μm, including 1-5 μm, and the emitter size is 2 μm or less, 1.7 μm or less, or 1.3 μm or less. In some embodiments, the emitter size is within a range having an upper size limit as noted above and a lower size limit of 1 μm. In some embodiments, the emitter size to pitch ratio is between 1:1 and 1:5, between 1:2 and 1:4, or between 1:2 and 1:3.

[0218] It should be understood that given some light emitter device architectures and materials, current crowding can reduce emitter efficiency and pixel droop can result in unintended activation of pixels (e.g., due to energy directed to one light emitter leaking to a neighboring light emitter). As a result, a relatively large area 1045 can beneficially reduce current crowding and pixel droop. In some embodiments, the emitter size-to-pitch ratio is preferably 1:2 to 1:4 or 1:2 to 1:3.

[0219] However, it should also be understood that large separations between light emitters (e.g., small light emitter-to-pitch ratios) can undesirably cause visible gaps or dark areas between the light emitters. Even when translated laterally as discussed herein, some gaps may still be visible depending on the size of the original gap, the translation distance, and the number of subframes utilized (and the resulting translation increments). In some embodiments, lens structures such as light collimators may be utilized to effectively fill or partially fill these dark areas. For example, a light-collimating lens may extend over and around the light emitter 1044 such that light from the emitter 1044 completely fills the lens. For example, the light-collimating lens may have a width greater than the light emitter 1044, and in some embodiments, the width of the collimating lens may be approximately equal to the pitch. As a result, the size of the emitter 1044 is effectively increased to extend across the area of ​​the lens, thereby filling some or all of the area 1045. In some other embodiments, the width of the collimating lens may be approximately equal to the distance the projection system is translated per sub-frame, as discussed herein. Lens structures such as optical collimators are further discussed herein (e.g., in FIG. 30A and related discussion).

[0220] As discussed herein, the light emitter 1044 may be an OLED or a micro-LED. It should be understood that an OLED may utilize, for example, a layer of organic material disposed between electrodes to emit light. A micro-LED may utilize inorganic materials, for example, III-V materials such as GaAs, GaN, and / or GaIn, for light emission. An example of a GaN material includes InGaN, which may be used to form a blue or green light emitter in some embodiments. An example of a GaIn material includes AlGaInP, which may be used to form a red light emitter in some embodiments. In some embodiments, the light emitter 1044 may emit light of an initial color, which may be converted to another desired color using phosphor material or quantum dots. For example, the light emitter may emit blue light, which excites phosphor material or quantum dots, which convert the blue wavelength light to green or red wavelengths.

[0221] Referring now to FIG. 12 , another example of a wearable display system is illustrated, which includes a light projection system having multiple emissive microdisplays 1030 a, 1030 b, and 1030 c. The illustrated display system is similar to the display system of FIG. 11A , except that the optical combiner 1050 has a standard X-cube prism configuration and includes light redirecting structures 1080 a and 1080 c to modify the angle of incidence of light onto the reflective surfaces 1052 and 1054 of the X-cube prism. It should be understood that a standard X-cube prism configuration would receive light normal to the faces of the X-cube and redirect this light at 45° so that it exits the lateral faces of the X-cube at a normal angle. However, this would cause the image light 1032 a, 1032 b, and 1032 c to be incident on the same internal coupling optical element of the eyepiece 1020. Light redirecting structures 1080a, 1080c may be utilized to provide different paths for the image light 1032a, 1032b, 1032c so that the image light is incident on associated ones of the internal coupling optical elements 1022a, 1022b, 1022c of the waveguide assembly.

[0222] In some embodiments, the light redirecting structures 1080a, 1080c may be lens structures. It should be understood that the lens structures may be configured to receive incident light and redirect the incident light at an angle such that the light reflects off a corresponding one of the reflective surfaces 1052, 1054 and propagates along the optical path toward a corresponding one of the internal coupling optical elements 1022a, 1022c. As an example, the light redirecting structures 1080a, 1080c may comprise microlenses, nanolenses, reflective wells, metasurfaces, and liquid crystal gratings. In some embodiments, the microlenses, nanolenses, reflective wells, metasurfaces, and liquid crystal gratings may be organized in an array. For example, each light emitter of the microdisplays 1030a, 1030c may be matched with one microlens. In some embodiments, the microlenses or reflective wells may be asymmetric and / or the light emitters may be positioned off-center relative to the microlenses to redirect light in a particular direction. Additionally, in some embodiments, the light redirecting structures 1080a, 1080c may be collimators, which narrow the angular emission profile of the associated light emitter and ultimately increase the amount of light that is coupled in-coupling into the eyepiece 1020. Further details regarding such light redirecting structures 1080a, 1080c are discussed below with respect to Figures 24A-27C.

[0223] 13A , in some embodiments, two or more of the in-coupling optical elements 1022a, 1022b, 1022c may overlap (e.g., as seen in a head-on view in the direction of light propagation into the in-coupling optical elements 1022a, 1022b, 1022c). FIG. 13A illustrates an example of a side view of a wearable display system with an optical projection system 1010 having multiple emissive microdisplays 1032a, 1032b, 1032c and an eyepiece 1020 with overlapping optical in-coupling optical elements 1022a, 1022c and a non-overlapping optical in-coupling optical element 1022b. As shown, the in-coupling optical elements 1022a, 1022c overlap, while the in-coupling optical element 1022b is laterally offset. In other words, the internal coupling optical elements 1022a, 1022c are aligned directly within the paths of the image lights 1032a, 1032c, while the image light 1032b follows a different path to the eyepiece 1020 so that it is incident on an area of ​​the eyepiece 1020 that is laterally shifted relative to the area on which the image lights 1032a, 1032c are incident.

[0224] As shown, a distinction between the paths for image light 1032b and image light 1032a, 1032c may be established using light redirecting structures 1080a, 1080c. In some embodiments, image light 1032b from emissive microdisplay 1030b travels directly through optical combiner 1052. Image light 1032a from emissive microdisplay 1032a is redirected by light redirecting structure 1080a to reflect from reflective surface 1054 and out of optical combiner 1050 propagating in the same direction as image light 1032c. It should be appreciated that image light 1032c from emissive microdisplay 1032c is redirected by light redirecting structure 1080c to reflect from reflective surface 1052 at an angle such that image light 1032c out of optical combiner 1050 propagating in the same direction as image light 1032b. Thus, the redirection of light by light redirecting structures 1080a, 1080c and the angles of reflective surfaces 1052, 1054 are configured to provide a common path for image light 1032a, 1032c out of optical combiner 1050, which common path is different from the path of image light 1032b. In some other embodiments, one or both of light redirecting structures 1080a, 1080c may be omitted, and reflective surfaces 1052, 1054 within optical combiner 1050 may be configured to reflect image light 1032a, 1032c in appropriate individual directions so that it exits optical combiner 1050 and propagates in the same direction, different from the direction of image light 1032b. Thus, after propagating through projection optics 1070, image light 1032a, 1032c emerges from one exit pupil, while image light 1032b emerges from another exit pupil. In this configuration, optical projection system 1010 may be referred to as a two-pupil projection system.

[0225] In some embodiments, the light projection system 1010 may have a single output pupil and may be referred to as a single-pupil projection system. In such embodiments, the light projection system 1010 may be configured to direct image light 1032a, 1032b, 1032c onto a single common area of ​​the eyepiece 1020. Such a configuration is shown in FIG. 13B, which illustrates a wearable display system with the light projection system 1010 having multiple emissive microdisplays 1030a, 1030b, 1030c configured to direct light to a single light in-coupling area of ​​the eyepiece 1020. In some embodiments, as discussed further herein, the eyepiece 1020 may include a stack of waveguides with overlapping light in-coupling optical elements. In some other embodiments, a single light in-coupling optical element may be configured to in-couple light of all primary colors into a single waveguide. The display system of Figure 13B is similar to the display system of Figure 13A, except for the omission of light redirecting structures 1080a, 1080c and the inclusion of an in-coupling optical element 1122a associated with waveguide 1020a. As shown, in-coupling optical element 1122a in-couples each of image lights 1032a, 1032b, 1032c into waveguide 1020a, which then relays the image light to eye 210. In some embodiments, in-coupling optical element 1122a may comprise a diffraction grating. In some embodiments, in-coupling optical element 1122a is a metasurface and / or a liquid crystal grating.

[0226] As discussed herein, in some embodiments, the emissive microdisplays 1030a, 1030b, 1030c may be monochrome microdisplays configured to emit light of different colors. In some embodiments, one or more of the emissive microdisplays 1030a, 1030b, 1030c may have groups of light emitters configured to emit light of two or more, but not all, primary colors. For example, a single emissive microdisplay may have groups of light emitters with at least one light emitter per group configured to emit blue light and at least one light emitter per group configured to emit green light, while separate emissive microdisplays on different faces of the X-cube 1050 may have light emitters configured to emit red light. In some other embodiments, the emissive microdisplays 1030a, 1030b, 1030c may each be a full-color display having light emitters of all primary colors, respectively. As described herein, utilizing multiple similar microdisplays can provide advantages for dynamic range and increased display brightness.

[0227] In some embodiments, a single full-color emissive microdisplay may be utilized. FIG. 14 illustrates an example of a wearable display system with a single emissive microdisplay 1030b. The wearable display system of FIG. 14 is similar to the wearable display system of FIG. 14, except that the single emissive microdisplay 1030b is a full-color microdisplay configured to emit light of all primary colors. As shown, the microdisplay 1030b emits image light 1032a, 1032b, 1032c for each primary color. In such an embodiment, the optical combiner 1050 (FIG. 13B) may be omitted, which may advantageously reduce the weight and size of the wearable display system relative to a system with an optical combiner.

[0228] As discussed above, the internal coupling optical elements of the eyepiece 1020 may take on a variety of configurations. Some example configurations for the eyepiece 1020 are discussed below in connection with Figures 15-23C.

[0229] Figure 15 illustrates a side view of an example eyepiece 1020 having a stack of waveguides 1020a, 1020b, 1020c with overlapping in-coupling optical elements 1022a, 1022b, 1022c, respectively. It should be understood that the illustrated waveguide stack may be utilized in place of the single illustrated waveguide 1020a of Figures 13B and 14. As discussed herein, the in-coupling optical elements 1022a, 1022b, 1022c are each configured to in-couple light having a specific color (e.g., light of a particular wavelength or range of wavelengths). In the illustrated orientation of the eyepiece 1020, where the image light propagates vertically down the page toward the eyepiece 1020, the internal coupling optical elements 1022a, 1022b, 1022c are aligned vertically with each other (e.g., along an axis parallel to the propagation direction of the image light 1032a, 1032b, 1032c) so that they spatially overlap each other, as seen in the top-down view (a head-on view in the direction of the image light 1032a, 1032b, 1032c propagating into the internal coupling optical elements).

[0230] Continuing with reference to FIG. 15, as discussed herein, the projection system 1010 (FIGS. 13, 14) is configured to output a first monochrome color image, a second monochrome color image, and a third monochrome color image (e.g., red, green, and blue color images) through a single pupil of the projection system, the monochrome images being formed by image light 1032a, 1032b, and 1032c, respectively. The internal coupling optical element 1022c is configured to internally couple image light 1032c into the waveguide 1020c for a first color image to propagate through the waveguide 1020c by multiple total internal reflections at the top and bottom major surfaces of the waveguide 1020c, the internal coupling optical element 1022b is configured to internally couple image light 1032b into the waveguide 1020b for a second color image to propagate through the waveguide 1020b by multiple total internal reflections at the top and bottom major surfaces of the waveguide 1020b, and the internal coupling optical element 1022a is configured to internally couple image light 1032a into the waveguide 1020a for a third color image to propagate through the waveguide 1020a by multiple total internal reflections at the top and bottom major surfaces of the waveguide 1020a.

[0231] As discussed herein, the in-coupling optical element 1022c is preferably configured to in-couple substantially all incident light 1032c corresponding to the first color image into the associated waveguide 1020c, while allowing substantially all incident light 1032b, 1032a corresponding to the second and third color images, respectively, to be transmitted without being in-coupled. Similarly, the in-coupling optical element 1022b is preferably configured to in-couple substantially all incident image light 1032b corresponding to the second color image into the associated waveguide 1020b, while allowing substantially all incident light corresponding to the third color image to be transmitted without being in-coupling.

[0232] It should be understood that in practice, the various in-coupling optical elements may not have perfect selectivity. For example, some of the image light 1032b, 1032a may be undesirably in-coupled into waveguide 1020c by in-coupling optical element 1022c, and some of the incident image light 1032a may be undesirably in-coupled into waveguide 1020b by in-coupling optical element 1022b. Furthermore, some of the image light 1032c may be transmitted through in-coupling optical element 1022c and in-coupled into waveguides 1020b and / or 1020a by in-coupling optical elements 1020b and / or 1020a, respectively. Similarly, some of the image light 1032b may be transmitted through in-coupling optical element 1022b and in-coupled into waveguide 1020a by in-coupling optical element 1022a.

[0233] Incoupling image light for a color image into an unintended waveguide can cause undesirable optical effects, such as crosstalk and / or afterimages. For example, incoupling image light 1032c for a first color image into unintended waveguides 1020b and / or 1020a can result in undesirable crosstalk between the first color image, the second color image, and / or the third color image, and / or can cause undesirable afterimages. As another example, incoupling image light 1032b, 1032a for the second or third color image, respectively, into unintended waveguides 1020c can result in undesirable crosstalk between the first color image, the second color image, and / or the third color image, and / or can cause undesirable afterimages. In some embodiments, these undesirable optical effects can be mitigated by providing color filters (e.g., light-absorbing color filters), which can reduce the amount of incident light that is incoupled into the unintended waveguides.

[0234] FIG. 16 illustrates a side view of an example of a stack of waveguides with color filters to reduce image retention or crosstalk between the waveguides. The eyepiece 1020 of FIG. 16 is similar to that of FIG. 15 except for the presence of one or more of color filters 1024c, 1024b, and 1028, 1026. Color filters 1024c, 1024b are configured to reduce the amount of light that is unintentionally incoupled into waveguides 1020b and 1020a, respectively. Color filters 1028, 1026 are configured to reduce the amount of unintentionally incoupled image light propagating through waveguides 1020b, 1020c, respectively.

[0235] 16 , a pair of color filters 1026 disposed on the upper and lower major surfaces of waveguide 1020c may be configured to absorb image light 1032a, 1032b that may be unintentionally incoupled into waveguide 1020c. In some embodiments, color filter 1024c disposed between waveguides 1020c and 1020b is configured to absorb image light 1032c that is not incoupled but is transmitted through incoupling optical element 1022c. A pair of color filters 1028 disposed on the upper and lower major surfaces of waveguide 1020b is configured to absorb image light 1032a that is incoupling into waveguide 1020b. Color filter 1024b disposed between waveguides 1020b and 1020a is configured to absorb image light 1032b that is transmitted through incoupling optical element 710.

[0236] In some embodiments, the color filters 1026 on each major surface of the waveguide 1020c are similar and configured to absorb light of both wavelengths of the image light 1032a, 1032b. In some other embodiments, the color filters 1026 on one major surface of the waveguide 1020c may be configured to absorb light of the color of the image light 1032a, and the color filters on the other major surface may be configured to absorb light of the color of the image light 1032b. In either arrangement, the color filters 1026 may be configured to selectively absorb the image light 1032a, 1032b propagating through the waveguide 1020c by total internal reflection. For example, upon TIR bounce of the image light 1032a, 1032b from the major surfaces of the waveguide 1020c, the image light 1032a, 1032b contacts the color filters 1026 on those major surfaces, and a portion of that image light is absorbed. Preferably, due to the selective absorption of image light 1032a, 1032b by color filter 1026, the propagation of image light 1032c incoupled via TIR through waveguide 1020c is not significantly affected.

[0237] Similarly, the plurality of color filters 1028 may be configured as absorption filters that absorb the incoupled image light 1032a propagating through the waveguide 1020b by total internal reflection. Upon TIR bouncing of the image light 1032a from the major surfaces of the waveguide 1020b, the image light 1032a contacts the color filters 1028 on those major surfaces, and a portion of the image light is absorbed. Preferably, the absorption of the image light 1032a is selective and does not affect the propagation of the incoupled image light 1032b, which also propagates via TIR through the waveguide 1020b.

[0238] 16 , color filters 1024c and 1024b may also be configured as light-absorbing filters. Color filter 1024c may be substantially transparent to light of the color of image light 1032a, 1032b, such that image light 1032a, 1032b is transmitted through color filter 1024c with little or no attenuation, while light of the color of image light 1032c is selectively absorbed. Similarly, color filter 1024b may be substantially transparent to light of the color of image light 1032a, such that incident image light 1032a is transmitted through color filter 1024b with little or no attenuation, while light of the color of image light 1032b is selectively absorbed. Color filter 1024c may be disposed on a major surface (e.g., the upper major surface) of waveguide 1020b, as shown in FIG. 16 . Alternatively, color filter 1024c may be disposed on a separate substrate positioned between waveguides 1020c and 1020b. Similarly, color filter 1024b may be disposed on a major surface (e.g., the upper major surface) of waveguide 1020a. Alternatively, color filter 1024b may be disposed on a separate substrate positioned between waveguides 1020b and 1020a. It should be understood that color filters 1024c and 1024b may be vertically aligned with a single pupil of the projector that outputs image light 1032a, 1032b, 1032c (in an orientation where image light 1032a, 1032b, 1032c propagates perpendicular to waveguide stack 1020, as shown).

[0239] In some embodiments, the color filters 1026 and 1028 may have a single-pass attenuation coefficient of less than about 10% (e.g., less than or equal to about 5%, less than or equal to about 2%, and greater than about 1%) to avoid significant undesirable absorption of light propagating through the thickness of the waveguides 1020c, 1020b (e.g., light of the color of the image light 1032a, 1032b propagating from the surrounding environment and / or other waveguides through the waveguides 1020c, 1020b). Various embodiments of the color filters 1024c and 1024b may be configured to have a low attenuation coefficient for wavelengths to be transmitted and a high attenuation coefficient for wavelengths to be absorbed. For example, in some embodiments, color filter 1024c may be configured to transmit greater than 80%, greater than 90%, or greater than 95% of incident light having the color of image light 1032a, 1032b and absorb greater than 80%, greater than 90%, or greater than 95% of incident light having the color of image light 1032a. Similarly, color filter 1024b may be configured to transmit greater than 80%, greater than 90%, or greater than 95% of incident light having the color of image light 1032a and absorb greater than 80%, greater than 90%, or greater than 95% of incident light having the color of image light 1032b.

[0240] In some embodiments, the color filters 1026, 1028, 1024c, 1024b may comprise a layer of color-selective light-absorbing material deposited on one or both surfaces of the waveguides 1020c, 1020b, and / or 1020a. The color-selective light-absorbing material may comprise dyes, inks, or other light-absorbing materials such as metals, semiconductors, and dielectrics. In some embodiments, the light absorption of materials such as metals, semiconductors, and dielectrics may be made color-selective by utilizing these materials to form subwavelength gratings (e.g., gratings that do not diffract light). The gratings may be made from plasmonics (e.g., gold, silver, and aluminum) or semiconductors (e.g., silicon, amorphous silicon, and germanium).

[0241] The color-selective material may be deposited on a substrate using various deposition methods. For example, the color-selective light-absorbing material may be deposited on a substrate using jet deposition techniques (e.g., inkjet deposition). Inkjet deposition can facilitate the deposition of thin layers of color-selective light-absorbing material. Because inkjet deposition allows deposition to be localized on selected areas of the substrate, inkjet deposition offers a high degree of control over the thickness and composition of the layer of color-selective light-absorbing material, including providing non-uniform thicknesses and / or compositions across the substrate. In some embodiments, the color-selective light-absorbing material deposited using inkjet deposition may have a thickness of about 10 nm to about 1 micron (e.g., about 10 nm to about 50 nm, about 25 nm to about 75 nm, about 40 nm to about 100 nm, about 80 nm to about 300 nm, about 200 nm to about 500 nm, about 400 nm to about 800 nm, about 500 nm to about 1 micron, or any value within a range / subrange defined by any of these values). Controlling the thickness of the deposited layer of color-selectively absorbing material can be advantageous in achieving a color filter with a desired attenuation coefficient. Furthermore, layers with different thicknesses may be deposited on different portions of the substrate. Additionally, different compositions of color-selectively absorbing material may be deposited on different portions of the substrate using inkjet deposition. Such variations in composition and / or thickness can advantageously allow for location-specific variations in absorptivity. For example, in areas of the waveguide where transmission of light from the surroundings is not required (to allow the viewer to see the surrounding environment), the composition and / or thickness may be selected to provide high absorptivity or attenuation of selected wavelengths of light. Other deposition methods, such as coating, spin-coating, spraying, etc., may also be employed to deposit color-selectively absorbing material on the substrate.

[0242] Figure 17 illustrates a top-down view example of the waveguide assembly of Figures 15 and 16. As shown, the in-coupling optical elements 1022a, 1022b, 1022c spatially overlap. Additionally, the waveguides 1020a, 1020b, 1020c may be vertically aligned with each waveguide's associated optically dispersive element 730, 740, 750 and associated external coupling optical element 800, 810, 820. The in-coupling optical elements 1022a, 1022b, 1022c are configured to in-couple incident image light 1032a, 1032b, 1032c (Figures 15 and 16), respectively, into the waveguides 1020a, 1020b, 1020c, respectively, such that the image light propagates toward the associated optically dispersive element 730, 740, 750 by TIR.

[0243] Figure 18 illustrates another embodiment of a top-down view of the waveguide assembly of Figures 15 and 16. As in Figure 17, in-coupling optical elements 1022a, 1022b, 1022c spatially overlap, and waveguides 1020a, 1020b, 1020c are vertically aligned. However, in place of each waveguide's associated optically dispersive element 730, 740, 750 and associated out-coupling optical element 800, 810, 820, there is a combined OPE / EPE 1281, 1282, 1283, respectively. The internal coupling optical elements 1022a, 1022b, 1022c are configured to internally couple the incident image light 1032a, 1032b, 1032c (Figures 15 and 16) into the waveguides 1020a, 1020b, 1020c, respectively, so that the image light propagates by TIR towards the associated combined OPE / EPE 1281, 1282, 1283.

[0244] While Figures 15-18 show overlapping in-coupling optical elements for a single-pupil configuration of the display system, it should be understood that the display system may have a two-pupil configuration in some embodiments. In such configurations where three primary colors are utilized, image light for two colors may have overlapping in-coupling optical elements, while image light for a third color may have a laterally shifted in-coupling optical element. For example, the optical combiner 1050 (Figures 11A, 12, 13A-13B) and / or light redirecting structures 1080a, 1080c may be configured to direct image light through the projection optics 1070 such that image light for two colors is incident directly onto the overlapping area of ​​the eyepiece 1020, while another color of image light is incident onto a laterally shifted area. For example, the reflective surfaces 1052, 1054 (FIG. 11A) may be angled so that image light of one color follows a common light path with image light from emissive microdisplay 1030b, while image light of another color follows a different light path. In some embodiments, rather than having both light redirecting structures 1080a, 1080c (FIG. 12), one of these light redirecting structures may be omitted so that only light from one of microdisplays 1030a, 1030c is angled to provide a different light path than light emitted by the other two microdisplays.

[0245] Figure 19A illustrates a side view of an example eyepiece having a stack of waveguides with some overlapping in-coupling optical elements and some laterally offset in-coupling optical elements. The eyepiece of Figure 19A is similar to the eyepiece of Figure 15, but one of the in-coupling optical elements is laterally offset relative to the other in-coupling optical element. In the illustrated orientation of the eyepiece 1020, where image light propagates vertically down the page toward the eyepiece 1020, the in-coupling optical elements 1022a, 1022c are vertically aligned with each other so that they spatially overlap each other (e.g., along an axis parallel to the propagation direction of the image light 1032a, 1032c), as seen in a head-on view in the direction of image light 1032a, 1032c propagating through the in-coupling optical elements 1022a, 1022b, 1022c. As seen in the same head-on view (e.g., as seen in a top-down view in the illustrated orientation), the in-coupling optical element 1022b is laterally offset relative to the other in-coupling optical elements 1022a, 1022c. Light for the in-coupling optical element 1022b is output to the eyepiece 1020 through a different exit pupil than light for the in-coupling optical elements 1022a, 1022c. It should be understood that the illustrated waveguide stack, comprising waveguides 1020a, 1020b, 1020c, may be utilized in place of the single illustrated waveguide 1020a of FIGS. 13 and 14.

[0246] Continuing with reference to FIG. 19, internal coupling optical element 1022c is configured to internally couple image light 1032c into waveguide 1020c by multiple total internal reflections between the top and bottom major surfaces of waveguide 1020c to propagate through waveguide 1020c, internal coupling optical element 1022b is configured to internally couple image light 1032b into waveguide 1020b by multiple total internal reflections between the top and bottom major surfaces of waveguide 1020b to propagate through waveguide 1020b, and internal coupling optical element 1022a is configured to internally couple image light 1032a into waveguide 1020a by multiple total internal reflections between the top and bottom major surfaces of waveguide 1020a to propagate through waveguide 1020a.

[0247] The in-coupling optical element 1022c is preferably configured to in-couple all incident light 1032c into the associated waveguide 1020c while transmitting all incident light 1032a. On the other hand, the image light 1032b may propagate to the in-coupling optical element 1022b without having to propagate through any other in-coupling optical elements. This may be advantageous in some embodiments by allowing light to which the eye is more sensitive to be incident on the desired in-coupling optical element without any loss or distortion associated with propagation through other in-coupling optical elements. Without being limited by theory, in some embodiments, the image light 1032b is green light to which the human eye is more sensitive. While the waveguides 1020a, 1020b, and 1020c are illustrated as being arranged in a particular order, it should be understood that in some embodiments, the order of the waveguides 1020a, 1020b, and 1020c may be different.

[0248] As discussed herein, it should be understood that the in-coupling optical element 1022c above the in-coupling optical element 1022a may not have perfect selectivity. Some of the image light 1032a may be undesirably in-coupled into the waveguide 1020c by the in-coupling optical element 1022c, and some of the image light 1032c may be transmitted through the in-coupling optical element 1022c, after which the image light 1032c may strike the in-coupling optical element 1020a and be in-coupled into the waveguide 1020a. As discussed herein, such undesirable in-coupling may be visible as afterglow or crosstalk.

[0249] FIG. 19B illustrates a side view of an example of the eyepiece of FIG. 19A with color filters to reduce image retention or crosstalk between waveguides. In particular, color filters 1024c and / or 1026 are added to the structure shown in FIG. 19A. As shown, in-coupling optical element 1022c may unintentionally in-couple a portion of image light 1032a into waveguide 1020c. Additionally, or alternatively, a portion of image light 1032c may undesirably be transmitted through in-coupling optical element 1022c and then unintentionally in-coupled by in-coupling optical element 1022a.

[0250] To mitigate unintentional in-coupling of image light 1032a propagating through the waveguide 1022c, an absorbing color filter 1026 may be provided on one or both major surfaces of the waveguide 1022c. The absorbing color filter 1026 may be configured to absorb light of the color of the image light 1032a that is unintentionally in-coupled. As shown, the absorbing color filter 1026 is positioned in the general direction of propagation of the image light through the waveguide 1020c. Thus, the absorbing color filter 1026 is configured to absorb image light 1032a as that light propagates through the waveguide 1020c by TIR and contacts the absorbing color filter 1026 as it reflects off one or both major surfaces of the waveguide 1020c.

[0251] 19B , an absorptive color filter 1024c may be provided in front of the in-coupling optical element 1022a to attenuate image light 1032c that propagates through the in-coupling optical element 1022c without being in-coupling. The absorptive color filter 1024c is configured to absorb light of the color of the image light 1032c and prevent that light from propagating to the in-coupling optical element 1022a. While illustrated between waveguides 1020c and 1020b, in some other embodiments, the absorptive color filter 1024c may be positioned between waveguides 1020b and 1020a. It should be understood that further details regarding the composition, formation, and properties of the absorptive color filters 1024c and 1026 are provided in the discussion of FIG. 16 .

[0252] It should also be understood that in the embodiments illustrated in Figures 16 and 19B, one or more of the color filters 1026, 1028, 1024c, and 1024b may be omitted if one or more of the internal coupling optical elements 1022a, 1022b, 1022c have sufficiently high selectivity with respect to the color of light that is intended to be internally coupled into the associated waveguides 1020a, 1020b, 1022c, respectively.

[0253] 20A illustrates an example of a top-down view of the eyepiece of FIGS. 19A and 19B. As shown, the in-coupling optical elements 1022a, 1022c are spatially overlapping, while the in-coupling optical element 1022b is laterally offset. Additionally, the waveguides 1020a, 1020b, 1020c may be vertically aligned, along with each waveguide's associated optically dispersive element 730, 740, 750 and associated out-coupling optical element 800, 810, 820. The internal coupling optical elements 1022a, 1022b, 1022c are configured to internally couple the incident image light 1032a, 1032b, 1032c (Figures 15 and 16) into the waveguides 1020a, 1020b, 1020c, respectively, so that the image light propagates by TIR towards the associated optical dispersion elements 730, 740, 750.

[0254] Figure 20B illustrates another embodiment of a top-down view of the waveguide assembly of Figures 19A and 19B. As in Figure 20A, the in-coupling optical elements 1022a, 1022c are spatially overlapping, the in-coupling optical elements are laterally offset, and the waveguides 1020a, 1020b, 1020c are vertically aligned. However, in place of each waveguide's associated optically dispersive element 730, 740, 750 and associated out-coupling optical element 800, 810, 820, there is a combined OPE / EPE 1281, 1282, 1283, respectively. The internal coupling optical elements 1022a, 1022b, 1022c are configured to internally couple the incident image light 1032a, 1032b, 1032c (Figures 15 and 16) into the waveguides 1020a, 1020b, 1020c, respectively, so that the image light propagates by TIR towards the associated combined OPE / EPE 1281, 1282, 1283.

[0255] 21 , it should be understood that rebouncing of incoupling light can undesirably occur within a waveguide. Rebouncing occurs when incoupling light propagating along a waveguide strikes an incoupling optical element a second or subsequent time after the initial incoupling incident. Rebouncing can result in some of the incoupling light being undesirably outcoupling and / or absorbed by the material of the incoupling optical element. The outcoupling and / or absorption can undesirably reduce the overall incoupling efficiency and / or the uniformity of the incoupling light.

[0256] FIG. 21 illustrates a side view of an example of rebouncing within a waveguide 1030a. As shown, image light 1032a is incoupling into the waveguide 1030a by an incoupling optical element 1022a. The incoupling optical element 1022a redirects the image light 1032a to propagate through the waveguide generally in direction 1033. Rebouncing can occur when the incoupling image light internally reflects or bounces off a major surface of the waveguide 1030a opposite the incoupling optical element 1022a and impinges on the incoupling optical element 1022a or undergoes a second bounce (rebounce). The distance between two nearby bounces on the same surface of the waveguide 1030a is indicated by spacing 1034.

[0257] Without being limited by theory, it should be understood that the internal coupling optical element 1022a may behave symmetrically. That is, it may redirect incident light so that it propagates through the waveguide at a TIR angle. However, light incident on the diffractive optical element at a TIR angle (such as upon rebounce) may also be outcoupled. Additionally or alternatively, in embodiments in which the internal coupling optical element 1022a is coated with a reflective material, it should be understood that reflection of light from a layer of material, such as metal, may also involve partial absorption of the incident light, as reflection may involve absorption and emission of light from the material. As a result, outcoupling and / or absorption of light may undesirably cause loss of incoupling light. Thus, the rebounced light may suffer significant loss compared to light that interacts with the internal coupling optical element 1022a only once.

[0258] In some embodiments, the internal coupling element is configured to mitigate incoupling image light loss due to rebouncing. Generally, rebouncing of incoupling light occurs in the propagation direction 1033 of the incoupling light toward the end 1023 of the internal coupling optical element 1022a. For example, light incoupling at the end of the internal coupling optical element 1022a opposite the end 1023 may rebounce if the spacing 1034 for that light is sufficiently short. To avoid such rebouncing, in some embodiments, the internal coupling optical element 1022a is truncated at the propagation direction end 1023 to reduce the width 1022w of the internal coupling optical element 1022a along which rebouncing is likely to occur. In some embodiments, the truncation may be complete truncation of all structures (e.g., metallization and diffraction grating) of the internal coupling optical element 1022a. In some other embodiments, for example, if the internal coupling optical element 1022a comprises a metallized diffraction grating, a portion of the internal coupling optical element 1022a at the propagation direction end 1023 may not be metallized, such that the propagation direction end 1023 of the internal coupling optical element 1022a absorbs less of the re-bounced light and / or outcouples it with less efficiency. In some embodiments, the diffractive region of the internal coupling optical element 1022a may have a width along the propagation direction 1033 that is shorter than its length perpendicular to the propagation direction 1033, and / or may be sized and shaped such that a first portion of the image light 1032a is incident on the internal coupling optical element 1022a and a second portion of the beam of light impinges on the waveguide 1030a without being incident on the internal coupling optical element 1022a. While the waveguide 1032a and the optical in-coupling optical element 1022a are shown alone for clarity, it should be understood that rebouncing and the discussed strategies for reducing rebouncing may be applied to any of the in-coupling optical elements disclosed herein. It should also be understood that the spacing 1034 is related to the thickness of the waveguide 1030a (greater thickness results in greater spacing 1034). In some embodiments, the thickness of the individual waveguides may be selected to set the spacing 1034 such that rebouncing does not occur.Further details regarding rebounce mitigation can be found in U.S. Provisional Application No. 62 / 702,707, filed July 24, 2018, the entire disclosure of which is incorporated herein by reference.

[0259] 22A-23C illustrate examples of top-down views of eyepieces having internal coupling optical elements configured to reduce rebounce. The internal coupling optical elements 1022a, 1022b, 1022c are configured to internally couple light to propagate in a propagation direction toward an associated optically dispersive element 730, 740, 750 (FIGS. 22A-22C) or a combined OPE / EPE 1281, 1282, 1283 (FIGS. 23A-23C). As shown, the internal coupling optical elements 1022a, 1022b, 1022c may have a shorter dimension along the propagation direction and a longer dimension along a horizontal axis. For example, the internal coupling optical elements 1022a, 1022b, 1022c may each be rectangular in shape with a shorter side along the propagation axis and a longer side along an orthogonal axis. It should be understood that the in-coupling optical elements 1022a, 1022b, 1022c may have other shapes (e.g., orthogonal, hexagonal, etc.). Additionally, different ones of the in-coupling optical elements 1022a, 1022b, 1022c may have different shapes in some embodiments. Also, preferably, as shown, non-overlapping in-coupling optical elements may be positioned such that they are not in the propagation direction of other in-coupling optical elements. For example, as shown in Figures 22A, 22B, 23A, and 23B, the non-overlapping in-coupling optical elements may be arranged in lines along an axis that intersects (e.g., orthogonal to) the axis of the propagation direction.

[0260] It should be understood that the waveguide assemblies of Figures 22A-22C are similar except for the overlap of the in-coupling optical elements 1022a, 1022b, 1022c. For example, Figure 22A illustrates in-coupling optical elements 1022a, 1022b, 1022c without overlap. Figure 22B illustrates overlapping in-coupling optical elements 1022a, 1022c and non-overlapping in-coupling optical element 1022b. Figure 22C illustrates overlap between all in-coupling optical elements 1022a, 1022b, 1022c.

[0261] The waveguide assemblies of Figures 23A-23C are also similar, except for the overlap of the in-coupling optical elements 1022a, 1022b, and 1022c. Figure 23A illustrates in-coupling optical elements 1022a, 1022b, and 1022c without overlap. Figure 23B illustrates overlapping in-coupling optical elements 1022a, 1022c and non-overlapping in-coupling optical element 1022b. Figure 23C illustrates overlap between all in-coupling optical elements 1022a, 1022b, and 1022c.

[0262] 24A , it should be understood that emissive microdisplays have a high etendue, which presents challenges for efficient light utilization. As discussed herein, emissive microdisplays may include multiple individual light emitters. Each of these light emitters may have a large-angle emission profile, e.g., a Lambertian or near-Lambertian emission profile. Undesirably, not all of this light may be captured and directed toward the eyepiece of the display system.

[0263] 24A illustrates an example of the angular emission profile of light emitted by individual light emitters 1044 of an emissive microdisplay 1032 and light captured by the projection optics 1070. The illustrated emissive microdisplay 1032 may correspond to any of the emissive microdisplays disclosed herein, including emissive microdisplays 1032a, 1032b, and 1032c. As illustrated, the projection optics 1070 may be sized to capture light having the angular emission profile 1046. However, the angular emission profile 1046 within the light emitter 1044 is significantly larger, such that not all of the light emitted by the light emitter 1044 is incident on the projection optics 1070, and not necessarily at an angle at which the light will propagate into and through the projection optics 1070. As a result, some of the light emitted by the light emitter 1044 may be undesirably captured and ultimately relayed to a user's eye, thereby not forming an image and therefore "wasted." This may result in an image appearing darker than would be expected if more of the light output by light emitter 1040 ultimately reached the user's eye.

[0264] In some embodiments, one strategy for capturing more of the light emitted by the light emitter 1040 is to increase the size of the projection optics 1070 and increase the size of the numerical aperture of the projection optics 1070 that captures the light. Additionally or alternatively, the projection optics 1070 may also be formed of a high refractive index material (e.g., having a refractive index greater than 1.5), which may also facilitate light collection. In some embodiments, the projection optics 1070 may utilize a lens that is sized to capture a desired high percentage of the light emitted by the light emitter 1044. In some embodiments, the projection optics 1070 may have an elongated exit pupil and be configured to emit a light beam having a cross-sectional profile similar to the shape of the internal coupling optical elements 1022a, 1022b, 1022c of FIGS. 22A-23C . For example, the projection optics 1070 may be elongated in a dimension corresponding to the elongated dimension of the internal coupling optical elements 1022a, 1022b, 1022c of FIGS. 22A-23C . Without being limited by theory, such elongated incoupling optical elements 1022a, 1022b, 1022c may improve the etendue mismatch between the emissive microdisplay and the eyepiece 1020 (FIGS. 22A-23C). In some embodiments, the waveguide thickness of the eyepiece 1020 (e.g., FIGS. 11A and 12-23C) may be selected to increase the percentage of light that is effectively captured by reducing rebounces, for example, by increasing the rebounce spacing as discussed herein.

[0265] In some embodiments, one or more light collimators may be utilized to reduce or narrow the angular emission profile of light from the light emitter 1044. As a result, more of the light emitted by the light emitter 1044 may be captured by the projection optics 1070 and relayed to the user's eye, which may advantageously increase the brightness of the image and the efficiency of the display system. In some embodiments, the light collimators may enable the light collection efficiency of the projection optics (the percentage of light emitted by the light emitter 1044 that is captured by the projection optics) to reach values ​​of 80% or greater, 85% or greater, or 90% or greater, including approximately 85-95% or 85-90%. Additionally, the angular emission profile of light from the light emitter 1044 may be reduced (e.g., from 180°) to 60° or less, 50° or less, or 40° or less. In some embodiments, the reduced angular emission profile may be within a range of approximately 30-60°, 30-50°, or 30-40°. It should be understood that the light from the light emitter 1044 may create a cone shape, with the light emitter 1044 at the apex of the cone. The angular emission profile refers to the angle created by the sides of the cone, with the associated light emitter 1044 at the apex of that angle (as viewed in cross section, taken along a plane extending through the center of the cone and including the cone apex).

[0266] 24B illustrates an example of narrowing an angular emission profile using an array of light collimators. As shown, an emissive microdisplay 1032 includes an array of light emitters 1044, which emit light with an angular emission profile 1046. An array 1300 of light collimators 1302 is disposed in front of the light emitters 1044. In some embodiments, each light emitter 1044 is matched one-to-one with an associated light collimator 1302 (one light collimator 1302 per light emitter 1044). Each light collimator 1302 redirects incident light from the associated light emitter 1044 to provide a narrowed angular emission profile 1047. Thus, a relatively large angular emission profile 1046 is narrowed to a smaller angular emission profile 1047.

[0267] In some embodiments, the light collimator 1302 and array 1300 may be part of the light redirecting structures 1080a, 180c of Figures 12 and 13A. Thus, the light collimator 1302 may narrow the angular emission profile of the light emitter 1044 and redirect the light so that it propagates at an appropriate angle into the optical combiner 1050 and defines multiple light paths and associated multiple exit pupils. It should be understood that light may be redirected in a particular direction by appropriately shaping the light collimator 1302.

[0268] Preferably, the light collimators 1302 are positioned in close proximity to the light emitters 1044 to capture a large proportion of the light output by the light emitters 1044. In some embodiments, a gap may exist between the light collimators 1302 and the light emitters 1044. In some other embodiments, the light collimators 1302 may contact the light emitters 1044. It should be understood that the angular emission profile 1046 may produce a wide cone of light. Preferably, all or a majority of the cone of light from the light emitters 1044 is incident on a single associated light collimator 1302. Thus, in some embodiments, each light emitter 1044 is smaller (occupies a smaller area) than the light-receiving surface of the associated light collimator 1302. In some embodiments, each light emitter 1044 has a width that is smaller than the spacing between neighboring, distant light emitters 1044.

[0269] Advantageously, the light collimator 1302 may increase the efficiency of light utilization and may also reduce the occurrence of crosstalk between nearby light emitters 1044. It should be understood that crosstalk between light emitters 1044 may occur when light from nearby light emitters is captured by a light collimator 1302 that is not associated with the nearby light emitter. That captured light may propagate to a user's eye, thereby providing erroneous image information for a given pixel.

[0270] 24A and 24B, the size of the beam of light captured by the projection optics 1070 can affect the size of the beam of light exiting the projection optics 1070. As shown in FIG. 24A, without the use of an optical collimator, the exit beam can have a relatively large width 1050. As shown in FIG. 24B, with the optical collimator 1302, the exit beam can have a smaller width 1052. Thus, in some embodiments, the optical collimator 1302 can be used to provide a desired beam size for incoupling into the eyepiece. For example, the amount by which the optical collimator 1302 narrows the angular emission profile 1046 can be selected based, at least in part, on the size of the incoupling optical element in the eyepiece to which the light output by the projection optics 1070 is directed.

[0271] It should be understood that the light collimator 1302 may take a variety of forms. For example, the light collimator 1302 may be a microlens or lenslet in some embodiments. As discussed herein, each microlens preferably has a width greater than the width of the associated light emitter 1044. The microlenses may be formed from curved transparent materials such as glass or polymers, including resins such as photoresist and epoxy. In some embodiments, the light collimator 1302 may be a nanolens, e.g., a diffractive optical grating. In some embodiments, the light collimator 1302 may be a metasurface and / or a liquid crystal grating. In some embodiments, the light collimator 1302 may take the form of a reflective well.

[0272] It should be understood that different light collimators 1302 may have different dimensions and / or shapes depending on the wavelength or color of light emitted by the associated light emitter 1044. Thus, for a full-color emissive microdisplay, the array 1300 may include multiple light collimators 1302 with different dimensions and / or shapes depending on the color of light emitted by the associated light emitter 1044. In embodiments where the emissive microdisplay is a monochrome microdisplay, the array 1300 may be simplified, with each of the light collimators 1302 in the array configured to redirect light of the same color. With such monochrome microdisplays, the light collimators 1302 may, in some embodiments, be similar across the array 1300.

[0273] 24B , as discussed herein, the light collimators 1302 may have a one-to-one association with the light emitters 1044. For example, each light emitter 1044 may have a discrete associated light collimator 1302. In some other embodiments, the light collimators 1302 may be elongated such that they extend across multiple light emitters 1044. For example, in some embodiments, the light collimator 1302 may be elongated across the page and extend in front of a row of multiple light emitters 1044. In some other embodiments, a single light collimator 1302 may extend across a column of light emitters 1044. In still other embodiments, the light collimator 1302 may comprise stacked columns and / or rows of lens structures (e.g., nanolens structures, microlens structures, etc.).

[0274] As mentioned above, the light collimators 1302 may take the form of reflective wells. FIG. 25A illustrates an example of a side view of an array of tapered reflective wells for directing light to a projection optical system. As shown, the light collimator array 1300 may include a substrate 1301 within which a plurality of light collimators 1302 in the form of reflective wells may be formed. Each well may include at least one light emitter 1044, which may emit light with a Lambertian angular emission profile 1046. The reflective walls 1303 of the wells of the light collimators 1302 are tapered to reflect the emitted light so that it exits the well with a narrower angular emission profile 1047. As shown, the reflective walls 1303 may be tapered such that their cross-sectional size increases with distance from the light emitter 1044. In some embodiments, the reflective walls 1303 may be curved. For example, side 1303 may have the shape of a compound parabolic concentrator (CPC).

[0275] 25B, an example of a side view of an asymmetric tapered reflective well is illustrated. As discussed herein, for example, as illustrated in FIGS. 12A-13A, it may be desirable to utilize a light collimator 1302 to steer light in a particular direction that is not normal to the surface of the light emitter 1044. In some embodiments, as viewed in the side view illustrated in FIG. 25B, the light collimator 1302 may be asymmetric, with the top side 1303a forming a different angle (e.g., a larger angle) with the surface of the light emitter 1044 and the bottom side 1303b. For example, the angle of the reflective walls 1303a, 1303b relative to the light emitter 1044 may be different on different sides of the light collimator 1302 to direct light in a particular non-normal direction. Thus, as shown, light exiting the light collimator 1302 may generally propagate in a direction 1048 that is not normal to the surface of the light emitter 1044. In some other embodiments, the taper of the top side 1303 a may be different than the taper of the bottom side to direct light in the direction 1048. For example, the top side 1303 a may flare out more than the bottom side 1303 b.

[0276] 25 , the substrate 1301 may be formed from a variety of materials that have sufficient mechanical integrity to maintain the desired shape of the reflective wall 1303. Examples of suitable materials include metal, plastic, and glass. In some embodiments, the substrate 1301 may be a plate of material. In some embodiments, the substrate 1301 is a continuous, unitary piece of material. In some other embodiments, the substrate 1301 may be formed by joining two or more pieces of material together.

[0277] The reflective walls 1303 may be formed in the substrate 1301 by a variety of methods. For example, the walls 1303 may be formed into the desired shape by machining or otherwise removing material from the substrate 1301 to define the walls 1303. In some other embodiments, the walls 1303 may be formed as the substrate 1301 is formed. For example, the walls 1303 may be molded into the substrate 1301 as the substrate 1301 is shaped into its desired shape. In some other embodiments, the walls 1303 may be defined by rearrangement of material after the formation of the body 2200. For example, the walls 1303 may be defined by imprinting.

[0278] Once the contours of the walls 1303 are formed, they may undergo further processing to form a surface with the desired degree of reflectivity. In some embodiments, the surface of the substrate 1301 itself may be reflective, for example, the body is formed from a reflective metal. In such cases, further processing may include smoothing or polishing the interior surface of the walls 1303 to increase its reflectivity. In some other embodiments, the interior surface of the reflector 2110 may be lined with a reflective coating, for example, by a vapor deposition process. For example, the reflective layer may be formed by physical vapor deposition (PVD) or chemical vapor deposition (CVD).

[0279] It should be understood that the location of a light emitter relative to an associated light collimator can affect the direction of light emitted out of the light collimator. This is illustrated, for example, in Figures 26A-26C, which illustrate examples of differences in light paths for light emitters at different positions relative to the centerline of the associated light collimator on the upper layer. As shown in Figure 26A, the emissive microdisplay 1030 has multiple light emitters 1044a, each with an associated light collimator 1302, which promotes the output of light having a narrowed angular emission profile 1047. The light passes through projection optics 1070 (represented as a simple lens for ease of illustration), which focuses the light from the various light emitters 1044a onto an area 1402a.

[0280] 26A , in some embodiments, the optical collimators 1302 may each be symmetrical and have a centerline that extends along the axis of symmetry of the optical collimator. In the illustrated configuration, the optical emitter 1044 a is positioned on the centerline of each of the optical collimators 1302.

[0281] 26B, light emitter 1044b is offset from the centerline of its respective light collimator 1302 by a distance 1400. This offset causes light from light emitter 1044b to follow a different path through light collimator 1302, which outputs light from light emitter 1044b with a narrowed angular emission profile 1047b. Projection optics 1070 then focuses the light from light emitter 1044b onto area 1402b, which is offset relative to area 1402a onto which the light from light emitter 1044a focuses.

[0282] 26C, light emitter 1044c is illustrated as being offset from both light emitters 1044a and 1044b. This offset causes light from light emitter 1044c to follow a different path through light collimator 1302 than light from light emitters 1044a and 1044b. This causes light collimator 1302 to output light from light emitter 1044c with a narrowed angular emission profile that follows a different path than light from light emitters 1044a and 1044b to projection optics 1070. Ultimately, projection optics 1070 focuses the light from light emitter 1044c onto area 1402c, which is offset relative to areas 1402a and 1402b.

[0283] 26A-26C, each three-fold symmetry axis of light emitters 1044a, 1044b, 1044c may share a common light collimator 1302. In some embodiments, microdisplay 1030 may be a full-color microdisplay, and each light emitter 1044a, 1044b, 1044c may be configured to emit light of a different primary color. Advantageously, offset areas 1402a, 1402b, 1402c may correspond to waveguide incoupling optical elements in some embodiments. For example, areas 1402a, 1402b, 1402c may correspond to incoupling optical elements 1022a, 1022b, 1022c of FIGS. 11A and 12, respectively. Thus, the offset orientation of the light collimator 1302 and light emitters 1044a, 1044b, 1044c can advantageously provide a simple three-pupil projection system 1010 using a full-color emissive microdisplay.

[0284] As described herein, the light collimators 1302 may also take the form of nanolenses. Figure 27 illustrates an example of a side view of an individual light emitter 1044 of an emissive microdisplay 1030 with an overlying array 1300 of light collimators 1302, which are nanolenses. As discussed herein, each individual one of the light emitters 1044 may have an associated light collimator 1302. The light collimators 1302 redirect light from the light emitters 1044, narrowing the large-angle emission profile 1046 of the light emitters 1044 and outputting light with a narrowed angular emission profile 1047.

[0285] 27 , in some embodiments, the light collimator 1302 may be a grating structure. In some embodiments, the light collimator 1302 may be a grating formed by alternating elongated discrete extensions (e.g., lines) of material having different refractive indices. For example, the extensions of material 1306 may extend into and out of the page and may be formed within and separated by the material of the substrate 1308. In some embodiments, the elongated extensions of material 1306 may have a sub-wavelength width and pitch (e.g., a width and pitch smaller than the wavelength of light that the light collimator 1302 is configured to receive from its associated light emitter 1044). In some embodiments, the pitch 1304 may be 30-300 nm, the grating depth may be 10-1,000 nm, the refractive index of the material forming the substrate 1308 may be 1.5-3.5, and the refractive index of the material forming the grating features 1306 may be 1.5-2.5 (and different from the refractive index of the material forming the substrate 1308).

[0286] The grating structures shown may be formed by a variety of methods. For example, the substrate 1308 may be etched or nanoimprinted to define grooves, which may be filled with a material of a different refractive index than the substrate 1308 to form the grating features 1306.

[0287] Advantageously, nanolens arrays may offer various benefits. For example, the light collection efficiency of the nanolenslets may be as high as 80-95%, including 85-90%, with excellent reduction in angular emission profile, e.g., down to 30-40° (from 180°). In addition, low levels of crosstalk may be achieved because each of the nanolens light collimators 1302 may have physical dimensions and properties (e.g., pitch, depth, refractive index of the material forming the features 1306 and substrate 1308) selected to act on light of a particular color and potentially a particular angle of incidence, while preferably providing a high extinction ratio (with respect to wavelengths of light of other colors). In addition, the nanolens array may have a flat profile (e.g., formed on a flat substrate), which may facilitate integration with microdisplays, which may be flat panels, and may also facilitate manufacturing and provide high reproducibility and precision in forming the nanolens array. For example, highly reproducible groove formation and deposition processes may be used to form each nanolens. Furthermore, these processes allow for greater ease and reproducibility in terms of variation between nanolenses in an array than is typically achieved when forming curved lenses with similar variations.

[0288] Referring now to FIG. 28 , a perspective view of an example emissive microdisplay 1030 is illustrated. It should be appreciated that the light collimator array 1300 advantageously allows light emitted from the microdisplay to be routed as desired. As a result, in some embodiments, the light emitters of a full-color microdisplay can be organized as desired, for example, for ease of manufacturing or implementation within a display device. In some embodiments, the light emitters 1044 may be arranged in rows or columns 1306 a, 1306 b, 1306 c. Each row or column may include light emitters 1044 configured to emit light of the same primary color. In a display in which three primary colors are utilized, there may be three groups of rows or columns that are repeated across the microdisplay 1030. It should be appreciated that if more primary colors are utilized, each repeating group may have that number of rows or columns. For example, if four primary colors are utilized, each group may have four rows or four columns, with one row or one column formed by light emitters configured to emit light of a single primary color.

[0289] In some embodiments, some rows or columns may be repeated to increase the number of light emitters of a particular primary color. For example, some primary color light emitters may occupy multiple rows or columns. This may promote color balancing and / or may be utilized to address differential degradation or reduction in light emission intensity over time.

[0290] 27 and 28 , in some embodiments, each light emitter 1044 may have an associated light collimator 1302. In some other embodiments, each line 1306 a, 1306 b, 1306 c of multiple light emitters 1044 may have a single associated light collimator 1302. The single associated light collimator 1302 may extend across substantially the entire associated line 1306 a, 1306 b, or 1306 c. In some other embodiments, the associated light collimator 1302 may be elongated and extend across multiple light emitters 1044 that form part of the associated line 1306 a, 1306 b, or 1306 c, and multiple similar light collimators 1302 may be provided along each of the associated lines 1306 a, 1306 b, 1306 c.

[0291] 28 , each light emitter 1044 may be elongated along a particular axis (e.g., along the y-axis as shown). That is, each light emitter has a length along the particular axis, the length being greater than the width of the light emitter. Additionally, a set of light emitters configured to emit light of the same primary color may be arranged in a line 1306 a, 1306 b, or 1306 c (e.g., row or column) extending along an axis (e.g., the x-axis) that intersects (e.g., orthogonal to) the elongation axis of the light emitter 1044. Thus, in some embodiments, light emitters 1044 of the same primary color form a line 1306 a, 1306 b, or 1306 c of light emitters, where the line extends along a first axis (e.g., the x-axis) and the individual light emitters 1044 within the line are elongated along a second axis (e.g., the y-axis).

[0292] In contrast, it should be understood that a full-color microdisplay typically includes a subpixel for each primary color, arranged in a particular, relatively closely packed spatial orientation in groups, and these groups are reproduced across the array. Each group of subpixels may form a pixel in an image. In some cases, the subpixels are elongated along an axis, with rows or columns of subpixels of the same primary color extending along that same axis. It should be understood that such an arrangement allows the subpixels of each group to be located closely together, which can have advantages in terms of image quality and pixel density. However, in the illustrated arrangement of FIG. 28 , the subpixels of different primary colors are relatively far apart due to the elongated shape of the light emitter 1044. That is, the light emitters in line 1306a are relatively far away from the light emitters in line 1306c because the elongated shape of the light emitters in line 1306b causes light emitters 1306a and 1306c to be spaced apart more than neighboring light emitters in a given line of light emitters. While this would be expected to provide unacceptably poor image quality if the image formed on the surface of microdisplay 1030 were relayed directly to a user's eye, the use of light collimator array 1300 advantageously allows light of different colors to be routed as desired to form a high-quality image. For example, light of each primary color may be used to form a separate monochrome image, which is then routed to and combined in an eyepiece, such as eyepiece 1020 (e.g., FIGS. 11A and 12-14).

[0293] 27 and 28 , in some embodiments, each light emitter 1044 may have an associated light collimator 1302. In some other embodiments, each line 1306 a, 1306 b, 1306 c of light emitters 1044 may have a single associated light collimator 1302. That single associated light collimator 1302 may extend across substantially the entire associated line 1306 a, 1306 b, or 1306 c. In some other embodiments, an associated light collimator 1302 may be elongated and extend across multiple light emitters 1044 that form part of the associated line 1306 a, 1306 b, or 1306 c, and multiple similar light collimators 1302 may be provided along each of the associated lines 1306 a, 1306 b, 1306 c.

[0294] It should be understood that the light collimator 1302 may be utilized to direct light along different light paths to form a multi-pupil projection system. For example, the light collimator 1302 may direct light of different primary colors to two or three areas, respectively, for light intercombining.

[0295] Figure 29 illustrates an example of a wearable display system with the full-color emissive microdisplay 1030 of Figure 28 used to form a multi-pupil projection system 1010. In the illustrated embodiment, the full-color emissive microdisplay 1030 emits light of three primary colors, forming a three-pupil projection system 1010. The projection system 1010 has three exit pupils through which image light 1032a, 1032b, 1032c of different primary colors propagates to three laterally shifted optical in-coupling optical elements 1022a, 1022b, 1022c of the eyepiece 1020, respectively. The eyepiece 1020 then relays the image light 1032a, 1032b, 1032c to the user's eye 210.

[0296] The emissive microdisplay 1030 includes an array of light emitters 1044, which may be subdivided into monochrome light emitters 1044a, 1044b, and 1044c, which emit image light 1032a, 1032b, and 1032c, respectively. It should be understood that the light emitters 1044 emit image light with a wide angular emission profile 1046. The image light propagates through an array of light collimators 1300, which reduces the angular emission profile to a narrowed angular emission profile 1047.

[0297] Additionally, the array of light collimators 1300 is configured to redirect the image light (image light 1032a, 1032b, 1032c) so that the image light is incident on the projection optics 1070 at an angle that causes the projection optics 1070 to output the image light so that the image light propagates to the appropriate in-coupling optical element 1022a, 1022b, 1022c. For example, the array of light collimators 1300 is preferably configured to direct image light 1032a to propagate through the projection optics 1070 and incident on the in-coupling optical element 1022a, direct image light 1032b to propagate through the projection optics 1070 and incident on the in-coupling optical element 1022b, and direct image light 1032c to propagate through the projection optics 1070 and incident on the in-coupling optical element 1022c.

[0298] Because different light emitters 1044 may emit light of different wavelengths and may need to be redirected in different directions to reach the appropriate interconnecting optical element, in some embodiments, the light collimators associated with different light emitters 1044 may have different physical parameters (e.g., different pitches, different widths, etc.). Advantageously, the use of flat nanolenses as light collimators facilitates the formation of light collimators with varying physical properties across the array of light collimators 1300. As described herein, nanolenses may be formed using patterning and deposition processes, which facilitate the formation of structures with different pitches, widths, etc. across the substrate.

[0299] Referring again to FIG. 24A , it should be understood that the illustrated display system shows a single-emissive microdisplay and omits the optical combiner 1050 (FIGS. 11A and 12-13B). In embodiments utilizing the optical combiner 1050, the reflective surfaces 1052, 1054 (FIGS. 11A, 12-13B, and 30B) within the optical combiner 1050 are preferably specular reflectors, and light from the light emitter 1044 would be expected to retain its large-angle emission profile after reflecting from the reflective surfaces 1052, 1054. Thus, the problem with wasted light shown in FIG. 24A is similarly present when the optical combiner 1050 is utilized.

[0300] Referring now to FIG. 30A, an example of a wearable display system with an emissive microdisplay and an associated array of light collimators is illustrated. FIG. 30A shows additional details regarding the interaction between the light emitter 1044, the light collimator 1302, and the internal coupling optical elements of the eyepiece 1020. The display system includes a microdisplay 1030b, which in some embodiments may be a full-color microdisplay. In some other embodiments, the microdisplay 1030b may be a monochrome microdisplay, and an additional monochrome microdisplay (not shown) may be provided on a different side of the optional optical combiner 1050 (as shown in FIG. 30C).

[0301] Continuing with reference to FIG. 30A , the microdisplay 1030b includes an array of light emitters 1044, each emitting light with a wide angular emission profile (e.g., a Lambertian angular emission profile). Each light emitter 1044 has an associated dedicated light collimator 1302, which effectively narrows the angular emission profile to a narrowed angular emission profile 1047. The light beams 1032b with the narrowed angular emission profile pass through projection optics 1070, which projects or focuses the light beams onto an in-coupling optical element 1022b. It should be understood that the light beams 1032b have a cross-sectional shape and size 1047a. In some embodiments, the in-coupling optical element 1022b has a size and shape that substantially matches or is larger than the cross-sectional shape and size of the light beams 1032b when the beams 1032b are incident on the in-coupling optical element 1022b. Thus, in some embodiments, the size and shape of the in-coupling optical element 1022b may be selected based on the cross-sectional size and shape of the light beam 1032b as it impinges on the in-coupling optical element 1022b. In some other embodiments, other factors (e.g., rebounce mitigation or the angle or field of view assisted by the in-coupling optical element 1022b) may be utilized to determine the size and shape of the in-coupling optical element 1022b, and the light collimator 1302 may be configured (e.g., sized and shaped) to provide the light beam 1032b with an appropriately sized and shaped cross-section that is preferably completely or nearly completely encompassed by the size and shape of the in-coupling optical element 1022b. In some embodiments, the physical parameters for the light collimator 1302 and the in-coupling optical element 1022b may be modified relative to one another to provide highly efficient light utilization in conjunction with other desired functionality (e.g., rebounce mitigation, assistance for a desired field of view, etc.). Advantageously, the above-described light collimation provided by the light collimator 1302 and the matching of the cross-sectional size and shape of the light beam 1032b with the size and shape of the internal coupling optical element 1022b enable the internal coupling optical element 1022b to capture a large percentage of the incident light beam 1032b.The incoupled light then propagates through waveguide 1020 b and is outcoupled to eye 210 .

[0302] As shown, the microdisplay 1030b may include an array 1042 of light emitters 1044, each surrounded by a non-emitting area 1045 having a total width 1045w. In addition, the light emitters 1044 have a width W and a pitch P. In an array in which the light emitters 1044 are regularly spaced, each light emitter 1044 and the surrounding area 1045 effectively forms a unit cell having a width 1045w, which may be equal to the pitch P.

[0303] In some embodiments, the light collimator 1302 is a microlens positioned directly over and surrounding the associated light emitter 1044. In some embodiments, the width of the microlens 1302 is equal to 1045w, such that adjacent microlenses 1302 are nearly or directly in contact with one another. It should be appreciated that light from the light emitter 1044 may fill the associated microlens 1302, effectively expanding the area encompassed by the light emitter 1044. Advantageously, such a configuration reduces the perceptibility of the area 1045, which does not emit light and may otherwise be visible to a user as a dark space. However, because the microlens 1302 effectively expands the associated light emitter 1044 to extend across the entire area of ​​the microlens 1302, the area 1045 may be masked.

[0304] 30A , the relative sizes of the light emitters 1044 and the light collimators 1302 may be selected so that light from the light emitters 1044 fills the associated light collimators 1302. For example, the light emitters 1044 may be spaced apart sufficiently so that microlens collimators 1302 having a desired curvature can be formed extending across each of the light emitters 1044. Additionally, as noted above, the size and shape of the internal coupling optical element 1022b is preferably selected to match or exceed the cross-sectional shape and size of the light beam 1032b when incident on that internal coupling optical element 1022b. As a result, in some embodiments, the width 1025 of the internal coupling optical element 1022b is greater than or equal to the width 1045w of the microlens 1302 (which may have a width equal to P). Preferably, the width 1025 exceeds the width or 1045w or P of the microlens 1302 to allow for some divergence of the light beam 1032b. As discussed herein, the width 1025 may also be selected to mitigate re-bouncing and may be shorter than the length (orthogonal to the width) of the in-coupling optical element 1022b. In some embodiments, the width 1025 may extend along the same axis as the propagation direction of the in-coupled light 1032b through the waveguide 1020b before being out-coupled for propagation to the eye 210.

[0305] 30B, an example of an optical projection system 1010 is illustrated with multiple emissive microdisplays 1030a, 1030b, 1030c and associated arrays of optical collimators 1300a, 1300b, 1300c, respectively. The angular emission profile of light emitted by the microdisplays 1030a, 1030b, 1030c is narrowed by the optical collimator arrays 1300a, 1300b, 1300c, thereby facilitating collection of a large percentage of the emitted light by the projection optics 1070 after the light propagates through the optical combiner 1050. The projection optics 1070 then directs the light to an eyepiece, such as eyepiece 1020 (e.g., FIGS. 11A and 12-14) (not shown).

[0306] FIG. 30C illustrates an example of a wearable display system with multiple emissive microdisplays 1030a, 1030b, and 1030c, each with an associated array 1300a, 1300b, and 1300c of light collimators, respectively. The illustrated display system includes multiple microdisplays 1030a, 1030b, and 1030c for emitting light with image information. As shown, the microdisplays 1030a, 1030b, and 1030c may be microLED panels. In some embodiments, the microdisplays may be monochrome microLED panels, each configured to emit a different primary color. For example, microdisplay 1030a may be configured to emit light 1032a that is red, microdisplay 1030b may be configured to emit light 1032b that is green, and microdisplay 1030c may be configured to emit light 1032c that is blue.

[0307] Each microdisplay 1030a, 1030b, 1030c may have an associated array 1300a, 1300b, 1300c of light collimators, respectively. The light collimators narrow the angular emission profile of light 1032a, 1032b, 1032c from the light emitters of the associated microdisplay. In some embodiments, each light emitter has its own associated light collimator (as shown in FIG. 30A).

[0308] 30C , arrays of light collimators 1300a, 1300b, 1300c are located between associated microdisplays 1030a, 1030b, 1030c and optical combiner 1050, which may be an X-cube. As shown, optical combiner 1050 has internal reflective surfaces 1052, 1054 to reflect incident light out of the output face of the optical combiner. In addition to narrowing the angular emission profile of the incident light, arrays of light collimators 1300a, 1300c may be configured to redirect light from associated microdisplays 1030a, 1030c such that the light strikes the internal reflective surfaces 1052, 1054 of optical combiner 1050 at the appropriate angle for propagation toward associated optical in-coupling optical elements 1022a, 1022c, respectively. In some embodiments, to redirect light in a particular direction, the arrays of light collimators 1300a, 1300c may comprise microlenses or reflective wells, which may be asymmetric and / or the light emitters may be positioned off-center relative to the microlenses or reflective wells as disclosed herein.

[0309] 30C , projection optics 1070 (e.g., a projection lens) is disposed at the output face of optical combiner 1050 and receives the image light emitted from the optical combiner. Projection optics 1070 may include a lens configured to converge or focus the image light onto eyepiece 1020. As shown, eyepiece 1020 may include multiple waveguides, each configured to in-couple and out-couple light of a particular color. For example, waveguide 1020a may be configured to receive red light 1032a from microdisplay 1030a, waveguide 1020b may be configured to receive green light 1032b from microdisplay 1030b, and waveguide 1020c may be configured to receive blue light 1032c from microdisplay 1030c. Each waveguide 1020a, 1020b, 1020c has an associated light incoupling optical element 1022a, 1022b, 1022c, respectively, for incoupling light therein. Additionally, as discussed herein, waveguides 1020a, 1020b, 1020c may correspond to waveguides 670, 680, 690, respectively, of Figure 9B and may each have an associated orthogonal pupil expander (OPE) and exit pupil expander (EPE), which ultimately outcouples light 1032a, 1032b, 1032c to a user.

[0310] As discussed herein, wearable display systems incorporating microdisplays are preferably configured to output light with different amounts of wavefront divergence to provide a comfortable accommodation-vergence-divergence match for the user. These different amounts of wavefront divergence may be achieved using outcoupling optics with different optical powers. As discussed herein, the outcoupling optics may be present on or within the waveguide of an eyepiece, such as eyepiece 1020 (e.g., FIGS. 11A and 12-14). In some embodiments, lenses may be utilized to increase the wavefront divergence provided by the outcoupling optics, or may be used to provide the desired wavefront divergence in configurations where the outcoupling optics is configured to output collimated light.

[0311] 31A and 31B illustrate examples of an eyepiece 1020 having a lens for varying the wavefront divergence of light to the viewer. FIG. 31A illustrates the eyepiece 1020 having a waveguide structure 1032. In some embodiments, as discussed herein, light of all primary colors may be internally coupled into a single waveguide such that the waveguide structure 1032 includes only a single waveguide. This advantageously provides a compact eyepiece. In some other embodiments, the waveguide structure 1032 may be understood to include multiple waveguides (e.g., waveguides 1032a, 1032b, 1032c of FIGS. 11A and 12-13A), each configured to relay light of a single primary color to the user's eye.

[0312] In some embodiments, variable-focus lens elements 1530, 1540 may be disposed on both sides of the waveguide structure 1032. The variable-focus lens elements 1530, 1540 may be in the path of image light from the waveguide structure 1032 to the eye 210 and also in the path of light from the surrounding environment through the waveguide structure 1032 to the eye 210. The variable-focus optical element 1530 may modulate the wavefront divergence of the image light output by the waveguide structure 1032 to the eye 210. It should be understood that the variable-focus optical element 1530 may have optical power that may distort the eye 210's view of the world. As a result, in some embodiments, a second variable-focus optical element 1540 may be provided on the world-side of the waveguide structure 1032. The second variable-focus optical element 1540 may provide a refractive power opposite to that of the variable-focus optical element 1530 (or opposite to the net refractive power of the optical element 1530 and the waveguide structure 1032, if the waveguide structure 1032 has a refractive power), so that the net refractive power of the variable-focus lens elements 1530, 1540 and the waveguide structure 1032 is substantially zero.

[0313] Preferably, the refractive power of the variable-focus lens elements 1530, 1540 may be dynamically altered, for example, by applying an electrical signal thereto. In some embodiments, the variable-focus lens elements 1530, 1540 may comprise transmissive optical elements such as dynamic lenses (e.g., liquid crystal lenses, electro-active lenses, conventional refractive lenses with movable elements, mechanical deformation-based lenses, electrowetting lenses, elastomeric lenses, or multiple fluids with different refractive indices). By altering the shape, refractive index, or other properties of the variable-focus lens elements, the wavefront of incident light may be changed. In some embodiments, the variable-focus lens elements 1530, 1540 may comprise a layer of liquid crystal sandwiched between two substrates. The substrates may comprise optically transmissive materials such as glass, plastic, acrylic, etc.

[0314] In some embodiments, in addition to or as an alternative to providing variable amounts of wavefront divergence to place virtual content at different depth planes, the variable-focus lens elements 1530, 1540 and waveguide structure 1032 may advantageously provide a net refractive power equal to the user's prescription refractive power for corrective lenses. Thus, the eyepiece 1020 may serve as a substitute for lenses used to correct refractive errors, including myopia, hyperopia, presbyopia, and astigmatism. Further details regarding the use of variable-focus lens elements as substitutes for corrective lenses may be found in U.S. Patent Application No. 15 / 481,255, filed April 6, 2017, the entire disclosure of which is incorporated herein by reference.

[0315] Referring now to FIG. 31B, in some embodiments, the eyepiece 1020 may include static lens elements rather than variable lenses. Similar to FIG. 31B, the waveguide structure 1032 may include a single waveguide (e.g., which may relay light of different colors) or multiple waveguides (e.g., which may each relay light of a single primary color). Similarly, the waveguide structure 1034 may include a single waveguide (e.g., which may relay light of different colors) or multiple waveguides (e.g., which may each relay light of a single primary color). One or both of the waveguide structures 1032, 1034 may have optical power and may output light with a particular amount of wavefront divergence, or may simply output collimated light.

[0316] 31B , the eyepiece 1020 may, in some embodiments, include static lens elements 1532, 1534, 1542. These lens elements are positioned in the path of light from the surrounding environment through the waveguide structures 1032, 1034, respectively, and into the eye 210. In addition, the lens element 1532 is between the waveguide structure 1032 and the eye 210. The lens element 1532 modifies the wavefront divergence of the light output by the waveguide structure 1032 to the eye 210.

[0317] Lens element 1534 modifies the wavefront divergence of light output by waveguide structure 1034 to eye 210. It should be understood that light from waveguide structure 1034 also passes through lens element 1532. Thus, the wavefront divergence of light output by waveguide structure 1034 is modified by both lens element 1534 and lens element 1532 (and waveguide structure 1032, if waveguide structure 1032 has refractive power). In some embodiments, lens elements 1532, 1534 and waveguide structure 1032 provide a particular net refractive power for light output from waveguide structure 1034.

[0318] The illustrated embodiment provides two different levels of wavefront divergence, one for light output from waveguide structure 1032 and a second for light output by waveguide structure 1034. As a result, a virtual object may be placed on two different depth planes corresponding to the different levels of wavefront divergence. In some embodiments, additional levels of wavefront divergence, and therefore additional depth planes, may be provided by adding an additional waveguide structure between lens element 1532 and eye 210, along with additional lens elements between the waveguide structure and eye 210. Additional levels of wavefront divergence may likewise be added by adding additional waveguide structures and lens elements.

[0319] 31B , it should be understood that the lens elements 1532, 1534 and the waveguide structures 1032, 1034 provide a net refractive power that may distort the user's view of the world. As a result, the lens element 1542 may be used to counteract the refractive power and distortion of ambient light. In some embodiments, the refractive power of the lens element 1542 is set to neutralize the aggregate refractive power provided by the lens elements 1532, 1534 and the waveguide structures 1032, 1034. In some other embodiments, the net refractive power of the lens element 1542, the lens elements 1532, 1534, and the waveguide structures 1032, 1034 is equal to the user's prescribed refractive power for corrective lenses. Exemplary Light Projection System with Emissive Microdisplay Providing Enhanced Resolution

[0320] As described above, a display system (e.g., a wearable display system presenting AR or VR content) may utilize one or more emissive microdisplays to reduce size, mass, and / or power consumption relative to systems utilizing various other display technologies. For example, a display system may optionally utilize a threshold number of emissive microdisplays (e.g., three displays, each including an array of light emitters, such as microLEDs). In this example, each emissive microdisplay may be configured to generate light of a particular primary color. The generated light may be combined, as discussed herein, to provide the appearance of a full-color image. Various examples in which multiple emissive microdisplays are utilized are discussed above and below with reference to FIGS. 36A-36B. As another example, a display system may optionally utilize a single emissive microdisplay. In this example, an emissive microdisplay may include a light emitter (e.g., a microLED) for each primary color.

[0321] Utilizing one or more emissive microdisplays, the display systems described herein may be configured to output AR or VR content (“virtual content”) directly at a resolution greater than the resolution corresponding to the number of light emitters included within the emissive microdisplay. For example, the display system may utilize one or more actuators to cause movement or adjustment of one or more portions of a light projection system configured to output light forming the virtual content to a user. For example, the actuators may adjust a geometric position associated with a light emitter. As an example, as illustrated in FIG. 36B, the actuators may cause a change in the position of an emissive microdisplay panel. In this example, the micro LED panel may be shifted along one or two axes. As another example, as illustrated in FIG. 36A, the actuators may cause a change in the position of the location of the projection optics (e.g., one or more projection lenses). As described herein, the projection optics may route light generated by one or more micro LED panels to one or more internal coupling optical elements, such as an internal coupling grating (ICG). The internal coupling optical elements may be configured to route the light to a user of the display system.

[0322] The adjustments described above may be utilized to cause the geometric location of a light emitter to assume a position located within an inter-emitter region of the array. As described above, an inter-emitter region (e.g., region 1045 illustrated in FIG. 32A ) may include a region of an emissive microdisplay that includes one light emitter therein. The inter-emitter region may therefore be defined based on one or more pixel pitches. For example, the inter-emitter region may be delineated by a first side having a length along a first axis equal to the pixel pitch and a second side having a length along a second axis equal to the pixel pitch.

[0323] FIG. 32A illustrates an example of an emissive microdisplay 1030 having an array 1042 of light emitters (e.g., light emitter 1044) separated by inter-emitter regions 1045. The light emitters 1044 may have an emitter size p and a pixel pitch Λ. As shown, the light emitters 1044 may have an emitter size p and a pixel pitch Λ that are substantially the same in the x and y directions. However, it should be understood that in some embodiments, the emitter size p and the pixel pitch Λ are different in the x and y directions. Additionally, different ones of the light emitters in the array 1042 may have different sizes, shapes (e.g., rounded), compositions, etc. In the illustrated array 1042, a second light emitter on an upper row of light emitters is indicated by a reference 1044′ to facilitate subsequent discussion herein.

[0324] 32A , in some embodiments, the magnitude of the pixel pitch Λ may be greater than the emitter size p. As discussed herein, the emissive microdisplay 1030 may have a relatively low fill factor due to various physical and electrical constraints. For example, each light emitter 1044 may be disposed within an associated area 1049, with only a small portion of that area occupied by the light emitter 1044. A majority of the area 1049 is occupied by the inter-emitter region 1045. The area 1049 may be defined as extending the pixel pitch from the end of the associated light emitter 1044 along the x-axis and the pixel pitch from the end of the associated light emitter 1044 along the y-axis. A low fill factor may undesirably limit the pixel density and ultimate resolution of images formed using the emissive microdisplay 1030.

[0325] In some embodiments, the positions of the light emitters of array 1042 as seen by a user at a first time point may be shifted to their original locations within inter-emitter region 1045 at a second time point, thereby displaying pixels corresponding to those locations in the image. Thus, a high-resolution image frame may be decomposed into lower-resolution sub-frames, with a first sub-frame having pixels at locations corresponding to first positions of the light emitters, a second sub-frame having pixels at locations corresponding to second positions of the light emitters, a third sub-frame having pixels at locations corresponding to third positions of the light emitters, and so on. Thus, the positions of the light emitters as seen by a user may be adjusted to effectively tile (e.g., substantially tile) the sub-frames of the high-resolution image. It should be understood that the sub-frames and the high-resolution image frame occupy substantially the same area (e.g., are substantially the same physical size) as perceived by the user. For example, the sub-frames are preferably 90%, 95%, 99%, or 100% of the size of the high-resolution image frame, except that they have a lower pixel density than the high-resolution image frame.

[0326] Figure 32B illustrates an example of how the emissive microdisplay 1030 of Figure 32A can be configured to emulate a higher fill-factor microdisplay through time multiplexing and repositioning of the array or associated optics. As described above, the emissive microdisplay 1030 of Figure 32A can be configured to form individual partial-resolution sub-frames in rapid succession and with an offset for the perceived position of the individual light emitters. In such an embodiment, a user's visual system can merge the sub-frames together so that the user perceives a full-resolution frame.

[0327] Continuing with reference to FIG. 32B, the illustrated pixels 1044a-1044c represent the location of the first light emitter 1044 (FIG. 32A) as seen by a user at different points in time. Additionally, the illustrated pixels 1044a'-1044c' represent the location of the second light emitter 1044' (FIG. 32A) at the same point in time as the illustrated pixels 1044a-1044c, respectively. The first light emitter 1044 at the first position may emit light for pixel 1044a of a first sub-frame. This pixel may represent the first pixel 1044a of a rendered frame of virtual content. The perceived position of the first light emitter 1044 may then be shifted along the shift axis by a distance (e.g., Δx and / or Δy) less than the pixel pitch Λ, and the first light emitter 1044 may emit light for pixel 1044b in the second sub-frame. As shown, the position of the first light emitter 1044 is shifted by Δx along the x-axis. Pixel 1044b may therefore become the second pixel 1044b of the rendered frame of virtual content. As will be explained below, the geometric position of the first light emitter 1044 may be shifted via an actuator connected to the array 1042 ( FIG. 32A ). Thus, the first light emitter 1044 may be physically repositioned in three-dimensional space. The geometric position may also be shifted via an actuator connected to the projection optics through which light from the first light emitter 1044 is routed. Thus, the first light emitter 1044 may remain in the same physical location, and its light may be shifted by shifting the position of the projection optics relative to the first light emitter 1044.

[0328] Continuing with reference to FIG. 32B , following shifting the geometric position of the first light emitter 1044 for pixel 1044b, the first light emitter 1044 may again be shifted (by Δx, as shown) so that the first light emitter 1044 emits light for pixel 1044c in a third sub-frame of the rendered frame of virtual content. This pixel 1044c may represent the third pixel of the rendered frame. This process may be repeated for a total of N sub-frames that together form the full-resolution rendered frame of the virtual content. The remaining light emitters of the array 1042 may similarly emit light across multiple offset sub-frames for multiple pixels in the full frame. In the example of FIG. 32B , this process is repeated for nine sub-frames so that the first emitter 1044 provides all nine pixels that fit within area 1049. In this way, array 1042 (FIG. 32A) can output virtual content with three times the resolution in the x-direction and three times the resolution in the y-direction compared to the resolution of array 1042.

[0329] In some embodiments, the perceived position of the array of light emitters 1042 may be updated in a substantially continuous movement. For example, the position of emitter 1044 may be continuously shifted along the x-direction until outputting pixel 1044c. A display system (e.g., one or more processors or processing elements) described herein may determine the extent to which the position of emitter 1044 has been shifted during this continuous movement. The display system may be configured to determine the time to output light corresponding to the new pixel. For example, the display system may identify that the position of emitter 1044 has reached a distance corresponding to pixel 1044b. The display system may then cause emitter 1044 to output light based on the image value associated with pixel 1044b in the second sub-frame. Utilizing such continuous adjustment of the geometric position may reduce jerkiness associated with shifting the geometric position. In some other embodiments, the geometric position may be shifted in discrete steps. For example, emitter 1044 may output light corresponding to pixel 1044a. The geometric position of emitter 1044 may then be shifted in discrete steps and paused to output light corresponding to pixel 1044b. Other light emitters in array 1042 may similarly be shifted along with light emitter 1044. For example, light emitter 1044' may be shifted in discrete steps to provide pixels 1044a', 1044b', and 1044c' in different ones of the discrete steps.

[0330] In some embodiments, the number of subframes N may be determined or limited by physical properties of the array 1042. Exemplary properties may include a maximum frame rate (e.g., N is preferably not so large that the subframes cannot merge together in the user's visual system, and all N subframes are preferably displayed for a duration that is below the user's flicker fusion threshold, e.g., less than 1 / 60 seconds). Additional exemplary properties may include emitter pitch Λ, emitter size p, etc. As explained above, the number of subframes N may be determined based on the number of positions in which emitters of the array 1042 can fit within the inter-emitter region 1045. In the example of FIG. 32B , the first emitter 1044 may be located at nine distinct positions within the inter-emitter region 1045. Thus, there may be a maximum of nine subframes. If the emitter size p is larger and / or the emitter pitch Λ is smaller, the number of subframes N may be reduced. Similarly, if the emitter size p is smaller and / or the emitter pitch Λ is larger, the number of subframes N may be increased.

[0331] In some embodiments, N may be determined based on calculating a floor value for the emitter pitch Λ divided by the emitter size p. The calculated floor value may represent the number of times the emitter can be adjusted or moved in each direction. For example, if Λ=2.5 microns and p=0.8 microns, N would be equal to 3. Thus, there may be nine subframes (e.g., 3×3). It should be understood that this determination may be adjusted depending on whether the emitter pitch Λ and / or the emitter size p vary along the x and y directions. For example, if the emitter pitch X Λ X and emitter pitch Y Λ Y In this embodiment, N may therefore vary based on the direction. The number of subframes is N X ×N Y can be determined as: Exemplary Emissive Microdisplays for Forming Foveated Images

[0332] Another potentially desirable feature in VR, AR, and MR applications is foveated imaging (also simply referred to as foveation), in which the resolution of a displayed image varies across the image. In particular, VR, AR, and MR may include an eye-tracking system that determines where a user is looking. Given the limitations of the human visual system, which generally detects less detail in portions of the field of view away from the user's point of fixation, it may not be desirable to present full-resolution content (e.g., content at full rendering resolution) to the periphery of the user's vision. Peripheral full-resolution content may consume excessive processing power in rendering and have excessive power consumption when displayed by a display system. In other words, significant benefits may be achieved in terms of reduced processing load and display power consumption by reducing the resolution of content away from the user's point of fixation and by delivering highest-resolution image content only to the portion of the field of view the user is looking at, e.g., at and immediately adjacent to the point of fixation. It should be appreciated that the fixation point corresponds to the portion of the visual field that is focused on the user's foveal eye, and therefore the eye has a relatively high sensitivity to detail in this portion of the visual field.

[0333] Foveation may also be based on factors other than the point of fixation. As an example, a content creator may specify that certain content, such as text, be displayed at full resolution even when the user looks away from the content. As another example, a content creator may specify that foveation should only be active under certain conditions. As yet another example, foveation may be a user-selectable setting or may be automatically enabled as a result of a low battery condition. In at least some embodiments, foveation may conserve display resources (data, pixels, bandwidth, computation) by delivering the highest resolution only to the portion of the image within the portion of the field of view where the user is fixating (e.g., represented by foveal region A in FIG. 33 ), while delivering lower resolution images to peripheral portions (e.g., represented by peripheral region B in FIG. 33 ).

[0334] 32C, 32D, and 33 illustrate examples of configurations of array 1042 of FIG. 32A to provide foveated images 1130, 1140, and 1200, respectively.

[0335] FIG. 32C illustrates an example of a foveated image 1130 formed by an emissive microdisplay, such as the emissive microdisplay 1030 of FIG. 32A. The array 1042 of FIG. 32A may be configured to provide two levels of resolution for the rendered image 1130 of virtual content. In particular, the light emitter array 1042 may provide full resolution (or relatively high resolution) in a foveal region 1132 (the portion of the image expected to be focused on the user's fovea) and partial resolution (relatively lower resolution) in a second region 1134. The location of the foveal region 1132 may be determined according to the location of the user's fixation point. For example, a gaze detection scheme may be employed. The gaze detection scheme may track the user's eyes. As an example, pupils may be identified in each eye. Vectors may be extended from each identified pupil, and an intersection of the vectors in three-dimensional space may be determined. This intersection may represent the user's fixation point. The foveal region 1132 may correspond to a portion of the rendered image 1130 that is within a threshold angular distance of the fixation point. Additional details regarding foveation and detecting a user's fixation point may be found in U.S. Patent Application Publication No. 2018 / 0275410, the entire disclosure of which is incorporated herein by reference.

[0336] To provide high resolution for the foveal region 1132 while maintaining lower resolution for the second region 1134, the light emitters 1044 of the array 1042 ( FIG. 32A ) may be updated differently. Referring again to the example of FIG. 32B , in which the emitters are updated N (e.g., 9) times for a full-resolution frame of virtual content, the portion of the light emitter 1044 corresponding to pixels in the foveal region 1132 may be updated N times. In contrast, the remaining portion of the light emitter 1044 corresponding to pixels in the second region 1134 may be updated less than N times (e.g., 1, 2, 3, etc.) per rendered frame. As an example, the perceived position of the light emitter 1044 may be shifted as described herein. For the light emitter corresponding to the foveal region 1132, the light emitter 1044 may be updated for each shift in perceived light emitter position. For example, these emitters may generate light corresponding to updated pixel values ​​included at each emitter position. The updated pixel values ​​may represent pixel values ​​included within individual sub-frames of a full-resolution frame of virtual content. For emitters included within the second region 1134, the emitters may not be updated at each shift. For example, these emitters may generate light corresponding to the same pixel value for two or more geometric locations, or may simply not output light. As a result, these emitters may skip (e.g., not present) pixel values ​​included within one or more of the sub-frames.

[0337] Second region 1134 is illustrated in FIG. 32C as having the resolution of the physical array of emitters contained within array 1042; however, this is only one option. If desired, the partial resolution within second region 1134 may have a lower resolution than the resolution of physical array 1042. For example, some of the emitters within region 1134 may be deactivated. Alternatively, second region may have a higher resolution than the resolution of physical array 1042. For example, the perceived location of the light emitters corresponding to region 1134 may be shifted to display pixels for multiple subframes per image frame 1130, but the number of pixels displayed within region 1134 per rendered frame is less than that within foveal region 1132.

[0338] Figure 32D illustrates an example of an emissive microdisplay, such as the emissive microdisplay 1030 of Figure 32A, configured to form a foveated image with three or more levels of resolution within the image. For the displayed images 1140 shown, the array 1042 of Figure 32A may be configured to provide three levels of resolution within each display image 1140, with a first region 1142 having full resolution, a second region 1144 having intermediate resolution, and a third region 1146 having low resolution. In general, the array 1042 may be configured to implement foveation with any desired number of regions of different resolution, and the resolution at any location within the display may be selected arbitrarily (e.g., by selecting the number of total sub-frame pixels utilized per light emitter location of the array). For example, for pixels in the first region 1142, the corresponding light emitter may provide pixel information per subframe, for the second region 1144, the corresponding light emitter may provide pixel information for fewer subframes, and for the third region 1146, the corresponding light emitter may provide pixel information for even fewer subframes. In some embodiments, it may be desirable to smoothly transition between high-resolution and low-resolution regions. Such a transition can be accomplished by gradually reducing the number of subframes for which each light emitter provides information, as discussed above.

[0339] 32C and 32D, the foveal region (e.g., regions 1132 and 1142) and transition region 1144 are shown as rectangles (squares) for ease of illustration, but it should be understood that these regions may take any shape. For example, these regions may have shapes such as circles, star shapes, ovals, etc. FIG. 33 illustrates another example of a foveated image provided by an emissive microdisplay. As shown in FIG. 33, the highest resolution may be provided only in the foveal portion 1202, which may have a circular shape and be represented by region A 1202. Outside the foveal portion of the field of view (e.g., within region B 1204), the resolution of the displayed image is reduced, thereby reducing the processing load associated with rendering and may reduce display power consumption.

[0340] 32C, 32D, and 33, the location of the high-resolution portion of the foveated image may be determined according to the user's eye posture or gaze direction (e.g., as determined by an eye-tracking system including components such as camera assembly 630 of FIG. 6). As an example, the high-resolution portion (e.g., foveal region 1132, first region 1142, and region A1202) may be approximately in the center of the foveated image when the user is looking straight ahead, and the high-resolution portion may be shifted to the left when the user is looking left. In this way, the user may be presented with a relatively high-resolution image along the user's direction of gaze (e.g., at the fixation point), while the user is presented with lower resolution in portions of the image within their peripheral vision. (Example Movements of Emissive Microdisplays and / or Display Optics)

[0341] As discussed herein, the position of a displayed pixel may be shifted by, for example, shifting the physical position of a portion of the optical projection system, such as light emitter 1044 (FIG. 32A) and / or projection optics 1070 (FIGS. 11A, 12-14, 24A-24B, 26A-26C, and 29-30C). Also, as discussed herein, the physical position may be shifted using an actuator mechanically connected to the portion to be shifted. It should be understood that the position of a light emitter may be shifted by, for example, shifting an array containing the light emitter.

[0342] In some embodiments, these shifts may occur in discrete steps. For example, the light emitters and / or projection optics may be stationary or substantially stationary while they emit light to form the pixels of each sub-frame. The positions of the light emitters and / or projection optics may then be shifted between presentations of different sub-frames.

[0343] In some embodiments, the light emitter and / or projection optics may be moved continuously between sub-frames, with or without a reduction in speed, while the light emitter displays or projects each sub-frame. Such continuous movement may advantageously be simpler to implement than precisely starting and stopping the movement of the light emitter and / or projection optics in small steps. In either case, the result remains that a relatively low-resolution and low-fill-factor array 1042 ( FIG. 32A ) emulates a relatively high-resolution and high-fill-factor array.

[0344] FIG. 34 illustrates various exemplary paths of movement of a portion of an emissive microdisplay to shift the position of a displayed pixel. For example, as described herein, the movement may be performed using an actuator connected to one or more emissive microdisplays or connected to one or more projection optics. The actuator may move the emissive microdisplay and / or the projection optics in a plane along the illustrated path. In FIG. 34, each numbered position can be understood to be a position where light for a pixel of a different subframe is emitted, and thus each numbered position is associated with a different subpixel. Preferably, one loop of the various paths of movement is completed within the flicker fusion threshold and returned to the initial position.

[0345] In some embodiments, movement may be achieved through the use of two actuators. For example, a first actuator may regulate movement in a first direction (e.g., the x-direction), and a second actuator may regulate movement in a second direction (e.g., the y-direction). In some embodiments, the first and second actuators may operate orthogonally (e.g., 180 degrees out of phase with each other). As an example, the first actuator may perform a cosine movement, while the second actuator may perform a sinusoidal movement, with the two movements combined to define a circle. As a result, it should be understood that in some embodiments, the various actuators herein (e.g., actuators 1504, 1504a-c, etc.) may be understood to be aggregate structures encompassing two constituent actuators, each providing movement along a particular axis.

[0346] Continuing with reference to FIG. 34 , in movement pattern 1300, the geometric location moves (e.g., oscillates) back and forth between two points, thus providing a perceived pixel resolution that is twice the base resolution of the array. It will be understood that base resolution, as discussed herein, is the resolution provided by the array without shifting the array. In movement pattern 1302, the path of movement may define a triangular shape, which may increase the base resolution of the array 1042 by up to three times. In movement pattern 1304, the path of movement may define a rectangular pattern, thus increasing the base resolution of the array 1042 by up to four times. In movement pattern 1306, the geometric location moves within a rectangular pattern, thus increasing the base resolution of the array 1042 by up to six times. It should be understood that different subframes are not necessarily presented at each numbered location, and as a result, the increase in resolution may be “up to” a certain factor, as explained above.

[0347] 34, it should be understood that in some embodiments, other additional sub-frames may be presented on each leg of the various illustrated paths. For example, on the leg of travel path 1300 from position 1 to position 2, one or more sub-frames may be presented at different points on that path between positions 1 and 2. In such an arrangement, the increase in resolution may be at least the associated multiple of the above for each travel pattern.

[0348] 35A and 35B, an example of how displacing a light emitter and projection optics can change the location of a displayed pixel is illustrated. As shown in FIG. 35A, displacing an object point (e.g., an individual light emitter) along a line on plane 1400 by δ, which may represent displacing an array along that line, changes the direction of a light ray transmitted through projection optics 1070 from a first direction α1 1404 to a second direction α2 1406. This change in direction will then shift the pixel provided by the illustrated light emitter because the position of the light emitter has shifted. The direction of the light ray transmitted through projection optics 1070 may have an approximately one-to-one correspondence with the position of the object point. Thus, an emissive microdisplay may be shifted along one or more axes, shifting the location of a displayed pixel.

[0349] 35B, displacing the projection optics 1070 by δ also changes the direction of the transmitted light rays from a first position α1 1404 to a second position α2 1406. It should be understood that the shift of the projection optics 1070 along one or more axes may be based on physical properties associated with the projection optics 1070. For example, the range to which the projection optics 1070 is adjusted upward may depend on the physical properties of the projection optics 1070 and the effect of the projection optics 1070 on the path of the light. Exemplary properties may include focal length, lens type, refractive index, radius of curvature, etc.

[0350] Thus, the techniques described herein for improving the resolution of an emissive microdisplay may be accomplished via displacement of projection optics or other optical components between the emissive microdisplay and a user. Furthermore, as described above with respect to Figures 9A-9E, a full-color emissive microdisplay may employ three emissive microdisplays, each with a different color (e.g., a red array, a green array, and a blue array), whose light is optically combined and then projected through a common projection optics. In such an embodiment, it may be simpler to implement controlled displacement of the common optics, since only a single displacement actuator (or set of actuators) may be required to displace the common optics, as described herein, instead of a displacement actuator (or set of actuators) for each emissive microdisplay. Exemplary Emissive Microdisplay Systems

[0351] As discussed herein, various parts of the optical projection system can be moved to provide a desired shift in the position of the displayed pixel, and this movement can be achieved using actuators that are mechanically connected to the parts to be moved.

[0352] Figure 36A illustrates an example of a wearable display system having an optical projection system with an actuator coupled to the projection optics. The optical projection system 1010 and the actuator 1504 may together be referred to as the projection system 1500. It should be understood that the optical projection system 1010 of the projection system 1500 may have any of the various configurations disclosed herein (e.g., as shown and discussed with respect to Figures 11A, 12-14, and 24A). To the extent that microlenses, microreflectors, or gratings are utilized with the light emitters of the projection system 1500 (e.g., as shown in Figures 24B, 26A-26C, and 29-30C), the microlenses, microreflectors, or gratings are preferably configured to provide an effective pixel size that is less than the pixel pitch to enable a sufficiently sparse array to facilitate the positional shifting described herein. Additionally, as discussed herein, actuator 1504, or actuators 1504a-1504c each may include or otherwise represent two actuators that may move along different axes.

[0353] 36A , an example of an actuator is a piezoelectric actuator. The actuator 1504 may adjust the position of the projection optics 1070 along one or more axes in a plane (e.g., a plane parallel to the plane on which the eyepiece 1020 is disposed), as described herein. For example, the actuator 1504 may move the projection optics 1070 along two intersecting axes in that plane (e.g., using a two-dimensional piezoelectric motor). As shown, the projection optics outputs light from the emissive microdisplays 1030a-1030c to the waveguide-interconnecting optical elements 1022a-1022c of the eyepiece 1020.

[0354] The optical projection system 1010 may utilize monochrome emissive microdisplays 1030a, 1030b, and 1030c, each configured to output a different primary color. An optical combiner 1050, such as a dichroic x-cube, may redirect light emitted from the emissive microdisplays 1030a-1030c to the projection optics 1070, as described above.

[0355] In some embodiments, the projection optics 1070 are configured to receive image light from the emissive microdisplays 1030a-1030c, and the actuators 1504 are configured to move the projection optics 1070, which in turn shifts the image light output by the optical projection system 1500. Thus, the pixels presented by the array may be perceived as being adjusted in location, e.g., to tile sub-frames across the inter-emitter area, as described herein, and the emissive microdisplays 1030a-1030c may output light corresponding to multiple sub-frames. These sub-frames may be presented in rapid succession (within flicker fusion thresholds) such that a user may perceive them as being presented simultaneously within a full-resolution frame of virtual content.

[0356] In some embodiments, one or more of the emissive microdisplays 1030a-1030c are independently movable relative to the others of the emissive microdisplays, the optical combiner 1050, and the projection optics 1070. In such embodiments, each independently movable microdisplay may have an associated independently movable actuator 1504. FIG. 36B illustrates an example of a wearable display system having an optical projection system 1500 with multiple actuators 1504a-1504c, each coupled to a different emissive microdisplay 1030a-1030c. The actuators 1504a-1504c may thus shift their associated primary color emissive microdisplays 1030a-1030c. This embodiment may allow each primary color emissive microdisplay 1030a-1030c to be shifted to an individual position and output identical subframes such that the subframes overlap within the user's eye. In some embodiments, the primary color emissive microdisplays 1030a-1030c may be shifted along different paths.

[0357] Referring again to FIGS. 36A-36B , the actuators 1504 or 1504a-1504c may be controlled via one or more processing elements included within the display system. Additionally, the output of image light by the emissive microdisplays 1030a-1030c may be synchronized with the movement of the actuators 1504 or 1504a-1504c. For example, the emissive microdisplays 1030a-1030c may output light based on a signal or command to the actuators 1504 or 1504a-1504c indicating that the portion of the optical projection system 1500 moved by the actuator has been shifted to one or more positions. This signal or command may be utilized for a continuous movement pattern or a discrete movement pattern, as described above. The signal or command may be generated by the display system, such as one or more processors or processing elements. In some embodiments, the optical projection system 1500 is part of the display system 60 (FIG. 9E), and the actuator 1504 and control elements for the emissive microdisplays 1030a-1030c may be part of the processing module 140 or 150 (FIG. 9E).

[0358] In some embodiments, the actuator 1504 or 1504a-1504c may continuously move mechanically coupled portions of the optical projection system 1500, for example, according to the movement pattern illustrated in FIG. 34, and the emissive microdisplays 1030a-1030c may periodically generate light. For a continuous movement pattern, the emissive microdisplays 1030a-1030c may be synchronized with a signal (e.g., a clock signal) also utilized by the actuators 1504 and / or 1504a-1504c, thereby presenting sub-frames at their desired locations. For example, the actuator 1504 may shift the projection optics 1070 according to a known movement pattern (e.g., at a known rate based on the clock signal). Thus, the emissive microdisplays 1030a-1030c may utilize the signal to identify the extent to which the projection optics 1070 is being moved along the movement pattern. The emissive microdisplays 1030a-1030c may then output light corresponding to the new sub-frame, for example, when the projection optics 1070 is in a position associated with the new sub-frame.

[0359] In some embodiments, as discussed herein, time division multiplexing may be utilized for the microdisplays 1030a, 1030b, 1030c. For example, different ones of the emissive microdisplays 1030a, 1030b, 1030c may be activated at different times to generate different primary color images.

[0360] In some embodiments, the actuators 1504a-1504c may be moved to complete at least one movement loop (e.g., the loop of movement paths 1300-1306 in FIG. 34 ), with only a single one of the emissive microdisplays 1030a-1030c generating subframes of a single primary color during that loop. In some embodiments, after completing one loop, subframes of a second primary color are generated by a second one of the microdisplays during a second loop of actuator movement, and after completing that second loop, subframes of a third primary color are generated by a third one of the microdisplays during a third loop of actuator movement. Thus, each loop of actuator movement generates a tiled set of subframes, for a total of three sets of loops and subframes if there are three primary colors (the number of sets equals the number of primaries). Preferably, the completion of the set of subframes of each primary color is completed within the flicker fusion threshold.

[0361] While the eyepiece 1020 is illustrated in Figures 36A-36B as including a stack of waveguides, it should be understood that the eyepiece 1020 may include a single waveguide in some embodiments as disclosed herein. Figure 37A illustrates an example of a wearable display system having an optical projection system 1500 with an eyepiece 1020 having a single waveguide 1020a. The illustrated single-waveguide eyepiece 1020 may be similar to those illustrated and discussed with respect to Figures 13B, 14, 30A, and 31A-31B.

[0362] Additionally, as discussed herein, a single microdisplay may emit light of two or more (e.g., all) primary colors (e.g., emitting red, green, and blue light). For example, FIG. 14 illustrates an example of a wearable display system with a single full-color emissive microdisplay 1030b that may emit light of each primary color. In some embodiments, such a microdisplay and / or associated projection optics may be shifted to present different pixels of an image, as discussed herein.

[0363] 37B illustrates an example of a wearable display system having a light projection system 1010 in which a single array of light emitters outputs light of one or more different primary colors. As shown, the light projection system 1010 may include an emissive microdisplay 1030b that includes an array 3742 of light emitters 3744. In some embodiments, the array 3742 may be configured to emit light of all primary colors utilized by the display system to form a full-color image. For example, the microdisplay 1030b may include an array of light emitters that generate red, green, and blue light, respectively.

[0364] Preferably, each primary color for a given pixel is emitted from overlapping areas of array 3742, which may advantageously facilitate the shifting described herein to provide different pixels of an image. For example, light emitters 3744 may each be understood to include a stack of component light generators, each configured to emit light of a different associated primary color. Microdisplay 1030b, in some embodiments, may include coaxial red, green, and blue stacked component light generators.

[0365] Advantageously, with continued reference to FIG. 37B, by emitting each primary color, microdisplay 1030b may therefore avoid the use of an optical combiner, such as optical combiner 1050 described herein. Light (e.g., multi-component light) from microdisplay 1030b may be routed through projection optics 1070 to eyepiece 1020. As described above with respect to at least FIG. 36A, actuator 1504 may adjust the position of the projection optics to form different pixels of the image. In some embodiments, a single waveguide eyepiece 1020 may be used to receive light (e.g., via incoupling optical element 1122a configured to incoupling incoming light of each of the primary colors).

[0366] 37B illustrates an actuator 1504 adjusting the position of the projection optics 1070, although it should be understood that the actuator 1504 may additionally or alternatively be attached to the microdisplay 1030b to adjust the position of the microdisplay 1030b and shift the microdisplay 1030b to provide different pixels as discussed herein. FIG. 37C illustrates a wearable display system similar to that of FIG. 37B, but in which the actuator 1504 is attached to the microdisplay 1030b instead of the projection optics 1070.

[0367] As described above, in some embodiments in which different microdisplays are utilized to generate light of different primary colors, the projection system may use an optical combiner to combine the separately generated light of different colors. For example, an x-cube may be employed to combine light from the different microdisplays 1030a-1030c (FIG. 36B). The combined light may be routed through projection optics 1070 and directed to the eyepiece 1020, which may include one or more waveguides.

[0368] In some embodiments, as illustrated in Figures 38A-38D, even when different microdisplays are utilized to generate light of different primary colors, the optical combiner may be omitted from projection system 1500. For example, microdisplays 1030a-1030c may each route light to the eyepiece 1020 through a dedicated associated one of projection optics 1070a-1070c. As shown, microdisplay 1030a has associated projection optics 1070a that focuses light onto associated incoupling optics 1022a, microdisplay 1030b has associated projection optics 1070b that focuses light onto associated incoupling optics 1022b, and microdisplay 1030c has associated projection optics 1070c that focuses light onto associated incoupling optics 1022c.

[0369] It should be understood that in embodiments in which the optical combiner 1500 is not used, several exemplary advantages may be achieved. As an example, when the intervening optical combiner 1500 is omitted, improved light collection may occur because the microdisplays 1030a-1030c may be located closer to the projection optics 1070a-1070c. As a result, higher light utilization efficiency and image brightness may be achieved. As another example, the projection system 1500 may be simplified and tuned to specific primary color light. For example, the optical system design for each individual projection optics 1070a-1070c may be separately calibrated for each primary color light generated by the microdisplays 1030a-1030c. In this manner, the projection system 1500 may avoid the need for achromatization of the projection optics.

[0370] As another exemplary advantage, as illustrated in FIG. 38A , light from each of the projection optics 1070a-1070c can advantageously be more uniquely focused onto a respective associated in-coupling optical element 1022a-1022c. With reference to FIGS. 36A-36B , the combined light is routed through the projection optics 1070 onto the eyepiece 1020. As illustrated, the light may be in-coupling through different in-coupling optical elements 1022a-1022c. In the example of FIGS. 36A-36B , the eyepiece 1020 includes three example waveguides that in-coupling the respective primary colors generated by the microdisplays 1030a-1030c. However, it should be understood that each primary color need not be precisely focused onto a respective in-coupling element 1022a-1022c of the eyepiece 1020. As a non-limiting example, Figures 36A-36B illustrate the combined light focusing at a depth between internal coupling elements 1022b and 1022c.

[0371] 38A-38B allows for more precise focusing of each primary color onto the respective internal coupling elements 1022a-1022c. The projection optics 1070a-1070c for each primary color may be configured to precisely focus the light onto the respective internal coupling elements 1022a-1022c. In some embodiments, this precise focusing may improve image quality by providing a clearly focused image of each primary color.

[0372] FIG. 38A illustrates an example of an optical projection system 1500 without an optical combiner (e.g., the optical combiner 1050 described above). In the example shown, three microdisplays 1030a-1030c provide light (e.g., primary color light) to individual projection optics 1070a-1070c. The projection optics 1070a-1070c may be connected to or otherwise positioned along a connecting element 3802. The connecting element 3802 may be positionally adjusted by an actuator 1504. Thus, the actuator 1504 may adjust the position of the projection optics 1070a-1070c, which form an integrated structure. Light from each microdisplay 1030a-1030c may be routed through the projection optics 1070a-1070c and focused onto individual internal combining elements 1022a-1022c included within the eyepiece 1020.

[0373] 38A illustrates actuators 1504 adjusting the positions of the projection optical systems 1070a-1070c via connecting elements 3802. In some embodiments, each projection optical system 1070a-1070c may include its own dedicated actuator. For example, projection system 1500 may include three actuators to adjust the positions of individual ones of the three projection optical systems 1070a-1070c.

[0374] FIG. 38B illustrates another example of a wearable display system having a light projection system without an optical combiner. In some embodiments, the microdisplays 1030a-1030c may form a single integrated unit, e.g., the microdisplays 1030a-1030c are mounted on a single backplane 3804. In some embodiments, the backplane 3804 may be a silicon backplane, which may include electrical components for the microdisplays 1030a-1030c. Similar to FIG. 38A, the illustrated actuator 1504 may adjust the position of the connecting element 3802.

[0375] It should be understood that the actuators of Figures 38A-B may be attached to and configured to move the microdisplays 1030a-1030c rather than the projection optics 1070a-1070c. For example, Figure 38C illustrates a wearable display system that is otherwise similar to the wearable display system of Figure 38A, but in which the actuator 1504 attached to the projection optics 1070a-1070c is omitted and each of the microdisplays 1030a-1030c instead has an associated actuator 1504a-1504c, respectively. In some embodiments in which the microdisplays 1030a-1030c are joined together (e.g., two or more of the microdisplays are physically connected, e.g., by sharing a common backplane), a single actuator 1504 may be utilized to change the position of the physically connected ones of the microdisplays 1030a-1030c. For example, FIG. 38D illustrates a wearable display system that is otherwise similar to the wearable display system of FIG. 38B, but in which the actuator 1504 attached to the projection optics 1070a-1070c is omitted, and the actuator 1504 is instead attached to the physically coupled microdisplays 1030a-1030c so as to move these microdisplays together.

[0376] In the above description with respect to at least Figures 36A-38D, one or more actuators are described as causing displacement or movement of different components of the wearable display system. In each of these figures, the one or more actuators are illustrated as moving the same type of component (e.g., the microdisplay or the projection optics) for ease of illustration and discussion, although in some embodiments, an actuator may be provided to adjust the position of more than one type of component (e.g., in the same display system, an actuator may be mounted and configured to adjust the position of both the microdisplay and the projection optics illustrated in these figures).

[0377] For example, FIG. 36A illustrates an actuator 1504 for adjusting the position of the projection optics 1070, and FIG. 36B illustrates actuators 1504a-1504c for adjusting the positions of the microdisplays 1030a-1030c, respectively. In some embodiments, the wearable display system may include both the actuator 1504 and the actuators 1504a-1504c, where the actuator 1504 is configured to adjust the position of the projection optics 1070, while the actuators 1504a-1504c are configured to adjust the positions of the microdisplays 1030a-1030c. For example, the position of the projection optics 1070 may be adjusted simultaneously with the positions of the microdisplays 1030a-1030c. As another example, the positions of the projection optics 1070 and the microdisplays 1030a-1030c may be adjusted at different times (e.g., sequentially).

[0378] 37B and 37C, in some embodiments, both the microdisplay 1030b and the projection optics 1070 may have associated actuators 1504 for moving the microdisplay 1030b and the projection optics 1070, respectively (e.g., for moving these components simultaneously or at different times). In some embodiments, the same display system may include and use both actuators 1504 (illustrated in FIG. 38A) and actuators 1504a-1504c (illustrated in FIG. 38C) for adjusting the positions of the projection optics 1070a-1070c and the microdisplays 1030a-1030c, respectively. Referring to FIG. 38B and 38D, in some embodiments, the same display system may include the projection optics 1070a-1070c with a first associated actuator 1504, and each of the coupled microdisplays 1030a-1030c may have a second associated actuator 1504. (Example Flowchart)

[0379] 39 illustrates a flowchart of an exemplary process for outputting sub-frames of a rendered frame of virtual content. For convenience, the process will be described as being performed by a display system having one or more processors (e.g., within local processing and data module 140 or remote processing module 150 of FIG. 9E).

[0380] In block 3902, the display system obtains a rendered frame of the virtual content. As described above, the display system may generate the frame of the virtual content for presentation to the user. For example, the local processor and data module 140 may include one or more graphics processing elements. The module 140 may then generate the rendered frame of the virtual content.

[0381] As illustrated in Figures 32-36, a rendered frame may be rendered, at least in part, at a resolution (e.g., pixel density) that exceeds that of a light emitter (e.g., micro-LED) included within a microdisplay of the optical projection system. Each sub-frame may be formed based on light generated by the light emitter. Each set of sub-frames to form a full-resolution frame may be output consecutively in rapid succession (e.g., within a flicker fusion threshold) such that they may be perceived by a user as being presented simultaneously to form a frame rendered at full resolution.

[0382] In block 3904, the display system outputs light that forms a first sub-frame. The display system may select pixels included in the rendered frame as forming the first sub-frame. For example, as illustrated in FIGS. 32A-32D, there may be a threshold number of sub-frames (e.g., nine sub-frames). Thus, the display system may select pixels that correspond to the first of these threshold number of sub-frames. For example, the display system may divide the rendered frame into a threshold number of sub-frames. The display system may then store these sub-frames and cause emissive elements to generate light that forms each sub-frame. Optionally, module 140 may be configured to generate sub-frames instead of complete rendered frames. The display system then causes the light projection system to output light that forms the first sub-frame. It should be understood that pixels of various sub-frames may be interleaved or intermixed such that pixels of different sub-frames may occupy spaces separating pixels of other sub-frames.

[0383] In block 3906, the display system shifts the position of the displayed pixel. As illustrated in Figures 35A-35B, the pixel position may be perceived as being adjusted via an actuator that moves a portion of the optical projection system, changing the path of light through the optical projection system. For example, one or more actuators may adjust the position of one or more emissive microdisplays included in the optical projection system. As another example, one or more actuators may adjust the position of projection optics included in the optical projection system. These adjustments may cause the position of light from the emissive elements to be adjusted.

[0384] As described above, the display system may continuously move portions of the optical projection system. For example, an actuator may follow a movement pattern such as the movement pattern illustrated in FIG. 34. The display system may also cause discrete adjustments of the optical projection system via one or more actuators. For example, the positions of various portions of the optical projection system may be discretely adjusted every subframe.

[0385] In block 3908, the display system outputs light that forms a second sub-frame after changing the position of the movable portion of the light projection system so that the light provides pixels in desired locations for the second sub-frame. The display system may select pixels of the rendered frame that form the second sub-frame. Optionally, module 140 may render the second sub-frame. The display system may then cause the light projection system to output light that forms the second sub-frame.

[0386] The display system may then continue to shift the position of successive sub-frames that are displayed and output until a complete rendered frame is formed.

[0387] Various exemplary embodiments of the present invention are described herein. Reference is made to these examples in a non-limiting sense. They are provided to illustrate the broader and more applicable aspects of the present invention. Various modifications may be made to the invention described, and equivalents may be substituted without departing from the true spirit and scope of the invention.

[0388] For example, although advantageously utilized in conjunction with an AR display that provides images across multiple depth planes, the virtual content disclosed herein may also be displayed by a system that provides images on a single depth plane.

[0389] In addition, many modifications may be made to adapt a particular situation, material, composition of matter, process, process acts, or steps to the objective, spirit, or scope of the present invention. Moreover, as will be understood by those skilled in the art, each of the individual variations described and illustrated herein has discrete components and features that can be readily separated from or combined with the features of any of the other several embodiments without departing from the scope or spirit of the present invention. All such modifications are intended to be within the scope of the claims associated with this disclosure.

[0390] The present invention includes methods that may be implemented using the subject devices. The methods may include the act of providing such a suitable device. Such provisioning may be performed by an end user. In other words, the act of "providing" merely requires the user to obtain, access, approach, locate, configure, activate, power on, or otherwise act to provide the device required in the subject methods. The methods recited herein may occur in any order of the recited events and the recited sequence of events that is logically possible.

[0391] In addition, it should be understood that each of the processes, methods, and algorithms described herein and / or depicted in the figures may be embodied in code modules executed by one or more physical computing systems, hardware computer processors, application-specific circuits, and / or electronic hardware configured to execute specific and particular computer instructions, and thereby may be fully or partially automated. For example, a computing system may include a general-purpose computer (e.g., a server) or a special-purpose computer programmed with specific computer instructions, special-purpose circuitry, etc. Code modules may be compiled and linked into an executable program, installed within a dynamic link library, or written in an interpreted programming language. In some implementations, particular operations and methods may be performed by circuitry specific to a given function.

[0392] Furthermore, certain implementations of the functionality of the present disclosure may be sufficiently mathematically, computationally, or technically complex that special-purpose hardware (utilizing appropriate specialized executable instructions) or one or more physical computing devices may be required to perform the functionality, e.g., due to the amount or complexity of the calculations involved or to provide results in substantially real time. For example, a video may contain many frames, each frame may have millions of pixels, and specifically programmed computer hardware may be required to process the video data to provide the desired image processing task or application in a commercially reasonable amount of time.

[0393] Code modules or any type of data may be stored on any type of non-transitory computer-readable medium, such as physical computer storage, including hard drives, solid-state memory, random access memory (RAM), read-only memory (ROM), optical disks, volatile or non-volatile storage devices, combinations of the same, and / or the like. In some embodiments, the non-transitory computer-readable medium may be part of one or more of the local processing and data module (140), the remote processing module (150), and the remote data repository (160). The methods and modules (or data) may also be transmitted as a data signal generated over various computer-readable transmission media, including wireless-based and wired / cable-based media, (e.g., as part of a carrier wave or other analog or digital propagated signal), and may take various forms (e.g., as part of a single or multiplexed analog signal, or as multiple discrete digital packets or frames). The results of the disclosed processes or process steps may be stored, persistently or otherwise, in any type of non-transitory tangible computer storage device or communicated via a computer-readable transmission medium.

[0394] Any process, block, state, step, or functionality described herein and / or depicted in the accompanying figures should be understood as potentially representing a code module, segment, or portion of code, comprising one or more executable instructions for implementing a specific function (e.g., logical or arithmetic) or step in a process. Various processes, blocks, states, steps, or functionality may be combined, rearranged, added to, deleted from, modified, or otherwise altered from the illustrative examples provided herein. In some embodiments, additional or different computing systems or code modules may perform some or all of the functionality described herein. The methods and processes described herein are also not limited to any particular sequence, and the blocks, steps, or states associated therewith may be performed in other sequences, e.g., serially, in parallel, or in some other manner, as appropriate. Tasks or events may be added to or removed from the disclosed exemplary embodiments. Furthermore, the separation of various system components in the embodiments described herein is for illustrative purposes and should not be understood as requiring such separation in all embodiments. It should be understood that the program components, methods, and systems described may generally be integrated together in a single computer product or packaged into multiple computer products.

[0395] Exemplary aspects of the invention have been described above, along with details regarding material selection and manufacturing. As for other details of the invention, these may be understood in connection with the above-referenced patents and publications and are generally known or may be understood by those skilled in the art. The same may be true with respect to method-based embodiments of the invention in terms of additional acts as commonly or logically adopted.

[0396] Additionally, while the present invention has been described with reference to several embodiments, optionally incorporating various features, the present invention is not limited to those described and indicated as being contemplated with respect to each variation of the invention. Various modifications may be made to the invention as described, and equivalents (whether recited herein or not included for purposes of brevity to some extent) may be substituted without departing from the true spirit and scope of the invention. Additionally, when a range of values ​​is provided, it is understood that all intervening values ​​between the upper and lower limits of that range, and any other stated value or intervening value within the stated range, are encompassed within the invention.

[0397] It is also contemplated that any optional features of the described inventive variations may be set forth and claimed independently or in combination with any one or more of the features described herein. Reference to a singular item includes the possibility that multiple identical items are present. More specifically, as used in this specification and the claims associated herewith, the singular forms "a," "an," "said," and "the" include plural references unless specifically stated otherwise. In other words, the use of articles allows for "at least one" of the items of the present subject matter in the above description and in the claims associated wit...

Claims

[Claim 1] A display system as shown in the drawings.

Images