Variable pixel density display system with a mechanically actuated image projector

Abstract

To provide a suitable variable pixel density display system.SOLUTION: A head-mounted virtual and augmented reality display system includes a light projector with one or more light-emissive type micro-displays having a first resolution and a pixel pitch. The projector outputs light that forms a frame of a virtual content having at least a portion associated with a second resolution exceeding the first resolution. A projector outputs light that forms a first sub-frame of a rendered frame at the first resolution, and a portion of the projector shifts using an actuator so that a physical position of the light outputted for an individual pixel occupies a gap between the old places of the light outputted for an individual pixel. Then, the projector outputs light that forms a second sub-frame of a rendered frame. The first and second sub-frames are outputted within the flicker fusion threshold.SELECTED DRAWING: Figure 36A

Description

Technical Field

[0001] (Claim of Priority) This application claims priority to U.S. Provisional Application No. 62 / 911,018, filed October 4, 2019, entitled "AUGMENTED AND VIRTUAL REALITY DISPLAY SYSTEMS WITH SHARED DISPLAY FOR LEFT AND RIGHT EYES"; U.S. Provisional Application No. 62 / 800,363, filed February 1, 2019, entitled "VIRTUAL AND AUGMENTED REALITY DISPLAY SYSTEMS WITH EMISSIVE MICRO-DISPLAYS"; and U.S. Provisional Application No. 62 / 786,199, filed December 28, 2018, entitled "LOW MOTION-TO-PHOTON LATENCY ARCHITECTURE FOR AUGMENTED AND VIRTUAL REALITY DISPLAY SYSTEMS". The above applications are hereby incorporated by reference in their entireties. (Incorporation by Reference)

[0002] This application incorporates by reference in its entirety each of the following: U.S. Patent Application No. 14 / 555,585, filed November 27, 2014 and published as U.S. Patent Publication No. 2015 / 0205126 on July 23, 2015; U.S. Patent Application No. 14 / 690,401, filed April 18, 2015 and published as U.S. Patent Publication No. 2015 / 0302652 on October 22, 2015; U.S. Patent Application No. 14 / 212,961, filed March 14, 2014 and issued as U.S. Patent No. 9,417,452 on August 16, 2016; U.S. Patent Application No. 14 / 331,218, filed July 14, 2014 and published as U.S. Patent Publication No. 2015 / 0309263 on October 29, 2015; U.S. Patent Application Publication No. 2018 / 0061121, published March 1, 2018; U.S. Patent Application No. 16 / 221,065, filed December 14, 2018; U.S. Patent Application Publication No. 2018 / 0275410, published September 27, 2018; U.S. Provisional Application No. 62 / 786,199, filed December 28, 2018; U.S. Patent Application No. 16 / 221,359, filed December 14, 2018; U.S. Provisional Application No. 62 / 702,707, filed July 24, 2018; U.S. Patent Application No. 15 / 481,255, filed April 6, 2017; and U.S. Patent Application No. 15 / 927,808, filed April 21, 2018 and published as U.S. Patent Application Publication No. 2018 / 0275410 on September 27, 2018. (Technical Field)

[0003] The present disclosure relates to display systems, and more particularly, to augmented and virtual reality display systems. [Background Art]

[0004] Modern computing and display technologies have facilitated the development of systems for so-called "virtual reality" or "augmented reality" experiences, in which digitally reproduced images or portions thereof are presented to a user in a manner that appears or can be perceived as being real. Virtual reality, i.e., the "VR" scenario, typically involves the presentation of digital or virtual image information without transparency to other actual real-world visual inputs, and augmented reality, i.e., the "AR" scenario, typically involves the presentation of digital or virtual image information as an augmentation to the visualization of the actual world surrounding the user. Mixed reality or "MR" scenarios are a type of AR scenario that typically involves virtual objects integrated into and responsive to the natural world. For example, an MR scenario may include AR image content that appears to be blocked by or otherwise interact with objects within the real world.

[0005] Referring to FIG. 1, an AR scene 10 is depicted. To a user of AR technology, a real-world park-like setting 20 is visible, featuring people, trees, buildings in the background, and a concrete platform 30. The user also "sees" "virtual content" such as a robot figure 40 standing on the real-world platform 30 and an avatar character 50 in the form of a flying cartoon that appears anthropomorphic like a bumblebee. These elements 50, 40 are "virtual" in that they do not exist in the real world. The human visual perception system is complex, and it is difficult to produce AR technology that facilitates a comfortable, natural, and rich presentation of virtual image elements among other virtual or real-world image elements. SUMMARY OF THE INVENTION MEANS FOR SOLVING THE PROBLEM

[0006] According to some embodiments, a head-mounted display system includes a support structure configured to be mounted on a user's head, a light projection system supported by the support structure, an eyepiece lens, and one or more processors. The light projection system includes a microdisplay including an array of light emitters associated with a first resolution, the array of light emitters being configured to output light to form a frame of virtual content, a projection optical system, and one or more actuators. The eyepiece lens is supported by the support structure and is configured to receive light from the light projection system and direct the received light to the user. The one or more processors are configured to receive a rendered frame of virtual content, the rendered frame including at least a portion associated with a second resolution, the second resolution being higher than the first resolution. The one or more processors are further configured to cause the emissive microdisplay projector to output light to form a first sub-frame of the rendered frame, the first sub-frame and the rendered frame being configured to be substantially the same size. The one or more processors are further configured to adjust a position associated with the light emitter light output from the light projection system by offsetting one or more movable parts of the light projection system via the one or more actuators, and cause the light projection system to output light to form a second sub-frame of the rendered frame.

[0007] According to some other embodiments, a method implemented by a head-mounted display system of one or more processors includes providing a frame in which virtual content is rendered, the frame including at least a portion associated with a second resolution. A light-emitting microdisplay projector is caused to output light to form a first sub-frame of the rendered frame, the first sub-frame having a first resolution less than the second resolution, the light-emitting microdisplay projector including an array of light emitters associated with the first resolution and having a certain pixel pitch. The light-emitting microdisplay projector is displaced via one or more actuators to adjust a geometric position associated with the light output by the light-emitting microdisplay projector, the geometric position being adjusted by a distance less than the pixel pitch. The light-emitting microdisplay projector is caused to output light to form a second sub-frame of the rendered frame, the second sub-frame having the first resolution.

[0008] According to yet another embodiment, the system comprises one or more processors and one or more computer storage media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations. The operations include generating a frame of virtual content to be displayed as augmented reality content via a light-emitting microdisplay projector system of the system, wherein the frame to be rendered is associated with a second resolution and the light-emitting microdisplay projector is configured to output light to form virtual content that is associated with a first, lower resolution. The frame of virtual content to be rendered is divided into a plurality of sub-frames, each sub-frame including a subset of the pixels included in the frame to be rendered. The light is output continuously via the light-emitting microdisplay projector system, the light forming a plurality of sub-frames, and the light-emitting microdisplay projector system is displaced according to a certain movement pattern for each sub-frame via one or more actuators, and the light-emitting microdisplay projector system is displaced on a plane parallel to the plane of the output pupil of the projector system along one or more axes.

[0009] According to some other embodiments, a method implemented by a head-mounted display system of one or more processors includes generating a frame of virtual content to be displayed as virtual content via a light-emitting microdisplay projector system of the head-mounted display system, the frame to be rendered being associated with a second resolution and the light-emitting microdisplay projector being associated with a first, lower resolution, the method including the step of providing an emitter configured to output light to form the virtual content. The frame of virtual content to be rendered is divided into a plurality of sub-frames, each sub-frame including a subset of the pixels included in the frame to be rendered. The light is output continuously via the light-emitting microdisplay projector system, the light forming the plurality of sub-frames, and the light-emitting microdisplay projector system is displaced according to a movement pattern for each sub-frame via one or more actuators along one or more axes, the light-emitting microdisplay projector system being displaced on a plane parallel to the plane of the output pupil of the projector system along one or more axes.

[0010] Some additional examples are provided below.

[0011] (Example 1) A head-mounted display system, comprising: a support structure configured to be mounted on a user's head; a microdisplay supported by the support structure and comprising an array of light emitters associated with a first resolution, the array of light emitters being configured to output light to form a frame of virtual content; a light projection system comprising a projection optical system and one or more actuators; an eyepiece supported by the support structure, configured to receive light from the light projection system and direct the received light towards the user; and one or more processors, at least a portion of which receive a rendered frame of virtual content associated with a second resolution, the second resolution being higher than the first resolution, the one or more processors being configured to cause the emissive microdisplay projector to output light to form a first sub-frame of the rendered frame, the first sub-frame and the rendered frame being substantially the same size, and to adjust the position associated with the light emitter light output from the light projection system by offsetting one or more movable parts of the light projection system via the one or more actuators, and to cause the light projection system to output light to form a second sub-frame of the rendered frame.

[0012] (Example 2) The head-mounted display according to Example 1, wherein the portion associated with the second resolution is associated with the foveal region of the user's eye.

[0013] (Example 3) The head-mounted display according to Example 2, wherein the one or more processors are configured to determine that the light forming the portion is within a threshold angular distance of the user's fovea.

[0014] (Example 4) One or more processors are configured to cause a light emitter to update light emitted to form a portion with respect to a second sub-frame, and not to update light emitted to form a portion of a rendered frame outside the portion with respect to a first sub-frame, for the head-mounted display according to Example 2.

[0015] (Example 5) Each light-emitting microdisplay array has an associated emitter size, and the emitter size is less than the pixel pitch, for the head-mounted display system according to Example 1.

[0016] (Example 6) The total number of sub-frames of a rendered frame is determined based on a size associated with the pixel pitch and the emitter size, for the head-mounted display according to Example 5.

[0017] (Example 7) One or more processors are configured to cause a light projection system to continuously output light that forms the total number of sub-frames, for the head-mounted display according to Example 6.

[0018] (Example 8) One or more processors are configured to multiplex a rendered frame in time by causing one or more actuators to shift a part of a light projection system for each sub-frame, for the head-mounted display according to Example 7.

[0019] (Example 9) One or more processors are configured to cause one or more actuators to shift a part of a light projection system such that geometric positions associated with an array of light emitters are tiled within an inter-emitter region, for the head-mounted display according to Example 8.

[0020] (Example 10) One or more processors are configured to cause one or more actuators to displace a portion of the light projection system according to a movement pattern, the movement pattern being a continuous movement pattern, the head-mounted display according to Example 1.

[0021] (Example 11) The first sub-frame and the second sub-frame each comprise pixels associated with an individual portion of the frame to be rendered, the head-mounted display according to Example 1.

[0022] (Example 12) The light projection system comprises a plurality of arrays of light emitters, the head-mounted display according to Example 1.

[0023] (Example 13) Further comprising an X-cube prism, each array of light emitters facing a different side of the X-cube prism, the head-mounted display according to Example 12.

[0024] (Example 14) Each array of light emitters is configured to direct light into a dedicated associated projection optical system, the head-mounted display according to Example 12.

[0025] (Example 15) The arrays of light emitters are mounted on a common back plane, the head-mounted display according to Example 12.

[0026] (Example 16) One or more actuators are configured to displace the projection optical system, the head-mounted display according to Example 1.

[0027] (Example 17) One or more actuators are piezoelectric motors, the head-mounted display according to Example 1.

[0028] (Example 18) One or more actuators displace the light-emitting microdisplay projector along two axes, the head-mounted display according to Example 1.

[0029] (Example 19) The light emitter comprises a light-emitting diode, the head-mounted display according to Example 1.

[0030] (Example 20) The array of light emitters is configured to emit light of a plurality of primary colors, the head-mounted display according to Example 1.

[0031] (Example 21) Each light emitter comprises a stack of component light generators, each component light generator emitting light of a different color, the head-mounted display according to Example 20.

[0032] (Example 22) The eyepiece comprises one or more waveguides, a waveguide assembly, each waveguide comprising an internal coupling optical element configured to internally couple light from the microdisplay into the waveguide, and an external coupling optical element configured to externally couple the internally coupled light out of the waveguide, the head-mounted display according to Example 1.

[0033] (Example 23) A method implemented by a head-mounted display system of one or more processors, the method comprising: providing a frame for rendering virtual content, the frame comprising at least a portion associated with a second resolution; causing a light-emitting microdisplay projector to output light to form a first sub-frame of the frame to be rendered, the first sub-frame having a first resolution less than the second resolution, the light-emitting microdisplay projector comprising an array of light emitters associated with the first resolution and having a certain pixel pitch; adjusting, via one or more actuators, the geometric position associated with the light output by the light-emitting microdisplay projector by offsetting the light-emitting microdisplay projector, the geometric position being adjusted by a distance less than the pixel pitch; and causing the light-emitting microdisplay projector to output light to form a second sub-frame of the frame to be rendered, the second sub-frame having the first resolution.

[0034] (Example 24) A system comprising: one or more processors; and one or more computer storage media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform steps of: generating a frame of virtual content to be rendered as augmented reality content via a light-emitting microdisplay projector system of the system, the frame to be rendered being associated with a second resolution and the light-emitting microdisplay projector being associated with a first lower resolution, the virtual content being formed by one or more light emitter arrays configured to output light; splitting the frame of virtual content to be rendered into a plurality of sub-frames, each sub-frame including a subset of pixels included within the frame to be rendered; and continuously outputting light to form the plurality of sub-frames via the light-emitting microdisplay projector system, the light-emitting microdisplay projector system being displaced according to a movement pattern for each sub-frame via one or more actuators and the light-emitting microdisplay projector system being displaced on a plane parallel to a plane of an exit pupil of the projector system along one or more axes.

[0035] (Example 25) The system according to Example 24, wherein the one or more processors are configured to displace the light-emitting microdisplay projector via the one or more actuators such that geometric positions associated with the light-emitting microdisplay array are tiled within an inter-emitter region.

[0036] (Example 26) The system according to Example 25, wherein the one or more processors are configured to displace the light emitter array along one or more axes via the one or more actuators.

[0037] (Example 27) The microdisplay projector system described in Example 25 includes a projection optical system, and one or more processors are configured to displace the projection optical system along one or more axes via one or more actuators, and the projection optical system is configured to output light to a user of the system.

[0038] (Example 28) A method implemented by a head-mounted display system of one or more processors, the method comprising: generating a frame of virtual content to be rendered as virtual content via a light-emitting microdisplay projector system of the head-mounted display system, the frame to be rendered being associated with a second resolution and the light-emitting microdisplay projector being associated with a first lower resolution, the light-emitting microdisplay projector being configured to output light to form the virtual content; splitting the frame of virtual content to be rendered into a plurality of sub-frames, each sub-frame including a subset of pixels included within the frame to be rendered; and continuously outputting light via the light-emitting microdisplay projector system, the light forming the plurality of sub-frames, the light-emitting microdisplay projector system being displaced along one or more axes via one or more actuators according to a certain movement pattern for each sub-frame, and the light-emitting microdisplay projector system being displaced on a plane parallel to the plane of the output pupil of the projector system along one or more axes.

[0039] (Example 28) The method described in Example 28, wherein the geometric positions of the light-emitting microdisplay projector system, which is associated with an array of light emitters, are displaced in a tiled manner within the regions between individual emitters.

[0040] (Example 29) One or more actuators displace the light emitter array along one or more axes, according to the method described in Example 29.

[0041] (Example 30) One or more actuators displace the projection optical system of the micro-LED projector system along one or more axes, the projection optical system being configured to output light to a user of the head-mounted display system, according to the method described in Example 29. The present invention provides, for example, the following. (Item 1) A head-mounted display system, A support structure configured to be mounted on a user's head, and An optical projection system supported by the support structure, the optical projection system comprising: A microdisplay including an array of light emitters associated with a first resolution, the array of light emitters being configured to output light forming a frame of virtual content, the microdisplay; and A projection optical system; and One or more actuators An optical projection system comprising: An eyepiece supported by the support structure, the eyepiece being configured to receive light from the optical projection system and direct the received light to the user, the eyepiece; and One or more processors, the one or more processors being configured to: Receive a rendered frame of virtual content, the rendered frame comprising at least a portion associated with a second resolution, the second resolution being higher than the first resolution, and Cause the light-emitting microdisplay projector to output light forming a first sub-frame of the rendered frame, the first sub-frame and the rendered frame being substantially the same size, and Offset one or more movable parts of the light projection system via the one or more actuators to adjust the position associated with the emitter light output from the light projection system, Cause the light projection system to output light that forms a second sub-frame of the rendered frame One or more processors configured to perform A head-mounted display system comprising (Item 2) The portion associated with the second resolution is associated with the foveal region of the user's eye, the head-mounted display according to item 1. (Item 3) The one or more processors are configured to determine that the light forming the portion is within a threshold angular distance of the user's fovea, the head-mounted display according to item 2. (Item 4) The one or more processors With respect to the second sub-frame, update the light emitter to emit light that forms the portion, With respect to the first sub-frame, do not update the light emitter to emit light that forms the portion of the rendered frame outside the portion The head-mounted display according to item 2, configured as (Item 5) Each light-emitting microdisplay array has an associated emitter size, and the emitter size is less than the pixel pitch, the head-mounted display system according to item 1. (Item 6) The total number of sub-frames of the rendered frame is determined based on a size associated with the pixel pitch and the emitter size, the head-mounted display according to item 5. (Item 7) The one or more processors are configured to cause the light projection system to continuously output light that forms the total number of sub-frames, the head-mounted display according to item 6. (Item 8) The one or more processors are configured to time - multiplex the rendered frames by causing the one or more actuators to displace a part of the light projection system for each sub - frame, the head - mounted display according to item 7. (Item 9) The one or more processors are configured to cause the one or more actuators to displace a part of the light projection system such that the geometric positions associated with the array of light emitters are tiled within the inter - emitter regions, the head - mounted display according to item 8. (Item 10) The one or more processors are configured to cause the one or more actuators to displace a part of the light projection system according to a movement pattern, the movement pattern being a continuous movement pattern, the head - mounted display according to item 1. (Item 11) The first sub - frame and the second sub - frame each comprise pixels associated with individual parts of the rendered frame, the head - mounted display according to item 1. (Item 12) The light projection system comprises a plurality of arrays of light emitters, the head - mounted display according to item 1. (Item 13) Further comprising an X - cube prism, each array of light emitters facing a different side of the X - cube prism, the head - mounted display according to item 12. (Item 14) Each array of light emitters is configured to direct light into an associated dedicated projection optical system, the head - mounted display according to item 12. (Item 15) The arrays of light emitters are attached to a common back plane, the head - mounted display according to item 12. (Item 16) The head-mounted display according to item 1, wherein the one or more actuators are configured to displace the projection optical system. (Item 17) The head-mounted display according to item 1, wherein the one or more actuators are piezoelectric motors. (Item 18) The head-mounted display according to item 1, wherein the one or more actuators displace the light-emitting microdisplay projector along two axes. (Item 19) The head-mounted display according to item 1, wherein the light emitter includes a light-emitting diode. (Item 20) The head-mounted display according to item 1, wherein the array of light emitters is configured to emit light of a plurality of primary colors. (Item 21) The head-mounted display according to item 20, wherein each light emitter includes a stack of component light generators, and each component light generator emits light of a different color. (Item 22) The eyepiece includes a waveguide assembly including one or more waveguides, and each waveguide an internal coupling optical element configured to internally couple light from the microdisplay into the waveguide, and an external coupling optical element configured to externally couple the internally coupled light out of the waveguide The head-mounted display according to item 1. (Item 23) A method implemented by a head-mounted display system of one or more processors, the method comprising: providing a frame in which virtual content is rendered, the rendered frame comprising at least a portion associated with a second resolution; Causing the light-emitting microdisplay projector to output light that forms a first sub-frame of the rendered frame, the first sub-frame having a first resolution less than the second resolution, the light-emitting microdisplay projector being associated with the first resolution and comprising an array of light emitters having a pixel pitch, and Shifting the light-emitting microdisplay projector via one or more actuators to adjust a geometric position associated with the light output by the light-emitting microdisplay projector, the geometric position being adjusted to a distance less than the pixel pitch, and Causing the light-emitting microdisplay projector to output light that forms a second sub-frame of the rendered frame, the second sub-frame having the first resolution, and A method comprising. (Item 24) A system, One or more processors, and One or more computer storage media, the one or more computer storage media storing instructions that, when executed by the one or more processors, cause the one or more processors to Generate a rendered frame of virtual content for display as augmented reality content via the light-emitting microdisplay projector system of the system, the rendered frame being associated with a second resolution, the light-emitting microdisplay projector comprising one or more light emitter arrays configured to output light that forms virtual content associated with a first, lower resolution, and Dividing the rendered frame of the virtual content into a plurality of sub-frames, each sub-frame including a subset of the pixels included in the rendered frame, and Outputting light continuously via the light-emitting microdisplay projector system, the light forming the plurality of sub-frames, the light-emitting microdisplay projector system being offset according to a movement pattern for each of the sub-frames via one or more actuators, the light-emitting microdisplay projector system being offset on a plane parallel to the plane of the output pupil of the projector system along one or more axes, and One or more computer storage media for causing an operation including A system comprising (Item 25) The system according to item 24, wherein the one or more processors are configured to offset the light-emitting microdisplay projector via the one or more actuators such that geometric positions associated with the light-emitting microdisplay array are tiled within regions between individual emitters. (Item 26) The system according to item 25, wherein the one or more processors are configured to offset the light emitter array along the one or more axes via the one or more actuators. (Item 27) The system according to item 25, wherein the microdisplay projector system comprises a projection optical system, the one or more processors are configured to offset the projection optical system along the one or more axes via the one or more actuators, and the projection optical system is configured to output light to a user of the system. (Item 28) A method implemented by a head-mounted display system of one or more processors, the method comprising Generating a frame of virtual content to be displayed as virtual content via the light-emitting microdisplay projection system of the head-mounted display system, wherein the rendered frame is associated with a second resolution, and the light-emitting microdisplay projector comprises an emitter configured to output light forming virtual content associated with a first, lower resolution, Dividing the rendered frame of the virtual content into a plurality of sub-frames, each sub-frame including a subset of pixels included in the rendered frame, Continuously outputting light via the light-emitting microdisplay projection system, the light forming a plurality of sub-frames, and the light-emitting microdisplay projection system being offset according to a movement pattern for each sub-frame via one or more actuators along one or more axes, the light-emitting microdisplay projection system being offset on a plane parallel to a plane of an exit pupil of the projection system along one or more axes, A method comprising. (Item 29) The method according to item 28, wherein the geometric positions associated with the light emitter array of the light-emitting microdisplay projection system are offset to be tiled within the inter-emitter regions. (Item 30) The method according to item 29, wherein the one or more actuators offset the light emitter array along the one or more axes. (Item 31) The method according to item 29, wherein the one or more actuators offset a projection optical system of the micro-LED projection system along the one or more axes, the projection optical system being configured to output light to a user of the head-mounted display system.

Brief Description of the Drawings

[0042]

Figure 1

[0043]

Figure 2

[0044]

Figure 3

[0045]

Figure 4A

[0046]

Figure 4B

[0047]

Figure 4C

[0048]

Figure 4D

[0049]

Figure 5

[0050]

Figure 6

[0051]

Figure 7

[0052]

Figure 8

[0053]

Figure 9A

[0054]

Figure 9B

[0055]

Figure 9C

[0056]

Figure 9D

[0057]

Figure 9E

[0058]

Figure 10

[0059]

Figure 11A

[0060]

Figure 11B

[0061]

Figure 12

[0062]

Figure 13A

[0063]

Figure 13B

[0064]

Figure 14

[0065]

Figure 15

[0066]

Figure 16

[0067]

Figure 17

[0068]

Figure 18

[0069]

Figure 19A

[0070]

Figure 19B

[0071]

Figure 20A

[0072]

Figure 20B

[0073]

Figure 21

[0074]

Figure 22A

Figure 22B

Figure 22C

[0075]

Figure 23A

Figure 23B

Figure 23C

[0076]

Figure 24A

[0077]

Figure 24B

[0078]

Figure 25A

[0079]

Figure 25B

[0080]

Figure 26

[0081]

Figure 27

[0082]

Figure 28

[0083]

Figure 29

[0084]

Figure 30A

[0085]

Figure 30B

[0086]

Figure 30C

[0087]

Figure 31

[0088]

Figure 32A

[0089]

Figure 32B

[0090]

Figure 32C

[0091]

Figure 32D

[0092]

Figure 33

[0093]

Figure 34

[0094]

Figure 35

[0095]

Figure 36A

[0096]

Figure 36B

[0097]

Figure 37A

[0098]

Figure 37B

[0099]

Figure 37C

[0100]

Figure 38A

[0101]

Figure 38B

[0102]

Figure 38C

[0103]

Figure 38D

[0104]

Figure 39

[0105] An augmented reality (AR) or virtual reality (VR) system can display virtual content to a user or viewer. This content may be displayed, for example, on a head-mounted display as part of eyewear that projects image information onto the user's eyes. Additionally, if the system is an AR system, the display may also transmit light from the surrounding environment to the user's eyes, enabling a view of the surrounding environment. As used herein, it should be understood that a "head-mounted" or "head-mountable" display is a display that can be mounted on a user's head or viewer.

[0106] To improve the usability of an AR or VR system (also simply referred to as a "display system"), it can be beneficial to reduce the size, weight, and / or power consumption of the display system. As an example, a user may be more likely to utilize the display system if the size and general prominence of the display system are reduced. As another example, a user may be more likely to utilize the display system if the weight placed on the user's head is reduced. Similarly, reduced power consumption can enable the use of a smaller battery and can reduce the heat generated by the display system. The various embodiments described herein promote such advantages, including a reduction in the size of a portion of the display system.

[0107] As described herein, light (also referred to herein as image light) that forms virtual content can be generated by one or more display technologies. For example, the light may be generated by a light projection system included within the display system. This light can then be routed via an optical system to a user of the display system for output as virtual content. The virtual content can be represented as image pixels included within a rendered frame that is continuously presented to the user. To achieve high-quality (e.g., lifelike) virtual content, the display system can render and then output frames of the virtual content at a sufficient resolution (e.g., above a threshold resolution). Thus, the image pixels can be sufficiently close together to achieve a sufficient resolution.

[0108] However, it should be understood that the design constraints associated with a display system can limit the ability to achieve such proximity within an image pixel, and thus the resolution. For example, to miniaturize a display system, the display system may be required to have a reduced display size (e.g., projector size). An exemplary display may include a liquid crystal on silicon (LCoS) display. To output image light that forms virtual content, the LCoS display may be required to utilize a separate illumination module that includes one or more light emitters. In this embodiment, the LCoS panel can impose spatially varying modulation on the light generated to form virtual content. However, to reduce the size associated with the LCoS panel while maintaining high resolution, the pixel pitch associated with the LCoS panel may need to be reduced. Pixel pitch can represent the physical distance on the display between similar locations on similar elements of the display that form image pixels, as described herein. Due to the physical constraints associated with small pixel pitch that are tied to the need for a separate illumination module, the LCoS display can be larger than desired in some applications.

[0109] Some embodiments disclosed herein advantageously include emissive microdisplays such as microLED displays. In some embodiments, the microdisplay is a microOLED display. A display system that utilizes an emissive microdisplay can avoid the added bulk of an illumination module. Additionally, an emissive microdisplay can ostensibly advantageously facilitate the presentation of an image with a small pixel pitch. As described, an exemplary display system may utilize one or more emissive microdisplays to achieve, among other advantages, reduced size, weight, and power consumption.

[0110] Emissive microdisplays have several advantages for use in wearable display systems. As an example, the power consumption of an emissive microdisplay generally varies with the image content such that darker or sparser content requires less power to display. Since the AR environment is often desired to be such that the user can see their surrounding environment, and can thus often be sparse, emissive microdisplays can have an average power consumption that is lower than that of other display technologies that use a spatial light modulator to modulate light from a light source. In contrast, other display technologies can consume substantial power even for dark, sparse, or “all-off” virtual content. As another example, emissive microdisplays can provide a significantly high frame rate (which can enable the use of sub-resolution arrays) and can provide a low level of visually apparent motion artifacts (such as motion blur). As another example, emissive microdisplays may not require the type of polarization optics required by LCoS displays. Thus, emissive microdisplays can avoid the optical losses present within the polarization optics.

[0111] Arrays of light emitters such as microLEDs can provide substantial size, weight, and / or power savings, but current light emitters do not provide a sufficiently small pixel pitch and cannot enable high-resolution virtual content within a small display system form factor. As a non-limiting example, some microLED-based microdisplays can enable a pixel pitch of about 2 to about 3 microns. Even at such a pixel pitch, the microLED display can still undesirably be large for use in a wearable display system, especially since the goal for such a system can be to have a form factor and size similar to that of glasses and thus to provide the desired number of pixels.

[0112] As will be described in more detail, an optical projection system, including a light-emitting microdisplay, can achieve an effective small pixel pitch through high-speed physical adjustment or displacement to a part of the optical projection system. For example, the light-emitting microdisplay may be physically adjusted in position or displaced along one or more axes. As another example, an optical element (e.g., a projection optical system) may be physically adjusted in position or displaced along one or more axes.

[0113] As described herein, the size associated with a light emitter such as a micro-LED may be referred to as the emitter size. The emitter size may refer to the dimension of the light emitter along a particular axis (e.g., a lateral axis). The emitter size may also refer to the dimensions of the light emitter along two axes (e.g., lateral and vertical axes). Similarly, the pixel pitch may refer to the distance between similar points on directly adjacent light emitters along a particular axis (e.g., a lateral axis), and different axes may have their own pixel pitch. For example, in some embodiments, the light emitters may be placed closer along a first axis than along a second axis (e.g., an orthogonal axis). Examples of arrays of light emitters are described in more detail herein and illustrated in FIG. 32A.

[0114] It should be understood that the size of the light emitter may be less than the gap separating directly adjacent light emitters. For example, due to physical and electrical constraints, it may be difficult to form a light-emitting microdisplay with light emitters above a threshold density. Exemplary constraints may include current crowding, substrate droop, etc. Thus, there may be a substantial gap or space between adjacent light emitters. The gap between two light emitters is referred to herein as the inter-emitter region. An example of which is illustrated in FIG. 32A, the inter-emitter region can thus outline the area (e.g., the maximum area) of a light-emitting microdisplay that includes a single light emitter. The size of the inter-emitter region can thus limit the range within which the light-emitting microdisplay can achieve a certain high density or high resolution.

[0115] Advantageously, the ability to operate an optical emitter, such as a micro-LED, at high speed can enable time-multiplexed presentation of images using the same one or more emissive microdisplays. For example, the geometric position of the optical emitter with respect to the projection optics may be offset such that the same optical emitter is enabled to present different pixels of an image at different times. In some embodiments, the frames in which virtual content is rendered may be presented as a series of sub-frames in rapid succession via a time-multiplexing scheme. In this example, each sub-frame may be associated with a particular physical position of the optical emitter with respect to the projection optics. Thus, it should be understood that the geometric position can be varied by changing the locations of the optical emitter and the projection optics relative to each other (e.g., by changing the physical position of the optical emitter while keeping the projection optics stationary, by changing the physical position of the projection optics while keeping the optical emitter stationary, or by changing the physical positions of both the optical emitter and the projection optics). As will be described in further detail below, the geometric position may be adjusted to tile the regions between individual emitters on the optical emitter (e.g., via one or more actuators). Thus, the emissive microdisplay can advantageously achieve the output of high-resolution virtual content.

[0116] Accordingly, the light projection system may be configured to project individual full-resolution frames of virtual content by projecting one or more sub-resolution sub-frames. For example, one or more sub-resolution sub-frames may be projected. The sub-frames may be projected rapidly and continuously and may be offset from each other (e.g., by less than a full pixel pitch along one or more axes along which the sub-frames are translated). For example, a light-emitting microdisplay included within the projector may be physically displaced along one or more axes. As described above, the light-emitting microdisplay may include light emitters such as micro LEDs having a certain pixel pitch. This pixel pitch can thus indicate the resolution at which the light-emitting microdisplay can output frames of virtual content. To substantially reduce the gap between the functional pixel pitch and the light emitters and thus increase the resolution for the same size display, the light emitters may be positioned, for example, by less than the pixel pitch. As an example, the display may include light emitters separated by a pixel pitch of 2.5 microns, and each light emitter may have an emitter size of 0.833 microns. In some embodiments, the light emitters may be positioned a number of times based on the number of times the light emitters can be translated parallel to different (e.g., non-overlapping) positions within the emitter-to-emitter region. In this example, the emitter-to-emitter region may be 6.25 microns 2 and, by way of example, the exemplary light emitters may be positioned three times along a first axis and three times along a second orthogonal axis. Accordingly, the exemplary light emitters may, in effect, take on nine positions within the emitter-to-emitter region. For one or more of the nine positions, a particular sub-frame of the same rendered frame of virtual content may be presented. Accordingly, the sub-frames presented continuously may be perceived as a high-resolution frame of virtual content. In effect, the light emitters may form an image with an apparent pixel density higher than the physical density of the light emitters.

[0117] In some embodiments, the user's visual system may merge the sub - frames together such that the user perceives a full - resolution frame. For example, the pixels of the sub - frames may be combined to form a full - resolution frame. Preferably, the sub - frames may be sequentially displayed at a frame rate higher than the flicker fusion threshold of the human visual system. As an example, the flicker fusion threshold may be 60 Hz, which is considered fast enough such that most users do not perceive the sub - frames as being displayed at different times. In some embodiments, different sub - frames are sequentially displayed at a rate equal to or higher than the flicker fusion threshold (e.g., equal to or higher than 60 Hz).

[0118] As a result, the emissive microdisplay can be configured to have fewer light emitters than the number of image pixels contained within each rendered frame of the virtual content at full resolution. For example, a full - resolution image may include 2,000×2,000 pixels, while the emissive microdisplay may be an array of only 1,000×1,000 elements. The use of a lower - resolution emissive microdisplay can be beneficial, particularly for wearable systems such as the display systems described herein. As an example, a lower - resolution display may be smaller, lighter, and / or consume less power than a higher - resolution display.

[0119] The above has described the steps of moving or adjusting the position of a light-emitting microdisplay (e.g., including a micro-LED array), but it should be understood that the position of the projection optical system may alternatively or additionally be adjusted. For example, as will be further described in detail below with respect to FIGS. 35A - 35B, the projection optical system may route the light generated through the light-emitting microdisplay to the user of the display system. As an example, the projection optical system may route the light, which is encoded with image information (image light), to an input internal coupling optical element (e.g., an internal coupling grating) of an eyepiece lens configured to direct it to the user. Thus, instead of physically translating the light-emitting microdisplay, the projection optical system may be translated along one or more axes. During the translation, the projection optical system may change the geometric position of each array along one or more axes prior to outputting the light for internal coupling through the internal coupling grating. As described herein, the light projection system may include one or more light-emitting microdisplays, a projection optical system, etc. Thus, the light output of the light projection system may be adjusted by physical translation of a part of the system (e.g., by changing the location of the pixels presented by the light projection system).

[0120] It should be understood that certain portions of the frame in which virtual content is rendered may be more visually apparent to the user than other portions. For example, the user may have high visual acuity with respect to the portion of the virtual content that is on the user's fovea (referred to herein as the "foveal portion"). To determine the location of these foveal portions, the display system may determine the fixation point at which the user is fixating. A portion of the virtual content that is within a threshold angular distance of the present fixation point may be identified as being on the user's fovea. As will be described, the display system may be configured to increase the resolution associated with the foveal portion. The resolution of the remaining portions may be little increased or may not be increased at all.

[0121] As an example, the emissive microdisplay may be configured to update the pixels included in the foveal region at a rate that is higher than that for the pixels for other portions. As described above, the geometric position of the light emitters may be translated or adjusted to tile the emitter-to-emitter regions of the array. Optionally, the light emitters utilized to output light to form pixels including the foveal region may be updated for a relatively high percentage of different geometric positions (e.g., for each different geometric position), while the light emitters for forming pixels away from the foveal region may be updated for a lower percentage of different geometric positions (e.g., these light emitters may be “off” or may simply present the same information as at the previous position). The light emitters utilized to output light to form pixels included in other regions may be updated less frequently. For example, these light emitters may be updated only two or one time for a frame rendered at a given full resolution of the virtual content. For example, the light emitters for the foveal region may be updated for each of four different geometric positions within the emitter-to-emitter region, while the light emitters corresponding to the peripheral portion of the image may be updated for only every other geometric position.

[0122] As a result, the rendered frame, formed from sub-frames that are rapidly displayed or projected, may have an effective resolution that varies across the rendered frame. By using foveated imaging, the effective resolution of the emissive microdisplay can be increased in the foveal region (e.g., the region of interest, the region on which the user focuses, the region specified by the user, the region specified by the designer, etc.) and decreased in other regions (e.g., outside the region of interest). Configuring the emissive microdisplay to provide foveated images may also help conserve resources by eliminating and / or reducing processing and power loads associated with, for example, displaying or projecting less interesting regions (e.g., regions less likely to draw the user's attention). (Exemplary Display System with a Light-Emitting Microdisplay)

[0123] Advantageously, as described herein, a display system that utilizes a light-emitting microdisplay as described herein can enable a low weight and a compact form factor, and can also provide a high frame rate and low motion blur. Preferably, the microdisplay is a light-emitting microdisplay, which provides the advantages of high brightness and high pixel density. In some embodiments, the light-emitting microdisplay is a micro-LED display. In some other embodiments, the light-emitting microdisplay is a micro-OLED display. In some embodiments, the light-emitting microdisplay comprises an array of light emitters having a pitch of less than 10 μm, less than 8 μm, less than 6 μm, less than 5 μm, or less than 2 μm, including, for example, from 1 to 5 μm, and an emitter size of 2 μm or less, 1.7 μm or less, or 1.3 μm or less. In some embodiments, the emitter size is within a range having an upper limit of the above sizes and a lower limit of 1 μm. In some embodiments, the ratio of emitter size to pitch is from 1:1 to 1:5, from 1:2 to 1:4, or from 1:2 to 1:3, which can have the advantage of individual control of the emitters and efficient utilization of the light emitted by the eyepiece lens, as further discussed herein.

[0124] In some embodiments, a plurality of light-emitting microdisplays may be utilized to form an image for a head-mounted display system. The light containing image information for forming these images may be referred to as image light. It should be understood that the image light can vary, for example, in wavelength, intensity, polarization, etc. The light-emitting microdisplay outputs the image light to an eyepiece lens, which then relays the light to the user's eye.

[0125] In some embodiments, a plurality of emissive microdisplays may be utilized and positioned on different sides of an optical combiner, such as an X-cube prism or a dichroic X-cube. The X-cube prism receives light rays from different microdisplays on different faces of the cube and outputs the light rays from the same face of the cube. The output light may be directed towards a projection optical system configured to converge or focus the image light onto an eyepiece lens.

[0126] In some embodiments, the plurality of emissive microdisplays comprise monochrome microdisplays, which are configured to output light of a single primary color. Combining various primary colors forms a full-color image. In some other embodiments, one or more of the emissive microdisplays may have subpixels configured to emit light of two or more but not all of the primary colors utilized by the display system. For example, a single emissive microdisplay may have subpixels that emit light of the colors blue and green, while a separate emissive microdisplay on a different face of the X-cube may have pixels configured to emit red light. In some embodiments, the plurality of microdisplays are each full-color displays comprising pixels formed from a plurality of subpixels configured to emit light of different primary colors, for example. Advantageously, combining the light of a plurality of full-color microdisplays can increase display brightness and dynamic range.

[0127] It should be understood that the emissive microdisplay may comprise an array of light emitters. The light emitters may emit light with a Lambertian angular emission profile. Unfortunately, such an angular emission profile may "waste" light because only a small portion of the emitted light may ultimately be incident on the eyepiece. In some embodiments, an optical collimator may be utilized to narrow the angular emission profile of the light emitted by the light emitter. As used herein, an optical collimator is an optical structure that narrows the angular emission profile of incident light. That is, the optical collimator receives light from an associated light emitter with a relatively wide initial angular emission profile and outputs that light with an angular emission profile that is narrower than the wide initial angular emission profile. In some embodiments, the light rays of the light exiting the optical collimator are transmitted through the collimator and are more parallel than the light rays of the light received by the optical collimator before exiting therefrom. Examples of optical collimators include microlenses, nanolenses, reflective wells, metasurfaces, and liquid crystal gratings. In some embodiments, the optical collimator may be configured to direct light and ultimately converge it onto a light coupling optical element that is deflected to different sides. In some embodiments, each light emitter has a dedicated optical collimator. The optical collimator is preferably positioned directly adjacent to or in contact with the light emitter and captures a large proportion of the light emitted by the associated light emitter.

[0128] In some embodiments, a single emissive microdisplay may be utilized to direct light to an eyepiece. For example, the single emissive microdisplay may be a full-color display comprising light emitters that emit light of different primary colors. In some embodiments, the light emitters may form groups localized within a common area, with each group comprising light emitters that emit light of each primary color. In such embodiments, each group of light emitters may share a common microlens. Advantageously, light of different colors from different light emitters follows different paths through the microlens, which may appear as light of different primary colors incident on different internal coupling optical elements of the eyepiece, as discussed herein.

[0129] In some embodiments, the full-color microdisplay may comprise repeating groups of light emitters of the same primary color. For example, the microdisplay may include rows of light emitters, with the light emitters of each individual row configured to emit light of the same color. Thus, different rows may emit light of different primary colors. Additionally, the microdisplay may have an associated array of light collimators configured to direct light to a desired location on the eyepiece, such as an associated internal coupling optical element. Advantageously, although the individual light emitters of such a full-color microdisplay may not be positioned to form a high-quality full-color image such that they are directly viewable on the microdisplay, the lens array appropriately steers the light from the light emitters to the eyepiece, which combines the monochrome images formed by the light emitters of different colors, thereby forming a high-quality full-color image.

[0130] In some embodiments, an eyepiece that receives image light from a microdisplay may comprise a waveguide assembly. The area of the waveguide of the waveguide assembly onto which the image light is incident may include an internal coupling optical element that internally couples the incident image light such that light propagates through the waveguide by total internal reflection (TIR). In some embodiments, the waveguide assembly may include a stack of waveguides, each having an associated internal coupling optical element. Different internal coupling optical elements may be configured to internally couple different colors of light such that different waveguides are configured to propagate different colors of light therein. The waveguide may include an external coupling optical element that externally couples the light propagating therein such that externally coupled light propagates towards the user's eye. In some other embodiments, the waveguide assembly may include a single waveguide having associated internal coupling optical elements configured to internally couple different primary colors of light.

[0131] In some embodiments, the internal coupling optical elements are laterally offset as viewed from the projection optical system. Different internal coupling optical elements may be configured to internally couple different colors of light. Preferably, different colors of image light follow different paths to the eyepiece and thus impinge on different corresponding internal coupling optical elements.

[0132] In some other embodiments, other types of eyepieces or optical systems for relaying image light to the user's eye may be utilized. For example, as discussed herein, the eyepiece may include one or more waveguides that propagate image light therein by TIR. As another example, the eyepiece may include a water tank mirror combiner comprising a translucent mirror that both directs image light towards the viewer and enables a view of the surrounding environment.

[0133] In some embodiments, the eyepiece may be configured to selectively output light with different amounts of wavefront divergence to provide multiple virtual depth planes (also simply referred to herein as "depth planes") of virtual content that are perceived to be at different distances from the user. For example, the eyepiece may include a plurality of waveguides having external coupling optical elements with different refractive powers for outputting light with different amounts of wavefront divergence, respectively. In some other embodiments, a variable focus element may be provided between the eyepiece and the user's eye. The variable focus element may be configured to dynamically change the refractive power to provide the desired wavefront divergence for a particular virtual content. In some embodiments, as an alternative to or in addition to the waveguide optical structure for providing refractive power, the display system may also include a plurality of lenses that provide or additionally provide refractive power.

[0134] In addition to the compact form factor and high frame rate discussed above, the emissive microdisplay according to some embodiments may provide one or more of the following advantages. For example, the microdisplay may provide a significantly small pixel pitch and high pixel density. The microdisplay may also provide high brightness and efficiency. For example, the light emitters of the emissive microdisplay may consume only the power required to emit light when the light emitter is required to provide content at a certain brightness. This is in contrast to other display technologies where the light source may illuminate the entire panel of pixels regardless of whether some of those pixels are dark. Further, the human visual system integrates the received light over time, and the light emitters of emissive microdisplays such as microLEDs advantageously have a high duty cycle (e.g., the light emitter in the microdisplay includes a short activation period for rising from “off” to full “on” state and correspondingly a short time for falling from “on” state to “off” state, which enables the light emitter to emit light at an on level over a large percentage of each cycle). As a result, the power used to generate an image with a given perceived brightness may be less compared to conventional display technologies with a lower duty cycle. In some embodiments, the duty cycle may be 70% or more, 80% or more, or 90% or more. In some embodiments, the duty cycle may be about 99%. Additionally, as described herein, the microdisplay may facilitate a significantly high frame rate, which may provide advantages including reducing the mismatch between the position of the user's head and the content being displayed.

[0135] Reference is now made to the drawings, in which like reference numerals refer to like parts throughout. Unless otherwise indicated, the drawings are schematic and are not necessarily drawn to scale.

[0136] FIG. 2 illustrates a conventional display system for simulating a three-dimensional image for a user. It should be understood that the user's eyes are separated, and when looking at a real object in space, each eye has a slightly different view of the object and can form an image of the object at different locations on the retina of each eye. This can be referred to as binocular disparity and can be utilized by the human visual system to provide a perception of depth. The conventional display system simulates binocular disparity by presenting two distinct images 190, 200 with slightly different views of one identical virtual object for each of the eyes 210, 220, corresponding to the views of the virtual object that would be seen by each eye as if the virtual object were a real object at the desired depth. These images provide binocular cues that can be interpreted by the user's visual system to derive a perception of depth.

[0137] Continuing to refer to FIG. 2, the images 190, 200 are separated from the eyes 210, 220 by a distance 230 on the z-axis. The z-axis is parallel to the optical axis of the viewer in a state where the eye is fixating on an object at optical infinity directly in front of the viewer. The images 190, 200 are flat and at a fixed distance from the eyes 210, 220. Based on the slightly different views of the virtual object in the images presented to the eyes 210, 220 respectively, the eyes can necessarily rotate so that the images of the object come to corresponding points on the respective retinas of the eyes and a single binocular vision is maintained. This rotation can converge the respective lines of sight of the eyes 210, 220 onto a point in the space where the virtual object is perceived to exist. As a result, the provision of a three-dimensional image has conventionally involved being able to manipulate the convergence / divergence movement of the user's eyes 210, 220 and providing binocular cues that are interpreted by the human visual system to provide a perception of depth.

[0138] However, the generation of realistic and comfortable perception of depth is difficult. It should be understood that light from objects at different distances from the eye has wavefronts with different amounts of divergence. FIGS. 3A-3C illustrate the relationship between distance and the divergence of light rays. The distances between the object and the eye 210 are represented in the order of decreasing distances R1, R2, and R3. As shown in FIGS. 3A-3C, the light rays diverge more as the distance to the object decreases. Conversely, as the distance increases, the light rays become more collimated. In other words, it can be said that the light field generated by a point (object or part of an object) has a spherical wavefront curvature that is a function of the distance the point is away from the user's eye. The curvature increases as the distance between the object and the eye 210 decreases. Only the monocular 210 is illustrated in FIGS. 3A-3C and various other figures herein for clarity of illustration, but the discussion regarding the eye 210 can be applied to both eyes 210 and 220 of the viewer.

[0139] Continuing to refer to FIGS. 3A-3C, the light from the object on which the viewer's eye is fixated can have different wavefront divergence degrees. Due to the different amounts of wavefront divergence, the light can be focused differently by the eye's lens, which in turn can require the lens to take on different shapes to form a focused image on the retina of the eye. If the focused image is not formed on the retina, the resulting retinal blur acts as a cue for accommodation, causing a change in the shape of the eye's lens until the focused image is formed on the retina. For example, the cue for accommodation triggers the relaxation or contraction of the ciliary muscle surrounding the eye's lens, thereby modulating the force applied to the zonular ligaments that hold the lens, and thus changing the shape of the eye's lens until the retinal blur of the fixated image is eliminated or minimized, thereby forming a focused image of the fixated object on the retina (e.g., the fovea) of the eye. The process by which the eye's lens changes shape can be referred to as accommodation, and the shape of the eye's lens required to form a focused image of the fixated object on the retina (e.g., the fovea) of the eye can be referred to as the accommodative state.

[0140] Referring now to FIG. 4A, the representation of the accommodation-convergence / divergence motion response of the human visual system is illustrated. The movement of the eyes to fixate on an object causes the eyes to receive light from the object, and the light forms an image on each of the retinas of the eyes. The presence of retinal blur within the image formed on the retina can provide a cue for accommodation, and the relative location of the image on the retina can provide a cue for convergence / divergence motion. The cue for accommodation results in accommodation occurring and the eye's lens assuming a particular accommodation state that forms a focused image of the object on the eye's retina (e.g., the fovea). On the other hand, the cue for convergence / divergence motion causes a convergence / divergence motion (rotation of the eyes) such that the images formed on each retina of each eye are at corresponding retinal points that maintain single binocular vision. At these positions, it can be said that the eyes are in a particular convergence / divergence motion state. Continuing to refer to FIG. 4A, accommodation can be understood to be the process by which the eyes achieve a particular accommodation state, and convergence / divergence motion can be understood to be the process by which the eyes achieve a particular convergence / divergence motion state. As shown in FIG. 4A, the accommodation and convergence / divergence motion states of the eyes can change when the user fixates on another object. For example, the accommodated state can change when the user fixates on a new object at a different depth along the z-axis.

[0141] Although not limited by theory, it is thought that a viewer of an object may perceive the object as "three-dimensional" due to a combination of convergence / divergence motion and accommodation. As described above, the convergence / divergence movement of the two eyes relative to each other (e.g., the rotation of the eyes such that the pupils move towards or away from each other, converging the lines of sight of the eyes to fixate on an object) is closely associated with the accommodation of the eye's lens. Under normal conditions, a change in the focus of the eye's lens to change the focus from one object to another object at a different distance will automatically cause a corresponding change in convergence / divergence to the same distance under a relationship known as the "accommodation-convergence / divergence reflex". Similarly, a change in convergence / divergence will, under normal conditions, induce a corresponding change in the shape of the lens.

[0142] Referring now to FIG. 4B, an example of different accommodation and convergence / divergence states of the eyes is illustrated. The pair of eyes 222a fixates on an object at optical infinity, while the pair of eyes 222b fixates on an object 221 at less than optical infinity. It should be noted that the convergence / divergence states of each pair of eyes are different, with the pair of eyes 222a being directed straight, while the pair of eyes 222 converges onto the object 221. The accommodation states of the eyes forming each pair of eyes 222a and 222b also differ, as represented by the different shapes of the lenses 210a, 220a.

[0143] Unfortunately, many users of conventional "3-D" display systems find such conventional systems uncomfortable or perceive no sense of depth, due to the mismatch between the depth adjustment and the vergence / accommodation motion state in these displays. As described above, many stereoscopic or "3-D" display systems present a scene by providing slightly different images to each eye. Such systems are uncomfortable for many viewers because they simply provide different presentations of the scene, causing a change in the vergence / accommodation motion state of the eyes, but without a corresponding change in the state of depth adjustment of those eyes. Rather, the images are presented at a fixed distance from the eyes by the display such that the eyes view all the image information in a single depth adjustment state. Such an arrangement violates the "accommodation-vergence reflex" by causing a change in the vergence / accommodation motion state without a corresponding change in the depth adjustment state. This mismatch is thought to cause viewer discomfort. A display system that provides better alignment between depth adjustment and vergence / accommodation motion can create a more realistic and comfortable simulation of a three-dimensional image.

[0144] Although not limited by theory, the human eye is typically thought to be able to interpret a finite number of depth planes and provide depth perception. As a result, a highly realistic simulation of perceived depth can be achieved by providing the eyes with different presentations of images corresponding to each of these limited number of depth planes. In some embodiments, the different presentations may provide both a cue for vergence / accommodation motion and a corresponding cue for depth adjustment, thereby providing a physiologically correct accommodation-vergence motion alignment.

[0145] Continuing to refer to FIG. 4B, two depth planes 240 are illustrated, corresponding to different distances in space from the eyes 210, 220. For a given depth plane 240, a convergence / divergence motion cue may be provided by appropriately displaying images of different viewpoints for each of the eyes 210, 220. Additionally, for a given depth plane 240, the light forming the images provided to each eye 210, 220 may have a wavefront divergence corresponding to a light field generated by a point at the distance of that depth plane 240.

[0146] In the illustrated embodiment, the distance along the z-axis of the depth plane 240 containing the point 221 is 1 m. As used herein, the distance or depth along the z-axis may be measured using a zero point located at the exit pupil of the user's eye. Thus, the depth plane 240 located at a depth of 1 m corresponds to a distance 1 m away from the exit pupil of the user's eye on the optical axis of those eyes in a state where the eyes are directed towards optical infinity. As an approximation, the depth or distance along the z-axis is measured from the front of the user's eye (e.g., the surface of the waveguide), and a value related to the distance between the device and the exit pupil of the user's eye may be added. That value is referred to as the pupil distance and may correspond to the distance between the exit pupil of the user's eye and the display worn by the user in front of the eye. In practice, the value for the pupil distance may generally be a normalized value used for all viewers. For example, the pupil distance may be assumed to be 20 mm, and the depth plane at a depth of 1 m may be at a distance of 980 mm in front of the display.

[0147] Referring now to FIGS. 4C and 4D, examples of consistent vergence-accommodation-convergence / divergence movement distances and inconsistent vergence-accommodation-convergence / divergence movement distances are illustrated, respectively. As shown in FIG. 4C, the display system may provide an image of a virtual object to each eye 210, 220. The image may cause the eyes 210, 220 to assume a vergence-accommodation-convergence / divergence state in which the eyes converge on a point 15 on the depth plane 240. Additionally, the image may be formed by light having a wavefront curvature corresponding to an actual object in the depth plane 240. As a result, the eyes 210, 220 assume an accommodation state in which the image is in focus on their retinas. Thus, the user may perceive the virtual object as being at the point 15 on the depth plane 240.

[0148] It should be understood that the accommodation and vergence-accommodation-convergence / divergence states of the eyes 210, 220 are each associated with a specific distance on the z-axis. For example, an object at a specific distance from the eyes 210, 220 causes those eyes to assume a specific accommodation state based on the distance of the object. The distance associated with a specific accommodation state may be referred to as the accommodation distance A d Similarly, there exists a specific vergence-accommodation-convergence / divergence distance V d associated with a specific vergence-accommodation-convergence / divergence state or the eyes in a particular position relative to each other. When the accommodation distance and the vergence-accommodation-convergence / divergence distance are consistent, the relationship between accommodation and vergence-accommodation-convergence / divergence can be said to be physiologically correct. This is regarded as the most comfortable scenario for the viewer.

[0149] However, in a stereoscopic display, the focusing distance and the vergence / accommodation movement distance may not always match. For example, as shown in FIG. 4D, the images displayed on eyes 210, 220 may be displayed with wavefront divergence corresponding to depth plane 240, and eyes 210, 220 may take a specific focusing state in which points 15a, 15b on that depth plane are in focus. However, the images displayed on eyes 210, 220 may provide a cue for vergence / accommodation movement that converges eyes 210, 220 on points 15 that are not located on depth plane 240. As a result, in some embodiments, the focusing distance corresponds to the distance from the exit pupils of eyes 210, 220 to depth plane 240, while the vergence / accommodation movement distance corresponds to a greater distance from the exit pupils of eyes 210, 220 to point 15. The focusing distance is different from the vergence / accommodation movement distance. As a result, there is a focusing-vergence / accommodation movement mismatch. Such a mismatch is considered undesirable and can cause discomfort to the user. The mismatch corresponds to a distance (e.g., V d -A d ) and can be characterized using diopters.

[0150] It should be understood that in some embodiments, as long as the same reference point is used for the focusing distance and the vergence / accommodation movement distance, reference points other than the exit pupils of eyes 210, 220 may be used to determine the distance for determining the focusing-vergence / accommodation movement mismatch. For example, the distance can be measured from the cornea to the depth plane, from the retina to the depth plane, from the eyepiece (e.g., the waveguide of the display device) to the depth plane, etc.

[0151] While not limited by theory, it is believed that a user may perceive vergence-accommodation mismatches of up to about 0.25 diopters, up to about 0.33 diopters, and up to about 0.5 diopters as being physiologically correct, without the mismatch itself causing significant discomfort. In some embodiments, a display system disclosed herein (e.g., display system 250, FIG. 6) presents an image having a vergence-accommodation mismatch of about 0.5 diopters or less to a viewer. In some other embodiments, the vergence-accommodation mismatch of the image provided by the display system is about 0.33 diopters or less. In still other embodiments, the vergence-accommodation mismatch of the image provided by the display system is about 0.25 diopters or less, including about 0.1 diopters or less.

[0152] FIG. 5 illustrates a side view of an approach for simulating a three-dimensional image by modifying wavefront divergence. The display system includes a waveguide 270 configured to receive light 770 encoded with image information and output the light to the user's eye 210. The waveguide 270 may output light 650 with a defined amount of wavefront divergence corresponding to the wavefront divergence of a light field generated by a point on a desired depth plane 240. In some embodiments, the same amount of wavefront divergence is provided for all objects presented on that depth plane. Additionally, the user's other eye would be illustrated as being provided with image information from a similar waveguide.

[0153] In some embodiments, a single waveguide may be configured to output light with a set amount of wavefront divergence corresponding to a single or limited number of depth planes, and / or the waveguide may be configured to output light within a limited range of wavelengths. As a result, in some embodiments, multiple or stacked waveguides may be utilized to provide different amounts of wavefront divergence for different depth planes and / or output light within different ranges of wavelengths. It should be understood that as used herein, a depth plane may be a plane or may follow the contour of a curved surface.

[0154] FIG. 6 illustrates an example of a stack of waveguides for outputting image information to a user. Display system 250 includes a stack or stacked waveguide assembly 260 of waveguides 270, 280, 290, 300, 310 that can be utilized to provide three-dimensional perception to the eye / brain. It should be understood that display system 250 may be considered a light field display in some embodiments. Additionally, waveguide assembly 260 may also be referred to as an eyepiece.

[0155] In some embodiments, display system 250 may be configured to provide a substantially continuous cue for convergence / divergence movement and a plurality of discrete cues for depth adjustment. The cue for convergence / divergence movement may be provided by displaying different images to each of the user's eyes, and the cue for depth adjustment may be provided by outputting light that forms an image with a selectable discrete amount of wavefront divergence. In other words, display system 250 may be configured to output light with a variable level of wavefront divergence. In some embodiments, each discrete level of wavefront divergence may correspond to a particular depth plane and may be provided by a particular one of waveguides 270, 280, 290, 300, 310.

[0156] Continuing to refer to FIG. 6, waveguide assembly 260 may also include a plurality of features 320, 330, 340, 350 between the waveguides. In some embodiments, features 320, 330, 340, 350 may be one or more lenses. Waveguides 270, 280, 290, 300, 310 and / or

[0157] The plurality of lenses 320, 330, 340, 350 may be configured to transmit image information to the eye using various levels of wavefront curvature or ray divergence. Each waveguide level may be associated with a particular depth plane and may be configured to output image information corresponding to that depth plane. Image input devices 360, 370, 380, 390, 400 may function as light sources for the waveguides and may be utilized to input image information into waveguides 270, 280, 290, 300, 310 and may each be configured to disperse incident light across the respective waveguide to output towards the eye 210 as described herein. Light exits from the output surfaces 410, 420, 430, 440, 450 of the image input devices 360, 370, 380, 390, 400 and is input into the corresponding input surfaces 460, 470, 480, 490, 500 of the waveguides 270, 280, 290, 300, 310. In some embodiments, the input surfaces 460, 470, 480, 490, 500 may each be the edge of the corresponding waveguide or a portion of the major surface of the corresponding waveguide (i.e., one of the waveguide surfaces facing directly towards the world 510 or the viewer's eye 210). In some embodiments, a single beam of light (e.g., a collimated beam) is input into each waveguide and outputs an entire field of cloned collimated beams directed towards the eye 210 at a particular angle (and amount of divergence) corresponding to the depth plane associated with the particular waveguide. In some embodiments, a single one of the image input devices 360, 370, 380, 390, 400 may be associated with and input light into a plurality (e.g., three) of the waveguides 270, 280, 290, 300, 310.

[0158] In some embodiments, the image input devices 360, 370, 380, 390, 400 are discrete displays that each generate image information for input into their respective waveguides 270, 280, 290, 300, 310. In some other embodiments, the image input devices 360, 370, 380, 390, 400 are the output ends of a single multiplexed display that can send image information to each of the image input devices 360, 370, 380, 390, 400, for example, via one or more optical waveguides (such as optical fiber cables). It should be understood that the image information provided by the image input devices 360, 370, 380, 390, 400 may include light of different wavelengths or colors (e.g., different primary colors as discussed herein).

[0159] In some embodiments, the light input into waveguides 270, 280, 290, 300, 310 is provided by a light projection system 520, which includes a light module 530, which may include a light emitter such as a light emitting diode (LED). The light from the light module 530 may be directed and modified by a light modulator 540, such as a spatial light modulator, via a beam splitter 550. The light modulator 540 may be configured to vary the perceived intensity of the light input into waveguides 270, 280, 290, 300, 310 and to encode the light with image information. Examples of spatial light modulators include liquid crystal displays (LCDs), including liquid crystal on silicon (LCOS) displays. In some other embodiments, the spatial light modulator may be a MEMS device such as a digital light processing (DLP) device. Image input devices 360, 370, 380, 390, 400 are shown schematically, and in some embodiments, these image input devices may represent different optical paths and locations within a common projection system configured to output light into the associated ones of waveguides 270, 280, 290, 300, 310. In some embodiments, the waveguides of waveguide assembly 260 may function as an ideal lens while relaying the light input into the waveguides to the user's eye. In this concept, the object may be the spatial light modulator 540, and the image may be an image on a depth plane.

[0160] In some embodiments, the display system 250 may be a scanning fiber display comprising one or more scanning fibers configured to project light in various patterns (e.g., raster scan, helical scan, Lissajous pattern, etc.) into one or more waveguides 270, 280, 290, 300, 310 and ultimately into the viewer's eye 210. In some embodiments, the illustrated image input devices 360, 370, 380, 390, 400 may schematically represent a single scanning fiber or a bundle of scanning fibers configured to input light into one or more waveguides 270, 280, 290, 300, 310. In some other embodiments, the illustrated image input devices 360, 370, 380, 390, 400 may schematically represent a plurality of scanning fibers or a plurality of bundles of scanning fibers, each configured to input light into an associated one of the waveguides 270, 280, 290, 300, 310. It should be understood that one or more optical fibers may be configured to transmit light from the optical module 530 to one or more waveguides 270, 280, 290, 300, 310. It should be understood that one or more intervening optical structures may be provided between the scanning fiber or fibers and the one or more waveguides 270, 280, 290, 300, 310 and may, for example, redirect light exiting the scanning fiber into one or more of the waveguides 270, 280, 290, 300, 310.

[0161] Controller 560 controls the operation of one or more of the stacked waveguide assemblies 260, including the operation of the image input devices 360, 370, 380, 390, 400, the light source 530, and the optical module 540. In some embodiments, controller 560 is part of the local data processing module 140. Controller 560 includes programming (e.g., instructions in a non-transitory medium) that adjusts the timing and provision of image information to waveguides 270, 280, 290, 300, 310, for example, according to any of the various schemes disclosed herein. In some embodiments, the controller may be a single monolithic device or a distributed system connected by wired or wireless communication channels. Controller 560 may, in some embodiments, also be part of processing module 140 or 150 (FIG. 9E).

[0162] Continuing to refer to FIG. 6, the waveguides 270, 280, 290, 300, 310 may be configured to propagate light within each individual waveguide by total internal reflection (TIR). The waveguides 270, 280, 290, 300, 310 may each be planar, or have another shape (e.g., curved), with a major top surface and a bottom surface and an edge extending between those major top and bottom surfaces. In the illustrated configuration, the waveguides 270, 280, 290, 300, 310 each include external coupling optical elements 570, 580, 590, 600, 610 configured to extract light from the waveguide by redirecting the light propagating within each individual waveguide out of the waveguide and outputting the image information to the eye 210. The extracted light may also be referred to as external coupled light, and the external coupling optical elements may also be referred to as light extraction optical elements. The beam of extracted light may be output by the waveguide at the location where the light propagating within the waveguide impinges on the light extraction optical element. The external coupling optical elements 570, 580, 590, 600, 610 may be gratings, for example, including diffractive optical features as further discussed herein. For ease of explanation and clarity of the drawings, the external coupling optical elements 570, 580, 590, 600, 610 are shown disposed on the bottom major surface of the waveguides 270, 280, 290, 300, 310, but in some embodiments, the external coupling optical elements 570, 580, 590, 600, 610 may be disposed on the top and / or bottom major surfaces and / or directly within the volume of the waveguides 270, 280, 290, 300, 310, as further discussed herein. In some embodiments, the external coupling optical elements 570, 580, 590, 600, 610 may be attached to a transparent substrate and formed within a layer of material forming the waveguides 270, 280, 290, 300, 310. In some other embodiments, the waveguides 270, 280, 290, 300, 310 may be a monolithic piece of material, and the external coupling optical elements 570, 580, 590, 600, 610 may be formed on and / or within the surface of that piece of material.

[0163] Continuing to refer to FIG. 6, as discussed herein, each of the waveguides 270, 280, 290, 300, 310 is configured to output light and form an image corresponding to a particular depth plane. For example, the waveguide 270 closest to the eye may be configured to deliver collimated light (input into such a waveguide 270) to the eye 210. The collimated light may represent an optically infinite focal plane. The next upper waveguide 280 may be configured to deliver collimated light that passes through a first lens 350 (e.g., a negative lens) before reaching the eye 210. Such a first lens 350 may be configured to generate a somewhat convex wavefront curvature such that the eye / brain interprets the light originating from the next upper waveguide 280 as originating from a first focal plane that is closer inwardly from the optically infinite towards the eye 210. Similarly, the third upper waveguide 290 passes its output light through both the first 350 and second 340 lenses before reaching the eye 210. The combined refractive power of the first 350 and second 340 lenses may be configured to generate another incremental amount of wavefront curvature such that the eye / brain interprets the light originating from the third waveguide 290 as originating from a second focal plane that is closer inwardly from the optically infinite towards the person than the light from the next upper waveguide 280 was.

[0164] The other waveguide layers 300, 310 and lenses 330, 320 are similarly configured, and the top waveguide 310 in the stack sends its output through all of the lenses between it and the eye for the collective focusing power representing the focal plane closest to the person. When viewing / interpreting light originating from the world 510 on the other side of the stacked waveguide assembly 260, a compensating lens layer 620 may be disposed on top of the stack to compensate for the stack of lenses 320, 330, 340, 350. Such a configuration provides the same number of perceived focal planes as there are available waveguide / lens pairs. Both the external coupling optical elements of the waveguides and the focusing sides of the lenses may be static (i.e., not dynamic or electroactive). In some alternative embodiments, one or both may be dynamic using electroactive features.

[0165] In some embodiments, two or more of the waveguides 270, 280, 290, 300, 310 may have the same associated depth plane. For example, a plurality of waveguides 270, 280, 290, 300, 310 may be configured to output images set to the same depth plane, or a plurality of subsets of the waveguides 270, 280, 290, 300, 310 may be configured to output images set to the same plurality of depth planes, with one set per depth plane. This can provide the advantage of forming a tiled image to provide an extended field of view in those depth planes.

[0166] Continuing to refer to FIG. 6, the external coupling optical elements 570, 580, 590, 600, 610 may be configured to redirect light from their respective waveguides for a particular depth plane associated with the waveguide and output the light with an appropriate amount of divergence or collimation. As a result, waveguides having different associated depth planes may have different configurations of the external coupling optical elements 570, 580, 590, 600, 610, which output light with different amounts of divergence depending on the associated depth plane. In some embodiments, the light extraction optical elements 570, 580, 590, 600, 610 may be three-dimensional or surface features configured to output light at a specific angle. For example, the light extraction optical elements 570, 580, 590, 600, 610 may be a volume hologram, a surface hologram, and / or a diffraction grating. In some embodiments, the features 320, 330, 340, 350 may not be lenses. Rather, they may simply be spacers (e.g., cladding layers and / or structures for forming voids).

[0167] In some embodiments, the external coupling optical elements 570, 580, 590, 600, 610 are diffraction features or “diffractive optical elements” (also referred to herein as “DOEs”) that form a diffraction pattern. Preferably, the DOE has a sufficiently low diffraction efficiency such that only a portion of the light of the beam is deflected toward the eye 210 at each intersection of the DOE, while the remainder continues to travel through the waveguide via TIR. The light carrying the image information is thus split into several associated output beams that exit the waveguide at various locations, resulting in a very uniform pattern of output emission toward the eye 210 with respect to this particular collimated beam that bounces within the waveguide.

[0168] In some embodiments, one or more DOEs may be switchable between an “on” state in which they actively diffract and an “off” state in which they do not significantly diffract. For example, a switchable DOE may comprise a layer of polymer dispersed liquid crystal in which microdroplets have a diffraction pattern in a host medium, and the refractive index of the microdroplets may be switched to substantially match the refractive index of the host material (in which case the pattern does not significantly diffract the incident light), or the microdroplets may be switched to a refractive index that does not match that of the host medium (in which case the pattern actively diffracts the incident light).

[0169] In some embodiments, a camera assembly 630 (e.g., a digital camera including visible light and infrared light cameras) may be provided to capture an image of the eye 210 and / or the tissue surrounding the eye 210, and for example, detect user input and / or monitor the physiological state of the user. As used herein, a camera may be any image capture device. In some embodiments, the camera assembly 630 may include an image capture device and a light source (e.g., infrared light) that projects light onto the eye, and then the light may be reflected by the eye and detected by the image capture device. In some embodiments, the camera assembly 630 may be attached to a frame or support structure 80 (FIG. 9E) and may communicate electrically with a processing module 140 and / or 150 that may process the image information from the camera assembly 630. In some embodiments, one camera assembly 630 may be utilized per eye to monitor each eye separately.

[0170] In some embodiments, the camera assembly 630 may observe the movement of the user, such as the eye movement of the user. As an example, the camera assembly 630 may capture an image of the eye 210 and determine the size, position, and / or orientation of the pupil of the eye 210 (or some other structure of the eye 210). The camera assembly 630 may, if desired, acquire an image (processed by a processing circuitry network of the type described herein) that is used to determine the direction in which the user is looking (e.g., eye pose or line of sight direction). In some embodiments, the camera assembly 630 may include multiple cameras, at least one of which is utilized per eye and may independently determine the eye pose or line of sight direction of each eye separately. In some embodiments, the camera assembly 630 may, in combination with a processing circuitry such as the controller 560 or the local data processing module 140, determine the eye pose or line of sight direction based on a flash (e.g., reflection) of light (e.g., infrared light) reflected from a light source included within the camera assembly 630.

[0171] Referring now to FIG. 7, an embodiment of an output beam output by a waveguide is shown. Although one waveguide is illustrated, it should be understood that other waveguides within the waveguide assembly 260 (FIG. 6) may function in a similar manner, and the waveguide assembly 260 includes a plurality of waveguides. Light 640 is input into the waveguide 270 at the input surface 460 of the waveguide 270 and propagates within the waveguide 270 by TIR. At the point where the light 640 impinges on the DOE 570, a portion of the light exits the waveguide as the output beam 650. The output beam 650 is illustrated as being substantially parallel, but as discussed herein, and depending on the depth plane associated with the waveguide 270, it may be redirected to propagate at an angle (e.g., to form a diverging output beam) to the eye 210. It should be understood that a substantially parallel output beam may represent a waveguide with an external coupling optical element that externally couples the light to form an image that appears to be set in a depth plane at a long distance (e.g., optical infinity) from the eye 210. Other waveguides or other sets of external coupling optical elements may output a more divergent output beam pattern, which would require the eye 210 to focus at a closer distance and would be interpreted by the brain as light from a distance closer to the eye 210 than optical infinity.

[0172] In some embodiments, a full-color image may be formed in each depth plane by overlaying the image on each of the primary colors, e.g., three or more primary colors. FIG. 8 illustrates an example of a stacked waveguide assembly, and each depth plane includes an image formed using a plurality of different primary colors. The illustrated embodiment shows depth planes 240a-240f, although more or fewer depths may also be considered. Each depth plane may have three or more primary color images associated therewith, including a first image of a first color G, a second image of a second color R, and a third image of a third color B. Different depth planes are shown in the figure by different numbers associated with the diopters (dpt) following the letters G, R, and B. As a mere example, the numbers following each of these letters indicate the diopter (1 / m), i.e., the inverse distance of the depth plane from the viewer, and each box in the figure represents an individual primary color image. In some embodiments, to account for differences in the focusing of light of different wavelengths by the eye, the exact location of the depth planes for different primary colors may vary. For example, the different primary color images for a given depth plane may be placed on depth planes corresponding to different distances from the user. Such an arrangement may increase visual acuity and user comfort and / or reduce chromatic aberration.

[0173] In some embodiments, the light of each primary color may be output by a single dedicated waveguide, and as a result, each depth plane may have a plurality of waveguides associated therewith. In such embodiments, each box in the figure, including those containing the letters G, R, or B, may be understood to represent an individual waveguide, and three waveguides may be provided for each depth plane, with three primary color images being provided for each depth plane. The waveguides associated with each depth plane are shown adjacent to each other in this figure for ease of explanation, but it should be understood that in a physical device, all of the waveguides may be arranged in a stack with one waveguide per level. In some other embodiments, a plurality of primary colors may be output by the same waveguide, e.g., such that only a single waveguide is provided for each depth plane.

[0174] Continuing to refer to FIG. 8, in some embodiments, G is green, R is red, and B is blue. In some other embodiments, other colors associated with other wavelengths of light, including magenta and cyan, may also be used in addition to or in place of one or more of red, green, or blue.

[0175] It should be understood that references throughout this disclosure to the color of a given light are understood to encompass light of one or more wavelengths within the range of wavelengths of the light as perceived by a viewer as that given color. For example, red light may include light of one or more wavelengths within the range of about 620 - 780 nm, green light may include light of one or more wavelengths within the range of about 492 - 577 nm, and blue light may include light of one or more wavelengths within the range of about 435 - 493 nm.

[0176] In some embodiments, the light source 530 (FIG. 6) may be configured to emit light of one or more wavelengths outside the visual perception range of the viewer, such as infrared and / or ultraviolet wavelength light. Additionally, the internal coupling, external coupling, and other light redirection structures of the waveguide of the display 250 may be configured to direct and emit this light from the display towards the user's eye 210, for example, for imaging and / or user stimulation applications.

[0177] Referring now to FIG. 9A, in some embodiments, light that impinges on a waveguide may need to be redirected to internally couple that light into the waveguide. An internal coupling optical element may be used to redirect and internally couple the light into its corresponding waveguide. FIG. 9A illustrates a cross-sectional side view of an example of a stack 660 of multiple or a set of waveguides, each including an internal coupling optical element. Each waveguide may be configured to output light of one or more different wavelengths or one or more different wavelength ranges. Stack 660 may correspond to stack 260 (FIG. 6), and the illustrated waveguides of stack 660 may correspond to a portion of multiple waveguides 270, 280, 290, 300, 310, but it should be understood that light from one or more of image input devices 360, 370, 380, 390, 400 is input into the waveguide from a position where the light is required to be redirected for internal coupling.

[0178] The illustrated set 660 of stacked waveguides includes waveguides 670, 680, and 690. Each waveguide includes an associated internal coupling optical element (which may also be referred to as an optical input area on the waveguide). For example, internal coupling optical element 700 is disposed on a major surface (e.g., the upper major surface) of waveguide 670, internal coupling optical element 710 is disposed on a major surface (e.g., the upper major surface) of waveguide 680, and internal coupling optical element 720 is disposed on a major surface (e.g., the upper major surface) of waveguide 690. In some embodiments, one or more of the internal coupling optical elements 700, 710, 720 may be disposed on the bottom major surface of the individual waveguides 670, 680, 690 (in particular, one or more of the internal coupling optical elements are reflective deflecting optical elements). As illustrated, the internal coupling optical elements 700, 710, 720 may be disposed on the upper major surface (or the upper part of the next lower waveguide) of their individual waveguides 670, 680, 690, and in particular, those internal coupling optical elements are transmissive deflecting optical elements. In some embodiments, the internal coupling optical elements 700, 710, 720 may be disposed within the bodies of the individual waveguides 670, 680, 690. In some embodiments, as discussed herein, the internal coupling optical elements 700, 710, 720 are wavelength selective such that they selectively redirect one or more wavelengths of light while transmitting other wavelengths of light. Although illustrated on one side or corner of their individual waveguides 670, 680, 690, it should be understood that the internal coupling optical elements 700, 710, 720 may, in some embodiments, be disposed within other areas of their individual waveguides 670, 680, 690.

[0179] As shown, the internally coupled optical elements 700, 710, 720 may be laterally offset from each other in the direction of the light propagating through these internally coupled optical elements, as seen in the illustrated front view. In some embodiments, each internally coupled optical element may be offset so that its light is received without passing through another internally coupled optical element. For example, each of the internally coupled optical elements 700, 710, 720 may be configured to receive light from different image input devices 360, 370, 380, 390, and 400, as shown in FIG. 6, and may be separated (e.g., laterally spaced) from the other internally coupled optical elements 700, 710, 720 so as to substantially not receive light from the other internally coupled optical elements 700, 710, 720.

[0180] Each waveguide also includes an associated light dispersing element. For example, the light dispersing element 730 is disposed on the major surface (e.g., upper major surface) of the waveguide 670, the light dispersing element 740 is disposed on the major surface (e.g., upper major surface) of the waveguide 680, and the light dispersing element 750 is disposed on the major surface (e.g., upper major surface) of the waveguide 690. In some other embodiments, the light dispersing elements 730, 740, 750 may each be disposed on the bottom major surface of the associated waveguides 670, 680, 690. In some other embodiments, the light dispersing elements 730, 740, 750 may each be disposed on both the upper and bottom major surfaces of the associated waveguides 670, 680, 690, or the light dispersing elements 730, 740, 750 may each be disposed on different ones of the upper and bottom major surfaces within different associated waveguides 670, 680, 690.

[0181] Waveguides 670, 680, 690 may be separated and isolated, for example, by a gas, liquid, and / or solid layer of material. For example, as shown, layer 760a may separate waveguides 670 and 680, and layer 760b may separate waveguides 680 and 690. In some embodiments, layers 760a and 760b are formed from a low refractive index material (i.e., a material having a lower refractive index than the material forming the immediate waveguides 670, 680, 690). Preferably, the refractive index of the material forming layers 760a, 760b is 0.05 or more or 0.10 or less than the refractive index of the material forming waveguides 670, 680, 690. Advantageously, the lower refractive index layers 760a, 760b may function as cladding layers that promote total internal reflection (TIR) of light (e.g., TIR between the upper and bottom major surfaces of each waveguide) through waveguides 670, 680, 690. In some embodiments, layers 760a, 760b are formed from air. It should be understood that although not shown, the top and bottom of the illustrated set 660 of waveguides may include an immediate cladding layer.

[0182] Preferably, for ease of manufacturing and other considerations, the materials forming waveguides 670, 680, 690 are similar or identical, and the materials forming layers 760a, 760b are similar or identical. In some embodiments, the materials forming waveguides 670, 680, 690 may be different between one or more waveguides, and / or the materials forming layers 760a, 760b may still be different while maintaining the various refractive index relationships described above.

[0183] Continuing to refer to FIG. 9A, light rays 770, 780, 790 are incident on the set 660 of waveguides. It should be understood that light rays 770, 780, 790 may be input into waveguides 670, 680, 690 by one or more image input devices 360, 370, 380, 390, 400 (FIG. 6).

[0184] In some embodiments, the light rays 770, 780, 790 may have different properties, such as different wavelengths or different wavelength ranges, corresponding to different colors. Each of the internal coupling optical elements 700, 710, 720 deflects the incident light so that light propagates through an individual one of the waveguides 670, 680, 690 by TIR. In some embodiments, each of the internal coupling optical elements 700, 710, 720 selectively deflects one or more specific wavelengths of light while transmitting other wavelengths to the underlying waveguide and the associated internal coupling optical element.

[0185] For example, the internal coupling optical element 700 may be configured to deflect the light ray 770 having a first wavelength or wavelength range while transmitting the light rays 780 and 790 having different second and third wavelengths or wavelength ranges, respectively. The transmitted light ray 780 impinges on the internal coupling optical element 710 configured to deflect the light of the second wavelength or wavelength range, thereby being deflected. The light ray 790 is deflected by the internal coupling optical element 720 configured to selectively deflect the light of the third wavelength or wavelength range.

[0186] Continuing to refer to FIG. 9A, the deflected light rays 770, 780, 790 are deflected to propagate through the corresponding waveguides 670, 680, 690. That is, the internal coupling optical elements 700, 710, 720 of each waveguide deflect the light into its corresponding waveguide 670, 680, 690 and internally couple the light into the corresponding waveguide. The light rays 770, 780, 790 are deflected at an angle that causes the light to propagate through the individual waveguides 670, 680, 690 by TIR. The light rays 770, 780, 790 propagate through the individual waveguides 670, 680, 690 by TIR until they impinge on the corresponding light dispersion elements 730, 740, 750 of the waveguide.

[0187] Referring now to FIG. 9B, a perspective view of an embodiment of the plurality of stacked waveguides of FIG. 9A is illustrated. As described above, the internally coupled light rays 770, 780, 790 are each deflected by the internally coupled optical elements 700, 710, 720 and then each propagate by TIR within waveguides 670, 680, 690, respectively. The light rays 770, 780, 790 then each impinge on the light dispersion elements 730, 740, 750. The light dispersion elements 730, 740, 750 each deflect the light rays 770, 780, 790 so as to propagate towards the externally coupled optical elements 800, 810, 820, respectively.

[0188] In some embodiments, the optical dispersion elements 730, 740, 750 are orthogonal pupil expanders (OPEs). In some embodiments, the OPE deflects or disperses light to the external coupling optical elements 800, 810, 820, and in some embodiments, also increases the beam or spot size of the present light as it propagates to the external coupling optical elements. In some embodiments, the optical dispersion elements 730, 740, 750 may be omitted, and the internal coupling optical elements 700, 710, 720 may be configured to deflect light directly to the external coupling optical elements 800, 810, 820. For example, referring to FIG. 9A, the optical dispersion elements 730, 740, 750 may each be replaced with the external coupling optical elements 800, 810, 820. In some embodiments, the external coupling optical elements 800, 810, 820 are an exit pupil (EP) or an exit pupil expander (EPE) that directs light to the viewer's eye 210 (FIG. 7). The OPE may be configured to increase the size of the eyebox in at least one axis, and it should be understood that the EPE may increase the eyebox in an axis that intersects, for example, is orthogonal to, the axis of the OPE. For example, each OPE may be configured to redirect a portion of the light impinging on the OPE to the EPE of the same waveguide while allowing the remaining portion of the light to continue to propagate along the waveguide. In response to the collision with the OPE, again, another portion of the remaining light is redirected to the EPE, and the remaining portion of that portion continues to propagate further along the waveguide or the like. Similarly, in response to the collision with the EPE, a portion of the colliding light is directed from the waveguide towards the user, and the remaining portion of that light continues to propagate through the waveguide until it impinges on the EP again, at which point another portion of the colliding light is directed from the waveguide, etc. As a result, a single beam of internally coupled light is "replicated" each time a portion of that light is redirected by the OPE or the EPE, thereby forming a beam field of cloned light as shown in FIG. 6. In some embodiments, the OPE and / or the EPE may be configured to modify the size of the beam of light.

[0189] Thus, referring to FIGS. 9A and 9B, in some embodiments, a waveguide set 660 includes, for each primary color, waveguides 670, 680, 690, internal coupling optical elements 700, 710, 720, light dispersion elements (e.g., OPE) 730, 740, 750, and external coupling optical elements (e.g., EP) 800, 810, 820. The waveguides 670, 680, 690 may be stacked with a gap / cladding layer between each one. The internal coupling optical elements 700, 710, 720 redirect or deflect the incident light into their respective waveguides (using different internal coupling optical elements that receive light of different wavelengths). The light then propagates at an angle that will result in TIR within the individual waveguides 670, 680, 690. In the illustrated example, the ray 770 (e.g., blue light) is polarized by the first internal coupling optical element 700 in the manner described above and then continues to bounce along the waveguide, interacting with the light dispersion element (e.g., OPE) 730 and then the external coupling optical element (e.g., EP) 800. The rays 780 and 790 (e.g., green and red light, respectively) pass through the waveguide 670, and the ray 780 impinges on the internal coupling optical element 710 and is thereby deflected. The ray 780 then bounces along the waveguide 680 via TIR and will proceed to its light dispersion element (e.g., OPE) 740 and then the external coupling optical element (e.g., EP) 810. Finally, the ray 790 (e.g., red light) passes through the waveguide 690 and impinges on the internal coupling optical element 720 of the waveguide 690. The internal coupling optical element 720 deflects the ray 790 such that the ray propagates by TIR to the light dispersion element (e.g., OPE) 750 and then by TIR to the external coupling optical element (e.g., EP) 820. The external coupling optical element 820 then finally externally couples the ray 790 to the viewer, who also receives the externally coupled light from the other waveguides 670, 680.

[0190] FIG. 9C illustrates a top and bottom plan view of an embodiment of the plurality of stacked waveguides of FIGS. 9A and 9B. This top and bottom view may also be referred to as a front-on view, as seen in the direction of propagation of light towards the internal coupling optical elements 800, 810, 820, i.e., it should be understood that the top and bottom view is a view of the waveguide in which the image light is incident normal to the page. As shown, waveguides 670, 680, 690 may be vertically aligned with the associated light dispersion elements 730, 740, 750 and the associated external coupling optical elements 800, 810, 820 of each waveguide. However, as discussed herein, the internal coupling optical elements 700, 710, 720 are not vertically aligned. Rather, the internal coupling optical elements are preferably non-overlapping (e.g., laterally spaced as seen in the top and bottom view). As further discussed herein, this non-overlapping spatial arrangement facilitates the injection of light from different sources into different waveguides on a one-to-one basis, thereby enabling a specific light source to be uniquely coupled to a specific waveguide. In some embodiments, an array comprising non-overlapping spatially separated internal coupling optical elements may be referred to as a shifted pupil system, and the internal coupling optical elements within these arrays may correspond to sub-pupils.

[0191] It should be understood that the spatially overlapping area may have a lateral overlap of 70% or more, 80% or more, or 90% or more of that area, as seen in the top and bottom view. On the other hand, the laterally shifted area has less than 30%, less than 20%, or less than 10% of that area overlapping, as seen in the top and bottom view. In some embodiments, the laterally shifted area has no overlap.

[0192] FIG. 9D illustrates a top and bottom plan view of another embodiment of a plurality of stacked waveguides. As shown, waveguides 670, 680, 690 may be vertically aligned. However, compared to the configuration of FIG. 9C, separate optical dispersion elements 730, 740, 750 and associated external coupling optical elements 800, 810, 820 are omitted. Instead, the optical dispersion elements and the external coupling optical elements are, in effect, superimposed and occupy the same area as seen in the top and bottom views. In some embodiments, an optical dispersion element (e.g., OPE) may be disposed on one major surface of waveguides 670, 680, 690, and an external coupling optical element (e.g., EPE) may be disposed on the other major surface of those waveguides. Thus, each of waveguides 670, 680, 690 may collectively have superimposed optical dispersion and external coupling optical elements, respectively, referred to as combined OPE / EPEs 1281, 1282, 1283. Further details regarding such combined OPE / EPEs can be found in U.S. Patent Application No. 16 / 221,359, filed on December 14, 2018, the entire disclosure of which is incorporated herein by reference. Internal coupling optical elements 700, 710, 720 internally couple light and direct it, respectively, to combined OPE / EPEs 1281, 1282, 1283. In some embodiments, as shown, internal coupling optical elements 700, 710, 720 may be laterally offset if they have an offset pupil spatial arrangement (e.g., they are laterally separated as seen in the top and bottom views shown). Similar to the configuration of FIG. 9C, this laterally offset spatial arrangement facilitates the injection of light of different wavelengths into different waveguides on a one-to-one basis (e.g., from different light sources).

[0193] FIG. 9E illustrates an embodiment of a wearable display system 60 in which various waveguides and related systems disclosed herein may be integrated. In some embodiments, display system 60 is the system 250 of FIG. 6, which schematically shows some parts of that system 60 in more detail. For example, the waveguide assembly 260 of FIG. 6 may be part of display 70.

[0194] Continuing to refer to FIG. 9E, the display system 60 includes a display 70 and various mechanical and electronic modules and systems for supporting the functions of the display 70. The display 70 may be coupled to a frame 80, which is wearable by a display system user or viewer 90 and is configured to position the display 70 in front of the user 90's eyes. In some embodiments, the display 70 may be regarded as eyewear. The display 70 may include one or more waveguides, such as waveguide 270, configured to relay internally coupled image light and output the image light to the eyes of the user 90. In some embodiments, a speaker 100 is coupled to the frame 80 and configured to be positioned adjacent to the outer ear canal of the user 90 (in some embodiments, another speaker, not shown, may also be optionally positioned adjacent to the other outer ear canal of the user to provide stereo / formable sound control). The display system 60 may also include one or more microphones 110 or other devices for detecting sound. In some embodiments, the microphone is configured to enable the user to provide an input or command to the system 60 (e.g., selection of a voice menu command, natural language question, etc.) and / or enable audio communication with other persons (e.g., other users of similar display systems). The microphone may further be configured as a peripheral sensor to collect audio data (e.g., sound from the user and / or the environment). In some embodiments, the display system 60 may further include one or more outwardly directed environmental sensors 112 configured to detect objects, stimuli, people, animals, locations, or other aspects of the world around the user. For example, the environmental sensor 112 may include one or more cameras, which may be positioned facing outward to capture an image similar to at least a portion of the normal field of view of the user 90. In some embodiments, the display system may also include a peripheral sensor 120a, which is separate from the frame 80 and may be attached to the body of the user 90 (e.g., the head, torso, limbs, etc. of the user 90).In some embodiments, the peripheral sensor 120a may be configured to obtain data characterizing the physiological state of the user 90. For example, the sensor 120a may be an electrode.

[0195] Continuing to refer to FIG. 9E, the display 70 is operably coupled to a local data processing module 140 by a communication link 130 such as a wired conductor or wireless connectivity, which may be mounted in various configurations such as fixedly attached to the frame 80, fixedly attached to a helmet or hat worn by the user, built into headphones, or removably attached to the user 90 in some other way (e.g., in a backpack configuration, in a belt attachment configuration). Similarly, the sensor 120a may be operably coupled to the local data processing module 140 by a communication link 120b, such as a wired conductor or wireless connectivity. The local processing and data module 140 may comprise digital memory such as a hardware processor and non-volatile memory (e.g., flash memory or hard disk drive), both of which may be utilized to assist in the processing, caching, and storage of data. Optionally, the local processing and data module 140 may include one or more central processing units (CPUs), graphics processing units (GPUs), dedicated processing hardware, etc. The data may include a) data captured from sensors (image capture devices (such as cameras), microphones, inertial measurement units, accelerometers, compasses, GPS units, wireless devices, gyroscopes, and / or other sensors disclosed herein (e.g., operably coupled to the frame 80 or otherwise attachable to the user 90)), and / or b) potentially, data obtained and / or processed using the remote processing module 150 and / or remote data repository 160 (including data related to virtual content) for passage to the display 70 after processing or retrieval. The local processing and data module 140 may be operably coupled to the remote processing module 150 and remote data repository 160 by communication links 170, 180 via a wired or wireless communication link or the like such that these remote modules 150, 160 are operably coupled to each other and available as resources to the local processing and data module 140.In some embodiments, the local processing and data module 140 may include one or more of an image capture device, a microphone, an inertial measurement unit, an accelerometer, a compass, a GPS unit, a wireless device, and / or a gyroscope. In some other embodiments, one or more of these sensors may be attached to the frame 80 or may be an independent structure that communicates with the local processing and data module 140 via a wired or wireless communication path.

[0196] Continuing to refer to FIG. 9E, in some embodiments, the remote processing module 150 may include one or more processors configured to analyze and process data and / or image information, for example, including one or more central processing units (CPUs), graphics processing units (GPUs), dedicated processing hardware, etc. In some embodiments, the remote data repository 160 may include a digital data storage facility, which may be available through the Internet or other networking configurations in a “cloud” resource configuration. In some embodiments, the remote data repository 160 may include one or more remote servers that provide information, such as information for generating virtual content, to the local processing and data module 140 and / or the remote processing module 150. In some embodiments, all data is stored and all calculations are performed in the local processing and data module, enabling fully autonomous use from the remote module. Optionally, an external system (e.g., one or more processors, one or more computer systems) including a CPU, GPU, etc. may perform at least a portion of the processing (e.g., generating image information, processing data), for example, providing information to and receiving information from modules 140, 150, 160 via a wireless or wired connection.

[0197] FIG. 10 illustrates an embodiment of a wearable display system with an optical projection system 910 having a spatial light modulator 930 and a separate light source 940. The light source 940 may include one or more light emitters and illuminate a spatial light modulator (SLM) 930. A lens structure 960 may be used to focus light from the light source 940 onto the SLM 930. A beam splitter (e.g., a polarizing beam splitter (PBS)) 950 reflects light from the light source 940 to the spatial light modulator 930, which reflects and modulates the light. The reflected and modulated light, also referred to as image light, then propagates through the beam splitter 950 to an eyepiece lens 920. A projection optical system 970, which is another lens structure, may be utilized to converge or focus the image light onto the eyepiece lens 920. The eyepiece lens 920 may include one or more waveguides or waveguides that relay the modulated light to the eye 210.

[0198] As described herein, the separate light source 940 and associated lens structure 960 can undesirably add weight and size to the wearable display system. This can reduce the comfort of the display system, particularly for a user wearing the display system for an extended period of time.

[0199] In addition, the light source 940 can consume energy inefficiently in conjunction with the SLM 930. For example, the light source 940 may illuminate the entire SLM 930. The SLM 930 then selectively reflects the light towards the eyepiece lens 920. Thus, not all of the light produced by the light source 940 can be utilized to form an image. A portion of this light, e.g., the light corresponding to dark regions of the image, is not reflected to the eyepiece lens 920. As a result, the light source 940 utilizes energy to generate light to illuminate the entire SLM 930, but only a certain percentage of this light may be required to form some images.

[0200] Furthermore, as described herein, in some cases, the SLM930 can modulate light and selectively reflect incident light by using micromirrors or by using liquid crystal molecules that modify the amount of light reflected from the underlying mirror. As a result, such devices require physical movement of optical elements (e.g., micromirrors or liquid crystal molecules in an LCoS or DLP panel, respectively) to modulate the light from the light source 940. The physical movement required to modulate the light and encode the light with image information corresponding to, for example, pixels can occur at a relatively low speed compared to, for example, the ability to turn an LED or OLED "on" or "off". This relatively low speed movement can limit the frame rate of the display system and can be visible as, for example, motion blur, color breakup, and / or an inconsistent presented image with respect to the user's head pose or a change in that pose.

[0201] Advantageously, a wearable display that utilizes a light-emitting microdisplay as disclosed herein can facilitate a wearable display system that has relatively low weight and bulk, high energy efficiency, and a high frame rate, with low motion blur and short latency from motion to image rendering. The low blur and short latency from motion to image rendering are further discussed in U.S. Provisional Application No. 62 / 786,199, filed Dec. 28, 2018, the entire disclosure of which is incorporated herein by reference. Additionally, compared to a scanned fiber display, a light-emitting microdisplay can avoid artifacts caused by the use of a coherent light source.

[0202] Referring now to FIG. 11A, an example of a wearable display system is illustrated, with a light projection system 1010 having a plurality of light-emitting microdisplays 1030a, 1030b, 1030c. The light from the microdisplays 1030a, 1030b, 1030c is combined by an optical combiner 1050 and directed toward an eyepiece 1020, which relays the light to the user's eye 210. A projection optical system 1070 may be provided between the optical combiner 1050 and the eyepiece 1020. In some embodiments, the eyepiece 1020 may be a waveguide assembly comprising one or more waveguides. In some embodiments, the light projection system 1010 and the eyepiece 1020 may be supported (e.g., attached) by a frame 80 (FIG. 9E).

[0203] In some embodiments, the microdisplays 1030a, 1030b, 1030c may be monochrome microdisplays, each monochrome microdisplay outputting light of a different primary color and providing a monochrome image. As discussed herein, the monochrome images are combined to form a full-color image.

[0204] In some other embodiments, the microdisplays 1030a, 1030b, 1030c may each be a full-color display configured to output light of all primary colors. For example, the microdisplays 1030a, 1030b, 1030c each include red, green, and blue light emitters. The microdisplays 1030a, 1030b, 1030c may be the same and may display the same image. However, using a plurality of microdisplays can provide the advantage of increasing the brightness and brightness dynamic range of the image by combining the light from the plurality of microdisplays to form a single image. In some embodiments, two or more (e.g., three) microdisplays may be utilized, and the optical combiner 1050 is configured to combine the light from all of these microdisplays.

[0205] The microdisplay may comprise an array of light emitters. Examples of light emitters include organic light emitting diodes (OLEDs) and micro light emitting diodes (microLEDs). It should be understood that OLEDs utilize organic materials to emit light, and microLEDs utilize inorganic materials to emit light. Advantageously, some microLEDs provide higher brightness and higher efficiency (from the perspective of lux / W) than OLEDs. In some embodiments, the microdisplay is preferably an emissive microLED display.

[0206] Continuing to refer to FIG. 11A, the microdisplays 1030a, 1030b, 1030c may each be configured to emit image light 1032a, 1032b, 1032c. If the microdisplay is a monochrome microdisplay, the image lights 1032a, 1032b, 1032c may each be different primary colors. The optical combiner 1050 receives the image lights 1032a, 1032b, 1032c and effectively combines this light such that the light generally propagates in the same direction, e.g., towards the projection optical system 1070. In some embodiments, the optical combiner 1050 may be a dichroic X-cube prism having a reflective internal surface that redirects the image lights 1032a, 1032b, 1032c to the projection optical system 1070. It should be understood that the projection optical system 1070 may be a lens structure comprising one or more lenses that converge or focus the image light onto the eyepiece lens 1020. The eyepiece lens 1020 then relays the image lights 1032a, 1032b, 1032c to the eye 210.

[0207] In some embodiments, the eyepiece 1020 may include a plurality of stacked waveguides 1020a, 1020b, 1020c, each having an individual internal coupling optical element 1022a, 1022b, 1022c. In some embodiments, the number of waveguides is proportional to the number of primary colors provided by the microdisplays 1030a, 1030b, 1030c. For example, if there are three primary colors, the number of waveguides within the eyepiece 1020 may include a set of three waveguides or a plurality of sets of three waveguides each. In some embodiments, each set may output light with wavefront divergence corresponding to a particular depth plane, as discussed herein. It should be understood that the waveguides 1020a, 1020b, 1020c and the internal coupling optical elements 1022a, 1022b, 1022c may correspond to the waveguides 670, 680, 690 and the internal coupling optical elements 700, 710, 720 of FIGS. 9A-9C, respectively. As viewed from the projection optics 1070, the internal coupling optical elements 1022a, 1022b, 1022c may be laterally offset so that they do not overlap, at least partially, as seen in such figures.

[0208] As shown, the various internally coupled optical elements disclosed herein (e.g., internally coupled optical elements 1022a, 1022b, 1022c) may be disposed on a major surface of an associated waveguide (e.g., waveguides 1020a, 1020b, 1020c, respectively). Additionally, as shown, the major surface on which a given internally coupled optical element is disposed may be the back surface of the waveguide. In such a configuration, the internally coupled optical element may be a reflective light redirecting element, which internally couples light by reflecting the light at an angle that aids TIR through the associated waveguide. In some other configurations, the internally coupled optical element may be disposed on the front surface of the waveguide (closer to the projection optical system 1070 than the back surface). In such a configuration, the internally coupled optical element may be a transmissive light redirecting element, which internally couples light by changing the propagation direction of the light as it passes through the internally coupled optical element. It should be understood that any of the internally coupled optical elements disclosed herein may be either reflective or transmissive internally coupled optical elements.

[0209] Continuing to refer to FIG. 11A, image light 1032a, 1032b, 1032c from different ones of the microdisplays 1030a, 1030b, 1030c may follow different paths to the eyepiece 1020 such that they impinge on different ones of the internally coupled optical elements 1022a, 1022b, 1022c. If the image light 1032a, 1032b, 1032c includes light of different primary colors, the associated internally coupled optical elements 1022a, 1022b, 1022c may be configured to selectively internally couple light of different wavelengths, respectively, as described above with respect to the internally coupled optical elements 700, 710, 720 of FIGS. 9A-9C, for example.

[0210] Continuing to refer to FIG. 11A, the optical combiner 1050 may be configured to redirect the image light 1032a, 1032b, 1032c emitted by the microdisplays 1030a, 1030b, 1030c so that the image light propagates along different optical paths in order to impinge on appropriately associated ones of the internal coupling optical elements 1022a, 1022b, 1022c. Thus, the optical combiner 1050 combines the image light 1032a, 1032b, 1032c in the sense that the image light is output from a common plane of the optical combiner 1050, but the light exits the optical combiner in slightly different directions. For example, the reflective internal surfaces 1052, 1054 of the X - cube prism may each be angled to direct the image light 1032a, 1032b, 1032c along different paths toward the eyepiece lens 1020. As a result, the image light 1032a, 1032b, 1032c may impinge on different associated ones of the internal coupling optical elements 1022a, 1022b, 1022c. In some embodiments, the microdisplays 1030a, 1030b, 1030c are appropriately angled with respect to the reflective internal surfaces 1052, 1054 of the X - cube prism and may provide the desired optical paths to the internal coupling optical elements 1022a, 1022b, 1022c. For example, the surfaces of one or more of the microdisplays 1030a, 1030b, 1030c may be angled to match the surface of the optical combiner 1050 such that the image light emitted by the microdisplay impinges on the reflective internal surfaces 1052, 1054 at an appropriate angle and propagates toward the associated internal coupling optical element 1022a, 1022b, or 1022c. In addition to a cube, it should be understood that the optical combiner 1050 may take the form of various other polyhedra. For example, the optical combiner 1050 may be in the shape of a right - angled prism having at least two faces, rather than a square.

[0211] Continuing to refer to FIG. 11A, in some embodiments, the monochromatic microdisplay 1030b that directly faces the output surface 1051 advantageously may output green light. It should be understood that the reflective surfaces 1052, 1054 may have optical losses when reflecting light from the microdisplay. Additionally, the human eye is most sensitive to the color green. As a result, the monochromatic microdisplay 1030b that faces the output surface 1051 preferably outputs green light so that the green light can proceed directly through the optical combiner 1050 without the need to be reflected for output from the optical combiner 1050. However, it will be understood that the green monochromatic microdisplay may face other surfaces of the optical combiner 1050 in some other embodiments.

[0212] As discussed herein, the perception of a full-color image by a user can, in some embodiments, be achieved using time-division multiplexing. For example, different ones of the emissive microdisplays 1030a, 1030b, 1030c can be activated at different times to generate different primary-color images. In such embodiments, the different primary-color images that form a single full-color image can be sequentially displayed sufficiently rapidly such that the human visual system does not perceive that the primary-color images are being displayed at different times. That is, all of the different primary-color images that form a single full-color image can be displayed within a duration that is sufficiently short such that the user perceives the primary-color images as being presented simultaneously rather than being temporally separated. For example, it should be understood that the human visual system can have a flicker fusion threshold. The flicker fusion threshold can be understood as the duration at which the human visual system is unable to distinguish images presented at different times. Images presented within that duration are fused or combined and, as a result, can be perceived by the user as being presented simultaneously. Flicker images with a temporal gap between the images outside of that duration are not combined and the flicker of the images is perceivable. In some embodiments, the duration is 1 / 60 second or less, which corresponds to a frame rate of 60 Hz or more. Preferably, an image frame for any individual eye is provided to the user at a frame rate that is equal to or higher than the duration of the user's flicker fusion threshold. For example, the frame rate for each of the left and right eyepieces can be 60 Hz or more, or 120 Hz or more, and as a result, the frame rate provided by the light projection system 1010 can be 120 Hz or more, or 240 Hz or more, in some embodiments.

[0213] It should be understood that time-division multiplexing can advantageously reduce the computational load on a processor (e.g., a graphics processor) utilized to form the displayed image. In some other embodiments, such as when sufficient computational resources are available, all the primary color images that form a full-color image may be simultaneously displayed by the microdisplays 1030a, 1030b, 1030c.

[0214] As discussed herein, each of the microdisplays 1030a, 1030b, 1030c may include an array of light emitters. FIG. 11B illustrates an example of an array 1042 of light emitters 1044. When the associated microdisplay is a monochrome microdisplay, all of the light emitters 1044 may be configured to emit light of the same color.

[0215] When the associated microdisplay is a full-color microdisplay, different ones of the light emitters 1044 may be configured to emit light of different colors. In such embodiments, the light emitters 1044 may be regarded as subpixels and may be arranged in groups, with each group having at least one light emitter configured to emit light of each primary color. For example, when the primary colors are red, green, and blue, each group may have at least one red subpixel, at least one green subpixel, and at least one blue subpixel.

[0216] It will be understood that the light emitters 1044 are shown arranged in a grid pattern for ease of illustration, but the light emitters 1044 may have other regularly repeating spatial arrangements. For example, the number of light emitters of different primary colors may vary, the size of the light emitters may vary, the shape of the light emitters and / or the shape created by the groups of light emitters may vary, and so on.

[0217] Continuing to refer to FIG. 11B, it should be understood that the micro-emitter 1044 emits light. Additionally, manufacturing constraints such as lithography or other patterning and processing limitations and / or electrical considerations may limit the proximity at which neighboring light emitters 1044 are spaced apart. As a result, there may be an area 1045 surrounding the light emitter 1044 where it is not practical to form other light emitters 1044. This area 1045 forms an inter-emitter region between the light emitters 1044. In some embodiments, considering the area 1045, the light emitters have a pitch that is, for example, less than 10 μm, less than 8 μm, less than 6 μm, or less than 5 μm, and greater than 1 μm including 1 - 5 μm, and the emitter size is 2 μm or less, 1.7 μm or less, or 1.3 μm or less. In some embodiments, the emitter size is within a range having the upper limit of the above sizes and a lower limit of 1 μm. In some embodiments, the ratio of emitter size to pitch is 1:1 to 1:5, 1:2 to 1:4, or 1:2 to 1:3.

[0218] Assuming some light emitter device architectures and materials, it should be understood that current crowding can reduce the efficiency of the emitter and pixel droop can cause unintentional activation of the pixel (e.g., due to energy directed at one light emitter leaking to neighboring light emitters). As a result, the relatively large area 1045 can, beneficially, reduce current crowding and pixel droop. In some embodiments, the ratio of emitter size to pitch is preferably 1:2 to 1:4 or 1:2 to 1:3.

[0219] However, it should also be understood that a relatively large separation between light emitters (e.g., ratio of small light emitter pitch) can undesirably cause visible gaps or dark regions between the light emitters. As discussed herein, some gaps can still be visible depending on the size of the original gap, the distance of translation, and the number of sub-frames utilized (and resulting translation increments) even when translated laterally. In some embodiments, a lens structure such as a light collimator may be utilized to effectively fill or partially fill these dark regions. For example, a light collimating lens may extend over and around the light emitter 1044 such that light from the emitter 1044 completely fills the lens. For example, the light collimating lens may have a width larger than the light emitter 1044, and in some embodiments, the width of the collimating lens may be approximately equal to the pitch. As a result, the size of the emitter 1044 is effectively increased such that it extends across the area of the lens, thereby filling some or all of the area 1045. In some other embodiments, the width of the collimating lens may be approximately equal to the distance by which the projection system is translated for each sub-frame as discussed herein. Lens structures such as light collimators are further discussed herein (e.g., in FIG. 30A and related discussion).

[0220] As discussed herein, the light emitter 1044 may be an OLED or a micro-LED. It should be understood that an OLED may utilize, for example, a layer of organic material disposed between electrodes to emit light. A micro-LED may utilize an inorganic material, such as a Group III-V material such as GaAs, GaN, and / or GaIn, for light emission. An example of a GaN material includes InGaN, which may be used to form a blue or green light emitter in some embodiments. An example of a GaIn material includes AlGaInP, which may be used to form a red light emitter in some embodiments. In some embodiments, the light emitter 1044 may emit light of an initial color, which may be converted to another desired color using a phosphor material or quantum dots. For example, the light emitter may emit blue light, which excites a phosphor material or quantum dots that convert blue wavelength light to a green or red wavelength.

[0221] Referring now to FIG. 12, another example of a wearable display system with a light projection system having a plurality of emissive microdisplays 1030a, 1030b, 1030c is illustrated. The illustrated display system is similar to the display system of FIG. 11A, but the optical combiner 1050 has a standard X-cube prism configuration and includes light redirection structures 1080a and 1080c to modify the angle of incidence of light on the reflective surfaces 1052, 1054 of the X-cube prism. It should be understood that the standard X-cube prism configuration will redirect this light at 45° so as to receive light that is normal to the face of the X-cube and output it at a normal angle from the lateral face of the X-cube. However, this would cause the image light 1032a, 1032b, 1032c to impinge on the same internal coupling optical element of the eyepiece 1020. The light redirection structures 1080a, 1080c may be utilized to provide different paths for the image light 1032a, 1032b, 1032c such that the image light impinges on the associated ones of the internal coupling optical elements 1022a, 1022b, 1022c of the waveguide assembly.

[0222] In some embodiments, the light redirecting structures 1080a, 1080c may be lens structures. It should be understood that the lens structure is configured to receive incident light and redirect the incident light at an angle such that the light reflects from the corresponding ones of the reflective surfaces 1052, 1054 and propagates along the optical path towards the corresponding ones of the internal coupling optical elements 1022a, 1022c. As an example, the light redirecting structures 1080a, 1080c may comprise microlenses, nanolenses, reflective wells, metasurfaces, and liquid crystal gratings. In some embodiments, the microlenses, nanolenses, reflective wells, metasurfaces, and liquid crystal gratings may be arranged in an array. For example, each light emitter of the microdisplays 1030a, 1030c may be aligned with one microlens. In some embodiments, the microlens or the reflective well may be asymmetric and / or the light emitter may be disposed offset from the center with respect to the microlens in order to redirect light in a specific direction. Additionally, in some embodiments, the light redirecting structures 1080a, 1080c may be collimators, which narrow the angular emission profile of the associated light emitter and ultimately increase the amount of light internally coupled into the eyepiece lens 1020. Further details regarding such light redirecting structures 1080a, 1080c are discussed below with respect to FIGS. 24A-27C.

[0223] Referring now to FIG. 13A, in some embodiments, two or more of the internal coupling optical elements 1022a, 1022b, 1022c may overlap (e.g., as seen in a front view in the direction of light propagation into the internal coupling optical elements 1022a, 1022b, 1022c). FIG. 13A illustrates an example of a side view of a wearable display system with an optical projection system 1010 having a plurality of light-emitting microdisplays 1032a, 1032b, 1032c and an eyepiece 1020 with overlapping light internal coupling optical elements 1022a, 1022c and non-overlapping light internal coupling optical element 1022b. As shown, the internal coupling optical elements 1022a, 1022c overlap, while the internal coupling optical element 1022b is laterally offset. In other words, the internal coupling optical elements 1022a, 1022c are directly aligned within the paths of the image lights 1032a, 1032c, while the image light 1032b follows another path to the eyepiece 1020 so as to be incident on an area of the eyepiece 1020 that is laterally offset with respect to the area on which the image lights 1032a, 1032c are incident.

[0224] As shown, the differences between the paths for the image light 1032b and the image lights 1032a, 1032c may be established using the light redirection structures 1080a, 1080c. In some embodiments, the image light 1032b from the emissive microdisplay 1030b travels directly through the optical combiner 1052. The image light 1032a from the emissive microdisplay 1032a is redirected by the light redirection structure 1080a to reflect from the reflective surface 1054 and propagate out from the optical combiner 1050 in the same direction as the image light 1032c. It should be understood that the image light 1032c from the emissive microdisplay 1032c is redirected by the light redirection structure 1080c to reflect from the reflective surface 1052 at an angle such that the image light 1032c propagates out from the optical combiner 1050 in the same direction as the image light 1032b. Thus, the redirection of light by the light redirection structures 1080a, 1080c and the angles of the reflective surfaces 1052, 1054 are configured to provide a common path for the image lights 1032a, 1032c out from the optical combiner 1050, and this common path is different from the path of the image light 1032b. In some other embodiments, one or both of the light redirection structures 1080a, 1080c may be omitted, and the reflective surfaces 1052, 1054 within the optical combiner 1050 may be configured to reflect the image lights 1032a, 1032c in appropriate individual directions so as to propagate out from the optical combiner 1050 in the same direction that is different from the direction of the image light 1032b. Thus, after propagating through the projection optical system 1070, the image lights 1032a, 1032c exit from one exit pupil, while the image light 1032b exits from another exit pupil. In this configuration, the light projection system 1010 may be referred to as a two-pupil projection system.

[0225] In some embodiments, the light projection system 1010 may have a single output pupil and may be referred to as a single pupil projection system. In such embodiments, the light projection system 1010 may be configured to direct the image lights 1032a, 1032b, 1032c onto a single common area of the eyepiece 1020. Such a configuration is illustrated in FIG. 13B, which shows a wearable display system with a light projection system 1010 having a plurality of emissive microdisplays 1030a, 1030b, 1030c configured to direct light onto a single light internal coupling area of the eyepiece 1020. In some embodiments, as further discussed herein, the eyepiece 1020 may include a stack of waveguides having overlapping light internal coupling optical elements. In some other embodiments, the single light internal coupling optical element may be configured to internally couple light of all primary colors into a single waveguide. The display system of FIG. 13B is similar to the display system of FIG. 13A, except for the omission of the light redirecting structures 1080a, 1080c and the combined use of the internal coupling optical element 1122a associated with the waveguide 1020a. As shown, the internal coupling optical element 1122a internally couples each of the image lights 1032a, 1032b, 1032c into the waveguide 1020a, which then relays the image light to the eye 210. In some embodiments, the internal coupling optical element 1122a may comprise a diffraction grating. In some embodiments, the internal coupling optical element 1122a is a metasurface and / or a liquid crystal grating.

[0226] As discussed herein, in some embodiments, the emissive microdisplays 1030a, 1030b, 1030c may be monochrome microdisplays configured to emit light of different colors. In some embodiments, one or more of the emissive microdisplays 1030a, 1030b, 1030c may have a group of light emitters configured to emit light of two or more but not all of the primary colors. For example, a single emissive microdisplay may have a group of light emitters with at least one light emitter per group configured to emit blue light and at least one light emitter per group configured to emit green light, and separate emissive microdisplays on different faces of the X-cube 1050 may have light emitters configured to emit red light. In some other embodiments, the emissive microdisplays 1030a, 1030b, 1030c may each be a full-color display having light emitters for all of the primary colors. As described herein, using multiple similar microdisplays can provide advantages for dynamic range and increased display brightness.

[0227] In some embodiments, a single full-color emissive microdisplay may be utilized. FIG. 14 illustrates an example of a wearable display system with a single emissive microdisplay 1030b. The wearable display system of FIG. 14 is similar to the wearable display system of FIG. 14, but the single emissive microdisplay 1030b is a full-color microdisplay configured to emit light of all of the primary colors. As shown, the microdisplay 1030b emits image light 1032a, 1032b, 1032c for each primary color. In such embodiments, the optical combiner 1050 (FIG. 13B) may be omitted, which advantageously may reduce the weight and size of the wearable display system relative to a system with an optical combiner.

[0228] As discussed above, the internal coupling optical elements of the eyepiece 1020 may take various configurations. Some embodiments of the configuration regarding the eyepiece 1020 will be discussed below in relation to FIGS. 15 - 23C.

[0229] FIG. 15 illustrates side views of embodiments of the eyepiece 1020 having stacks of waveguides 1020a, 1020b, 1020c, each with overlapping internal coupling optical elements 1022a, 1022b, 1022c. It should be understood that the illustrated waveguide stacks may be utilized in place of the single illustrated waveguide 1020a of FIGS. 13B and 14. As discussed herein, the internal coupling optical elements 1022a, 1022b, 1022c are each configured to internally couple light having a specific color (e.g., light of a specific wavelength or range of wavelengths). In the illustrated orientation of the eyepiece 1020 in which image light propagates vertically as it traverses the page towards the eyepiece 1020, the internal coupling optical elements 1022a, 1022b, 1022c are aligned perpendicular to each other (e.g., along an axis parallel to the propagation direction of the image light 1032a, 1032b, 1032c) such that they spatially overlap each other as seen in the top - down view (front - on view in the direction of the image light 1032a, 1032b, 1032c propagating to the internal coupling optical elements).

[0230] Continuing to refer to FIG. 15, as discussed herein, the projection system 1010 (FIGS. 13, 14) is configured to output a first monochromatic image, a second monochromatic image, and a third monochromatic image (e.g., red, green, and blue color images) through a single pupil of the projection system, and the monochromatic images are formed by image lights 1032a, 1032b, 1032c, respectively. The internal coupling optical element 1022c is configured to internally couple the image light 1032c into the waveguide 1020c for the first color image so as to propagate through the waveguide 1020c by multiple total internal reflections at the upper and bottom major surfaces of the waveguide 1020c. The internal coupling optical element 1022b is configured to internally couple the image light 1032b into the waveguide 1020b for the second color image so as to propagate through the waveguide 1020b by multiple total internal reflections at the upper and bottom major surfaces of the waveguide 1020b. The internal coupling optical element 1022a is configured to internally couple the image light 1032a into the waveguide 1020a for the third color image so as to propagate through the waveguide 1020a by multiple total internal reflections at the upper and bottom major surfaces of the waveguide 1020a.

[0231] As discussed herein, the internal coupling optical element 1022c is preferably configured to internally couple substantially all of the incident light 1032c corresponding to the first color image into the associated waveguide 1020c, while allowing substantially all of the incident lights 1032b, 1032a corresponding to the second and third color images, respectively, to be transmitted without internal coupling. Similarly, the internal coupling optical element 1022b is preferably configured to internally couple substantially all of the incident image light 1032b corresponding to the second color image into the associated waveguide 1020b, while allowing substantially all of the incident light corresponding to the third color image to be transmitted without internal coupling.

[0232] In practice, it should be understood that various internal coupling optical elements may not have perfect selectivity. For example, a part of the image light 1032b, 1032a may undesirably be internally coupled into the waveguide 1020c by the internal coupling optical element 1022c, and a part of the incident image light 1032a may undesirably be internally coupled into the waveguide 1020b by the internal coupling optical element 1022b. Further, a part of the image light 1032c may be transmitted through the internal coupling optical element 1022c and may be internally coupled into the waveguides 1020b and / or 1020a by the internal coupling optical elements 1020b and / or 1020a respectively. Similarly, a part of the image light 1032b may be transmitted through the internal coupling optical element 1022b and may be internally coupled into the waveguide 1020a by the internal coupling optical element 1022a.

[0233] Internally coupling the image light for a color image into an unintended waveguide can cause undesirable optical effects such as crosstalk and / or afterimages. For example, the internal coupling of the image light 1032c for the first color image into the unintended waveguides 1020b and / or 1020a can result in undesirable crosstalk between the first color image, the second color image, and / or the third color image, and / or can result in undesirable afterimages. As another example, the internal coupling of the image light 1032b, 1032a for the second or third color image respectively into the unintended waveguide 1020c can result in undesirable crosstalk between the first color image, the second color image, and / or the third color image, and / or can cause undesirable afterimages. In some embodiments, these undesirable optical effects can be reduced by providing a color filter (e.g., an absorptive color filter) that can reduce the amount of incident light that is internally coupled into the unintended waveguide.

[0234] FIG. 16 illustrates a side view of an embodiment of a stack of waveguides with color filters to reduce afterglow or crosstalk between waveguides. The eyepiece lens 1020 of FIG. 16 is similar to that of FIG. 15 except for the presence of one or more of the color filters 1024c, 1024b, 1028, and 1026. The color filters 1024c and 1024b are each configured to reduce the amount of light that is unintentionally internally coupled into the waveguides 1020b and 1020a, respectively. The color filters 1028 and 1026 are each configured to reduce the amount of unintentionally internally coupled image light that propagates through the waveguides 1020b and 1020c, respectively.

[0235] Continuing to refer to FIG. 16, a pair of color filters 1026 disposed on the upper and lower major surfaces of the waveguide 1020c may be configured to absorb the image light 1032a and 1032b that may be unintentionally internally coupled into the waveguide 1020c. In some embodiments, the color filter 1024c disposed between the waveguides 1020c and 1020b is configured to absorb the image light 1032c that is transmitted through the internal coupling optical element 1022c without being internally coupled. A pair of color filters 1028 disposed on the upper and lower major surfaces of the waveguide 1020b are configured to absorb the image light 1032a that is internally coupled into the waveguide 1020b. The color filter 1024b disposed between the waveguides 1020b and 1020a is configured to absorb the image light 1032b that is transmitted through the internal coupling optical element 710.

[0236] In some embodiments, the color filters 1026 on each major surface of the waveguide 1020c are similar and configured to absorb light of both wavelengths of the image light 1032a, 1032b. In some other embodiments, the color filter 1026 on one major surface of the waveguide 1020c may be configured to absorb light of the color of the image light 1032a, and the color filter on the other major surface may be configured to absorb light of the color of the image light 1032b. In either arrangement, the color filters 1026 may be configured to selectively absorb the image light 1032a, 1032b propagating through the waveguide 1020c by total internal reflection. For example, in the TIR bounces of the image light 1032a, 1032b from the major surfaces of the waveguide 1020c, the image light 1032a, 1032b contacts the color filters 1026 on their major surfaces, and a portion of that image light is absorbed. Preferably, due to the selective absorption of the image light 1032a, 1032b by the color filters 1026, the propagation of the image light 1032c internally coupled through the waveguide 1020c via TIR is not significantly affected.

[0237] Similarly, a plurality of color filters 1028 may be configured as absorption filters that absorb the internally coupled image light 1032a propagating through the waveguide 1020b by total internal reflection. In the TIR bounces of the image light 1032a from the major surfaces of the waveguide 1020b, the image light 1032a contacts the color filters 1028 on their major surfaces, and a portion of that image light is absorbed. Preferably, the absorption of the image light 1032a is selective and does not affect the propagation of the internally coupled image light 1032b that also propagates through the waveguide 1020b via TIR.

[0238] Continuing to refer to FIG. 16, color filters 1024c and 1024b may also be configured as absorption filters. Color filter 1024c may be substantially transparent to light of the colors of image lights 1032a and 1032b such that image lights 1032a and 1032b are transmitted through color filter 1024c with little or no attenuation, while light of the color of image light 1032c is selectively absorbed. Similarly, color filter 1024b may be substantially transparent to light of the color of image light 1032a such that incident image light 1032a is transmitted through color filter 1024b with little or no attenuation, while light of the color of image light 1032b is selectively absorbed. Color filter 1024c may be disposed on a major surface (e.g., the upper major surface) of waveguide 1020b as shown in FIG. 16. Alternatively, color filter 1024c may be disposed on a separate substrate positioned between waveguides 1020c and 1020b. Similarly, color filter 1024b may be disposed on a major surface (e.g., the upper major surface) of waveguide 1020a. Alternatively, color filter 1024b may be disposed on a separate substrate positioned between waveguides 1020b and 1020a. Color filters 1024c and 1024b may be aligned perpendicular to a single pupil of a projector that outputs image lights 1032a, 1032b, and 1032c (in an orientation in which image lights 1032a, 1032b, and 1032c propagate perpendicular to waveguide stack 1020 as shown).

[0239] In some embodiments, color filters 1026 and 1028 may have a single-pass attenuation coefficient of less than about 10% (e.g., less than or equal to about 5%, less than or equal to about 2%, and greater than about 1%) to avoid significant unwanted absorption of light (e.g., light of the color of image light 1032a, 1032b propagating through the thickness of waveguides 1020c, 1020b and propagating from the surrounding environment and / or other waveguides through waveguides 1020c, 1020b). Various embodiments of color filters 1024c and 1024b may be configured to have a low attenuation coefficient for wavelengths to be transmitted and a high attenuation coefficient for wavelengths to be absorbed. For example, in some embodiments, color filter 1024c is configured to transmit more than 80%, more than 90%, or more than 95% of the incident light having the color of image light 1032a, 1032b and absorb more than 80%, more than 90%, or more than 95% of the incident light having the color of image light 1032a. Similarly, color filter 1024b may be configured to transmit more than 80%, more than 90%, or more than 95% of the incident light having the color of image light 1032a and absorb more than 80%, more than 90%, or more than 95% of the incident light having the color of image light 1032b.

[0240] In some embodiments, color filters 1026, 1028, 1024c, 1024b may comprise a layer of color-selective light-absorbing material deposited on one or both surfaces of waveguides 1020c, 1020b, and / or 1020a. The color-selective light-absorbing material may comprise a dye, ink, or other light-absorbing materials such as metals, semiconductors, and dielectrics. In some embodiments, the light absorption of materials such as metals, semiconductors, and dielectrics may be color-selectively achieved by using these materials to form a sub-wavelength grating (e.g., a grating that does not diffract light). The grating may be made of plasmonics (e.g., gold, silver, and aluminum) or semiconductors (e.g., silicon, amorphous silicon, and germanium).

[0241] Color - selective materials may be deposited on a substrate using various deposition methods. For example, color - selective light - absorbing materials may be deposited on a substrate using jet deposition techniques (e.g., ink - jet deposition). Ink - jet deposition can facilitate the deposition of thin layers of color - selective light - absorbing materials. Ink - jet deposition provides a high degree of control over the thickness and composition of the layer of color - selective light - absorbing material, including the ability to provide non - uniform thickness and / or composition across the substrate since the deposition can be localized on a selected area of the substrate. In some embodiments, the color - selective light - absorbing material deposited using ink - jet deposition may have a thickness of about 10 nm to about 1 micron (e.g., about 10 nm to about 50 nm, about 25 nm to about 75 nm, about 40 nm to about 100 nm, about 80 nm to about 300 nm, about 200 nm to about 500 nm, about 400 nm to about 800 nm, about 500 nm to about 1 micron, or any value within the range / sub - range defined by any of these values). Controlling the thickness of the layer on which the color - selective light - absorbing material is deposited can be advantageous in achieving a color filter with a desired attenuation coefficient. Additionally, layers with different thicknesses may be deposited on different parts of the substrate. In addition, different compositions of the color - selective light - absorbing material may be deposited on different parts of the substrate using ink - jet deposition. Such variations in composition and / or thickness may advantageously allow for location - specific variations in light absorption. For example, in areas of a waveguide where light transmission from the surroundings (to allow the viewer to see the surrounding environment) is not required, the composition and / or thickness may be selected to provide a high absorption or attenuation rate for a selected wavelength of light. Other deposition methods such as coating, spin - coating, spraying, etc. may also be employed to deposit the color - selective light - absorbing material on the substrate.

[0242] FIG. 17 illustrates an example of a top and bottom view of the waveguide assemblies of FIGS. 15 and 16. As shown, the internal coupling optical elements 1022a, 1022b, 1022c are spatially overlapping. Additionally, the waveguides 1020a, 1020b, 1020c may be vertically aligned together with the associated optical dispersion elements 730, 740, 750 and the associated external coupling optical elements 800, 810, 820 of each waveguide. The internal coupling optical elements 1022a, 1022b, 1022c are configured such that the incident image light 1032a, 1032b, 1032c (FIGS. 15 and 16) internally couples into the waveguides 1020a, 1020b, 1020c, respectively, so that the image light propagates towards the associated optical dispersion elements 730, 740, 750 by TIR.

[0243] FIG. 18 illustrates another example of a top and bottom view of the waveguide assemblies of FIGS. 15 and 16. As in FIG. 17, the internal coupling optical elements 1022a, 1022b, 1022c are spatially overlapping and the waveguides 1020a, 1020b, 1020c are vertically aligned. However, instead of the associated optical dispersion elements 730, 740, 750 and the associated external coupling optical elements 800, 810, 820 of each waveguide, there are respectively combined OPE / EPEs 1281, 1282, 1283. The internal coupling optical elements 1022a, 1022b, 1022c are configured such that the incident image light 1032a, 1032b, 1032c (FIGS. 15 and 16) internally couples into the waveguides 1020a, 1020b, 1020c, respectively, so that the image light propagates towards the associated combined OPE / EPEs 1281, 1282, 1283 by TIR.

[0244] Figures 15-18 show overlapping internal combining optical elements for a single pupil configuration of the display system, but it should be understood that the display system may have a two-pupil configuration in some embodiments. In such a configuration where three primary colors are utilized, the image light for two of the colors may have overlapping internal combining optical elements, while the image light for the third color may have laterally displaced internal combining optical elements. For example, the optical combiner 1050 (Figs. 11A, 12, 13A-13B) and / or the light redirecting structures 1080a, 1080c may be configured to direct the image light through the projection optics 1070 such that the image light of two colors is incident directly on the overlapping area of the eyepiece lens 1020, while the image light of another color is incident on an area that is laterally displaced. For example, the reflective surfaces 1052, 1054 (Fig. 11A) may be angled such that the image light of one color follows a common optical path with the image light from the emissive microdisplay 1030b, while the image light of another color follows a different optical path. In some embodiments, rather than having both of the light redirecting structures 1080a, 1080c (Fig. 12), one of these light redirecting structures may be omitted such that only the light from one of the microdisplays 1030a, 1030c is angled to provide a different optical path from the light emitted by the other two microdisplays.

[0245] FIG. 19A illustrates a side view of an embodiment of an eyepiece having a stack of waveguides with several overlapping internal coupling optical elements and several laterally offset internal coupling optical elements. The eyepiece of FIG. 19A is similar to the eyepiece of FIG. 15, except that one of the internal coupling optical elements is laterally offset with respect to the other internal coupling optical element. In the illustrated orientation of the eyepiece 1020 in which image light propagates vertically through the page towards the eyepiece 1020, the internal coupling optical elements 1022a, 1022c are mutually perpendicularly aligned (e.g., along an axis parallel to the propagation direction of the image light 1032a, 1032c) such that they spatially overlap each other as seen in a front-on view in the direction of the image light 1032a, 1032c propagating to the internal coupling optical elements 1022a, 1022c. As seen in the same front-on view (e.g., as seen in the top and bottom views in the illustrated orientation), the internal coupling optical element 1022b is laterally offset with respect to the other internal coupling optical elements 1022a, 1022c. Light for the internal coupling optical element 1022b is output to the eyepiece 1020 through a different exit pupil than the light for the internal coupling optical elements 1022a, 1022c. It should be understood that the illustrated waveguide stack comprising waveguides 1020a, 1020b, 1020c may be utilized instead of the single illustrated waveguide 1020a of FIGS. 13 and 14.

[0246] Continuing to refer to FIG. 19, the internal coupling optical element 1022c is configured to internally couple the image light 1032c into the waveguide 1020c so as to propagate through the waveguide 1020c by a plurality of total internal reflections between the upper major surface and the bottom major surface of the waveguide 1020c, the internal coupling optical element 1022b is configured to internally couple the image light 1032b into the waveguide 1020b so as to propagate through the waveguide 1020b by a plurality of total internal reflections between the upper major surface and the bottom major surface of the waveguide 1020b, and the internal coupling optical element 1022a is configured to internally couple the image light 1032a into the waveguide 1020a so as to propagate through the waveguide 1020a by a plurality of total internal reflections between the upper major surface and the bottom major surface of the waveguide 1020a.

[0247] The internal coupling optical element 1022c is preferably configured to internally couple all incident light 1032c into the associated waveguide 1020c while transmitting all incident light 1032a. On the other hand, the image light 1032b can propagate to the internal coupling optical element 1022b without having to propagate through any other internal coupling optical element. This can be advantageous in some embodiments by allowing light to which the eye is more sensitive to be incident on the desired internal coupling optical element without any loss or distortion associated with propagation through other internal coupling optical elements. Without being limited by theory, in some embodiments, the image light 1032b is green light to which the human eye is more sensitive. The waveguides 1020a, 1020b, 1020c are shown arranged in a particular order, but it should be understood that in some embodiments, the order of the waveguides 1020a, 1020b, 1020c can be different.

[0248] As discussed herein, it should be understood that the internal coupling optical element 1022c above the internal coupling optical element 1022a may not have perfect selectivity. A portion of the image light 1032a may undesirably be internally coupled by the internal coupling optical element 1022c into the waveguide 1020c, and a portion of the image light 1032c may be transmitted through the internal coupling optical element 1022c and then the image light 1032c may impinge on the internal coupling optical element 1020a and be internally coupled into the waveguide 1020a. As discussed herein, such undesirable internal coupling can be visible as ghosting or crosstalk.

[0249] FIG. 19B illustrates a side view of an embodiment of the eyepiece lens of FIG. 19A with a color filter for reducing afterimages or crosstalk between waveguides. In particular, color filters 1024c and / or 1026 are added to the structure shown in FIG. 19A. As shown, internal coupling optical element 1022c may inadvertently internally couple a portion of image light 1032a into waveguide 1020c. Additionally, or alternatively, a portion of image light 1032c may undesirably pass through internal coupling optical element 1022c and then be inadvertently internally coupled by internal coupling optical element 1022a.

[0250] To reduce the inadvertent internal coupling of image light 1032a propagating through waveguide 1022c, an absorptive color filter 1026 may be provided on one or both major surfaces of waveguide 1022c. The absorptive color filter 1026 may be configured to absorb light of the color of image light 1032a that is inadvertently internally coupled. As shown, the absorptive color filter 1026 is disposed in the general propagation direction of the image light through waveguide 1020c. Thus, the absorptive color filter 1026 is configured to absorb image light 1032a as the light propagates through waveguide 1020c by TIR and contacts the absorptive color filter 1026 as it reflects from one or both of the major surfaces of waveguide 1020c.

[0251] Continuing to refer to FIG. 19B, an absorptive color filter 1024c may be provided in front of the internally coupled optical element 1022a to reduce the image light 1032c that propagates through the internally uncoupled internal coupling optical element 1022c. The absorptive color filter 1024c is configured to absorb light of the color of the image light 1032c and prevent that light from propagating to the internally coupled optical element 1022a. Although shown between the waveguides 1020c and 1020b, in some other embodiments, the absorptive color filter 1024c may be disposed between the waveguides 1020b and 1020a. It should be understood that further details regarding the composition, formation, and properties of the absorptive color filters 1024c and 1026 are provided in the discussion of FIG. 16.

[0252] Also, in the embodiments shown in FIGS. 16 and 19B, one or more of the color filters 1026, 1028, 1024c, and 1024b may be omitted if one or more of the internally coupled optical elements 1022a, 1022b, 1022c are sufficiently highly selective with respect to the color of light such that they are each intended to be internally coupled into the associated waveguides 1020a, 1020b, 1022c.

[0253] FIG. 20A illustrates an example of a top - down view of the eyepiece of FIGS. 19A and 19B. As shown, the internally coupled optical elements 1022a, 1022c spatially overlap, while the internally coupled optical element 1022b is laterally offset. Additionally, the waveguides 1020a, 1020b, 1020c may be vertically aligned along with the associated light dispersing elements 730, 740, 750 and the associated external coupling optical elements 800, 810, 820 of each waveguide. The internally coupled optical elements 1022a, 1022b, 1022c are configured to internally couple the incident image lights 1032a, 1032b, 1032c (FIGS. 15 and 16) into the waveguides 1020a, 1020b, 1020c, respectively, such that the image light propagates towards the associated light dispersing elements 730, 740, 750 by TIR.

[0254] FIG. 20B illustrates another embodiment of a top and bottom view of the waveguide assembly of FIGS. 19A and 19B. As in FIG. 20A, the internal coupling optical elements 1022a, 1022c are spatially overlapping, the internal coupling optical elements are laterally offset, and the waveguides 1020a, 1020b, 1020c are vertically aligned. However, instead of the associated optical dispersion elements 730, 740, 750 and the associated external coupling optical elements 800, 810, 820 of each waveguide, there are combined OPE / EPEs 1281, 1282, 1283, respectively. The internal coupling optical elements 1022a, 1022b, 1022c are configured to internally couple the incident image light 1032a, 1032b, 1032c (FIGS. 15 and 16) into the waveguides 1020a, 1020b, 1020c, respectively, such that the image light propagates towards the associated combined OPE / EPEs 1281, 1282, 1283 by TIR.

[0255] Referring now to FIG. 21, it should be understood that unwanted rebounding of internally coupled light can occur within the waveguide. Rebounding occurs when the internally coupled light propagating along the waveguide strikes the internal coupling optical element a second time or at a subsequent time after the initial internal coupling incidence. Rebounding can result in a portion of the internally coupled light being undesirably externally coupled and / or absorbed by the material of the internal coupling optical element. External coupling and / or absorption can undesirably result in a reduction in the overall internal coupling efficiency and / or the uniformity of the internally coupled light.

[0256] FIG. 21 illustrates a side view of an example of a re-bounce within waveguide 1030a. As shown, image light 1032a is internally coupled into waveguide 1030a by internal coupling optical element 1022a. Internal coupling optical element 1022a generally redirects image light 1032a to propagate through the waveguide in direction 1033. A re-bounce can occur when the internally coupled image light internally reflects or bounces from the major surface of waveguide 1030a facing internal coupling optical element 1022a and impinges on internal coupling optical element 1022a or undergoes a second bounce (re-bounce). The distance between two neighboring bounces on the same surface of waveguide 1030a is indicated by interval 1034.

[0257] Although not limited by theory, it should be understood that internal coupling optical element 1022a can behave symmetrically. That is, it can redirect incident light so that the incident light propagates through the waveguide at the TIR angle. However, light incident on the diffractive optical element at the TIR angle (such as in response to a re-bounce) can also be externally coupled. Additionally, or alternatively, in embodiments where internal coupling optical element 1022a is coated with a reflective material, it should be understood that reflection of light from a layer of material such as metal can also involve partial absorption of the incident light because the reflection can involve absorption and emission of light from the material. As a result, external coupling and / or absorption of light can undesirably cause loss of the internally coupled light. Thus, the re-bounced light can experience significant loss compared to light that interacts with internal coupling optical element 1022a only once.

[0258] In some embodiments, the internal coupling element is configured to reduce internal-coupled image light loss due to rebounce. Generally, the rebounce of internally coupled light occurs in the propagation direction 1033 of the internally coupled light towards the end 1023 of the internal coupling optical element 1022a. For example, the light internally coupled at the end of the internal coupling optical element 1022a facing the end 1023 can rebounce if the spacing 1034 for that light is short enough. To avoid such rebounce, in some embodiments, the internal coupling optical element 1022a is truncated at the propagation direction end 1023 to reduce the width 1022w of the internal coupling optical element 1022a along which rebounce is likely to occur. In some embodiments, the truncation may be a complete truncation of all structures (e.g., metallization and diffraction gratings) of the internal coupling optical element 1022a. In some other embodiments, for example, when the internal coupling optical element 1022a comprises a metallized diffraction grating, a portion of the internal coupling optical element 1022a at the propagation direction end 1023 may not be metallized such that the propagation direction end 1023 of the internal coupling optical element 1022a hardly absorbs the rebounced light and / or externally couples the rebounced light with lower efficiency. In some embodiments, the diffraction region of the internal coupling optical element 1022a may have a width shorter than its length perpendicular to the propagation direction 1033 along the propagation direction 1033, and / or the first portion of the image light 1032a is incident on the internal coupling optical element 1022a and the second portion of the light beam is sized and shaped to impinge on the waveguide 1030a without being incident on the internal coupling optical element 1022a. The waveguide 1032a and the optical internal coupling element 1022a are shown separately for clarity, but it should be understood that the discussed strategies for reducing rebounce and rebounce apply to any of the internal coupling optical elements disclosed herein. Also, it should be understood that the spacing 1034 is related to the thickness of the waveguide 1030a (a larger thickness results in a larger spacing 1034). In some embodiments, the thickness of the individual waveguides may be selected to set the spacing 1034 such that no rebounce occurs.Further details regarding bounce reduction can be found in U.S. Provisional Application No. 62 / 702,707, filed Jul. 24, 2018, the entire disclosure of which is incorporated herein by reference.

[0259] FIGS. 22A-23C illustrate embodiments of top and bottom views of an eyepiece having internally coupled optical elements configured to reduce rebounce. The internally coupled optical elements 1022a, 1022b, 1022c are configured to internally couple light to propagate in a propagation direction towards associated light dispersing elements 730, 740, 750 (FIGS. 22A-22C) or combined OPE / EPEs 1281, 1282, 1283 (FIGS. 23A-23C). As shown, the internally coupled optical elements 1022a, 1022b, 1022c may have a shorter dimension along the propagation direction and a longer dimension along the transverse axis. For example, each of the internally coupled optical elements 1022a, 1022b, 1022c may be rectangular in shape with a shorter side along the axis of the propagation direction and a longer side along an orthogonal axis. It should be understood that the internally coupled optical elements 1022a, 1022b, 1022c may have other shapes (e.g., orthogonal, hexagonal, etc.). Additionally, different ones of the internally coupled optical elements 1022a, 1022b, 1022c may have different shapes in some embodiments. Also, preferably, as shown, non-overlapping internally coupled optical elements may be positioned such that they are not in the propagation direction of other internally coupled optical elements. For example, as shown in FIGS. 22A, 22B, 23A, and 23B, non-overlapping internally coupled optical elements may be arranged along a line along an axis that intersects (e.g., is orthogonal to) the axis of the propagation direction.

[0260] It should be understood that the waveguide assemblies of FIGS. 22A - 22C are similar except for the overlap of the internally coupled optical elements 1022a, 1022b, 1022c. For example, FIG. 22A illustrates the internally coupled optical elements 1022a, 1022b, 1022c without overlap. FIG. 22B illustrates the overlapping internally coupled optical elements 1022a, 1022c and the non - overlapping internally coupled optical element 1022b. FIG. 22C illustrates the overlap among all the internally coupled optical elements 1022a, 1022b, 1022c.

[0261] The waveguide assemblies of FIGS. 23A - 23C are also similar except for the overlap of the internally coupled optical elements 1022a, 1022b, 1022c. FIG. 23A illustrates the internally coupled optical elements 1022a, 1022b, 1022c without overlap. FIG. 23B illustrates the overlapping internally coupled optical elements 1022a, 1022c and the non - overlapping internally coupled optical element 1022b. FIG. 22C illustrates the overlap among all the internally coupled optical elements 1022a, 1022b, 1022c.

[0262] Referring now to FIG. 24A, it should be understood that the emissive microdisplay has a high étendue, which presents challenges regarding the efficient utilization of light. As discussed herein, the emissive microdisplay may include a plurality of individual light emitters. Each of these light emitters may have a large - angle emission profile, for example, a Lambertian or near - Lambertian emission profile. Unfortunately, not all of this light may be captured and directed to the eyepiece of the display system.

[0263] FIG. 24A illustrates an example of an angular emission profile of light emitted by individual light emitters 1044 of the emissive microdisplay 1032 and light captured by the projection optics 1070. The illustrated emissive microdisplay 1032 can correspond to any of the emissive microdisplays disclosed herein, including emissive microdisplays 1032a, 1032b, 1032c. As illustrated, the projection optics 1070 may be sized to capture light having an angular emission profile 1046. However, the angular emission profile 1046 within the light emitter 1044 is significantly larger, and not all of the light emitted by the light emitter 1044 is incident on the projection optics 1070, and necessarily does not enter at an angle at which the light would propagate through and within the projection optics 1070. As a result, a portion of the light emitted by the light emitter 1044 can be undesirably captured and ultimately relayed to the user's eye without forming an image, thus being “wasted”. This can result in an image that appears darker than would be expected if more of the light output by the light emitter 1040 ultimately reached the user's eye.

[0264] In some embodiments, one strategy for capturing more of the light emitted by the light emitter 1040 is to increase the size of the projection optical system 1070 and increase the size of the numerical aperture of the projection optical system 1070 that captures the light. Additionally, or alternatively, the projection optical system 1070 may also be formed of a high refractive index material (e.g., having a refractive index greater than 1.5), which may also facilitate focusing. In some embodiments, the projection optical system 1070 may utilize lenses sized to capture a desired high percentage of the light emitted by the light emitter 1044. In some embodiments, the projection optical system 1070 may have an extended exit pupil and be configured to emit a light beam having, for example, a cross-sectional profile similar to the shape of the internal coupling optical elements 1022a, 1022b, 1022c of FIGS. 22A-23C. For example, the projection optical system 1070 may be extended in dimensions corresponding to the extended dimensions of the internal coupling optical elements 1022a, 1022b, 1022c of FIGS. 22A-23C. Without being limited by theory, such extended internal coupling optical elements 1022a, 1022b, 1022c may improve the étendue mismatch between the light-emitting microdisplay and the eyepiece lens 1020 (FIGS. 22A-23C). In some embodiments, the waveguide thickness of the eyepiece lens 1020 (e.g., FIGS. 11A and 12-23C) may be selected to increase the percentage of light effectively captured, for example, by increasing the re-bounce interval and reducing re-bounce as discussed herein.

[0265] In some embodiments, one or more light collimators may be utilized to reduce or narrow the angular emission profile of the light from the light emitter 1044. As a result, more of the light emitted by the light emitter 1044 is captured by the projection optical system 1070 and relayed to the user's eye, and advantageously, the brightness of the image and the efficiency of the display system can be increased. In some embodiments, the light collimator enables the light collection efficiency of the projection optical system (the percentage of the light emitted by the light emitter 1044 that is captured by the projection optical system) to reach a value of 80% or more, 85% or more, or 90% or more, including from about 85% to 95% or from 85% to 90%. Additionally, the angular emission profile of the light from the light emitter 1044 can be reduced to 60° or less, 50° or less, or 40° or less (e.g., from 180°). In some embodiments, the reduced angular emission profile may be in the range of about 30° to 60°, 30° to 50°, or 30° to 40°. It should be understood that the light from the light emitter 1044 can create a conical shape, and the light emitter 1044 is at the apex of the cone. The angular emission profile refers to the angle created by the sides of the cone, and the associated light emitter 1044 is at the apex of that angle (as seen in a cross-section obtained along a plane that extends through the center of the cone and includes the cone apex).

[0266] FIG. 24B illustrates an example of narrowing of the angular emission profile using an array of optical collimators. As shown, the emissive microdisplay 1032 includes an array of light emitters 1044, which emit light with an angular emission profile 1046. An array 1300 of optical collimators 1302 is disposed in front of the light emitters 1044. In some embodiments, each light emitter 1044 is matched one-to-one with an associated optical collimator 1302 (one optical collimator 1302 per light emitter 1044). Each optical collimator 1302 redirects the incident light from the associated light emitter 1044 and provides a narrowed angular emission profile 1047. Thus, the relatively large angular emission profile 1046 is narrowed to a smaller angular emission profile 1047.

[0267] In some embodiments, the optical collimators 1302 and the array 1300 may be part of the light redirecting structures 1080a, 180c of FIGS. 12 and 13A. Thus, the optical collimators 1302 may propagate at an appropriate angle into the optical combiner 1050 and narrow the angular emission profile of the light emitters 1044 and redirect the light so as to define a plurality of optical paths and associated plurality of exit pupils. It should be understood that the light may be redirected in a particular direction by appropriately shaping the optical collimators 1302.

[0268] Preferably, the optical collimator 1302 is positioned in close proximity to the optical emitter 1044 and captures a large proportion of the light output by the optical emitter 1044. In some embodiments, a gap may exist between the optical collimator 1302 and the optical emitter 1044. In some other embodiments, the optical collimator 1302 may be in contact with the optical emitter 1044. It should be understood that the angular emission profile 1046 may create a wide cone of light. Preferably, all or most of the cone of light from the optical emitter 1044 is incident on a single associated optical collimator 1302. Thus, in some embodiments, each optical emitter 1044 is smaller (occupies a smaller area) than the light-receiving surface of the associated optical collimator 1302. In some embodiments, each optical emitter 1044 has a width that is smaller than the spacing between adjacent, distant optical emitters 1044.

[0269] Advantageously, the optical collimator 1302 can increase the efficiency of light utilization and can also reduce the occurrence of crosstalk between adjacent optical emitters 1044. It should be understood that crosstalk between the optical emitters 1044 can occur when light from a neighboring optical emitter is captured by an optical collimator 1302 that is not associated with that neighboring optical emitter. That captured light can propagate to the user's eye and thereby provide incorrect image information regarding a given pixel.

[0270] Referring to FIGS. 24A and 24B, the size of the beam of light captured by the projection optical system 1070 can affect the size of the beam of light exiting the projection optical system 1070. As shown in FIG. 24A, without using an optical collimator, the exiting beam can have a relatively large size 1050. As shown in FIG. 24B, with the optical collimator 1302, the exiting beam can have a smaller width 1052. Thus, in some embodiments, the optical collimator 1302 may be used to provide a desired beam size for internal coupling within the eyepiece lens. For example, the amount by which the optical collimator 1302 narrows the angular emission profile 1046 may be selected, at least in part, based on the size of the internal coupling optical element within the eyepiece lens to which the light output by the projection optical system 1070 is directed thereagainst.

[0271] It should be understood that the optical collimator 1302 may take various forms. For example, the optical collimator 1302 may be a microlens or a microlenslet in some embodiments. As discussed herein, each microlens preferably has a width that exceeds the width of the associated light emitter 1044. The microlens may be formed from a curved transparent material such as glass or polymer, including resins such as photoresist and epoxy. In some embodiments, the optical collimator 1302 may be a nanolens, e.g., a diffractive optical grating. In some embodiments, the optical collimator 1302 may be a metasurface and / or a liquid crystal grating. In some embodiments, the optical collimator 1302 may take the form of a reflective well.

[0272] It should be understood that the different light collimators 1302 may have different dimensions and / or shapes depending on the wavelength or color of the light emitted by the associated light emitter 1044. Thus, for a full-color emissive microdisplay, the array 1300 may include a plurality of light collimators 1302 with different dimensions and / or shapes depending on the color of the light emitted by the associated light emitter 1044. In an embodiment where the emissive microdisplay is a monochrome microdisplay, the array 1300 may be simplified, and each of the light collimators 1302 within the array may be configured to redirect light of the same color. By using such a monochrome microdisplay, the light collimators 1302 may be similar across the array 1300 in some embodiments.

[0273] Continuing to refer to FIG. 24B, as discussed herein, the light collimators 1302 may have a one-to-one association with the light emitters 1044. For example, each light emitter 1044 may have a discrete associated light collimator 1302. In some other embodiments, the light collimators 1302 may be elongated such that they extend across a plurality of light emitters 1044. For example, in some embodiments, the light collimators 1302 may extend towards the far side of the page and extend in front of a row of a plurality of light emitters 1044. In some other embodiments, a single light collimator 1302 may extend across a column of light emitters 1044. In yet other embodiments, the light collimators 1302 may comprise stacked columns and / or rows of lens structures (e.g., nanolens structures, microlens structures, etc.).

[0274] As described above, the optical collimator 1302 may take the form of a reflective well. FIG. 25A illustrates an example of a side view of an array of tapered reflective wells for directing light to a projection optical system. As shown, the optical collimator array 1300 may include a substrate 1301 in which a plurality of optical collimators 1302 in the form of reflective wells may be formed. Each well may include at least one light emitter 1044, which may emit light with a Lambertian angle emission profile 1046. The reflective wall 1303 of the well of the optical collimator 1302 is tapered and reflects the emitted light so as to be output from the well with a narrower angle emission profile 1047. As shown, the reflective wall 1303 may be tapered such that the cross-sectional size increases with the distance from the light emitter 1044. In some embodiments, the reflective wall 1303 may be curved. For example, side 1303 may have the shape of a compound parabolic concentrator (CPC).

[0275] Referring now to FIG. 25B, an example of a side view of an asymmetric tapered reflective well is illustrated. As discussed herein, for example, as illustrated in FIGS. 12A-13A, it may be desirable to use the optical collimator 1302 to steer light in a particular direction that is not normal to the surface of the light emitter 1044. In some embodiments, the optical collimator 1302 may be asymmetric, as viewed in the side view as illustrated in FIG. 25B, such that the upper side 1303a forms a different angle (e.g., a larger angle) with the surface of the light emitter 1044 than the lower side 1303b. For example, the angles of the reflective walls 1303a, 1303b with respect to the light emitter 1044 may be different on different sides of the optical collimator 1302 in order to direct the light in a particular non-normal direction. Thus, as shown, the light exiting the optical collimator 1302 may generally propagate in a direction 1048 that is not normal to the surface of the light emitter 1044. In some other embodiments, the taper of the upper side 1303a may be different from the taper of the lower side in order to direct the light in the direction 1048. For example, the upper side 1303a may flare out over a wider range than the lower side 1303b.

[0276] Continuing to refer to FIG. 25, the substrate 1301 may be formed from a variety of materials that have sufficient mechanical integrity to maintain the desired shape of the reflective wall 1303. Examples of suitable materials include metals, plastics, and glass. In some embodiments, the substrate 1301 may be a plate of material. In some embodiments, the substrate 1301 is a continuous, integral piece of material. In some other embodiments, the substrate 1301 may be formed by bonding together two or more pieces of material.

[0277] The reflective wall 1303 may be formed within the substrate 1301 in a variety of ways. For example, the wall 1303 may be formed into the desired shape by machining the substrate 1301 or otherwise removing material to define the wall 1303. In some other embodiments, the wall 1303 may be formed as the substrate 1301 is formed. For example, the wall 1303 may be molded into the substrate 1301 as the substrate 1301 is molded into its desired shape. In some other embodiments, the wall 1303 may be defined by rearrangement of material after formation of the body 2200. For example, the wall 1303 may be defined by imprinting.

[0278] Once the contours of the walls 1303 are formed, they may undergo further processing to form a surface having a desired reflectivity. In some embodiments, the surface of the substrate 1301 itself may be reflective, e.g., the body is formed from a reflective metal. In such cases, the further processing may include smoothing or polishing the inner surface of the wall 1303 to increase its reflectivity. In some other embodiments, the inner surface of the reflector 2110 may be lined with a reflective coating, e.g., by a vapor deposition process. For example, the reflective layer may be formed by physical vapor deposition (PVD) or chemical vapor deposition (CVD).

[0279] It should be understood that the location of the light emitter relative to the associated light collimator can affect the direction of the light emitted out from the light collimator. This is illustrated in FIGS. 26A-26C, which illustrate examples of differences in the optical paths for light emitters at different positions relative to the center line of the upper associated light collimator. As shown in FIG. 26A, the light-emitting microdisplay 1030 has a plurality of light emitters 1044a, each having an associated light collimator 1302, which promotes the output of light having a narrowed angular emission profile 1047. The light passes through a projection optical system 1070 (represented as a simple lens for ease of illustration), which converges the light from the various light emitters 1044a onto an area 1402a.

[0280] Continuing to refer to FIG. 26A, in some embodiments, the light collimators 1302 may each be symmetric and may have a center line that extends along the axis of symmetry of the light collimator. In the illustrated configuration, the light emitters 1044a are disposed on the respective center lines of the light collimators 1302.

[0281] Referring now to FIG. 26B, the light emitter 1044b is offset by a distance 1400 from the center line of its individual light collimator 1302. This offset causes the light from the light emitter 1044b to follow a different path through the light collimator 1302, which outputs the light from the light emitter 1044b with a narrowed angular emission profile 1047b. The projection optical system 1070 then converges the light from the light emitter 1044b onto an area 1402b, which is offset relative to the area 1402a onto which the light from the light emitter 1044a converges.

[0282] Referring now to FIG. 26C, an optical emitter 1044c offset from both optical emitters 1044a and 1044b is illustrated. This offset causes the light from optical emitter 1044c to follow a path different from that of the light from optical emitters 1044a and 1044b through the optical collimator 1302. This results in the optical collimator 1302 outputting the light from optical emitter 1044c with a narrowed angular emission profile that follows a path different from that of the light from optical emitters 1044a and 1044b to the projection optical system 1070. Ultimately, the projection optical system 1070 converges the light from optical emitter 1044c onto area 1402c, which is offset with respect to areas 1402a and 1402b.

[0283] Referring to FIGS. 26A - 26C, the three-fold symmetry axes of each of the optical emitters 1044a, 1044b, 1044c may share a common optical collimator 1302. In some embodiments, the microdisplay 1030 may be a full-color microdisplay, and each optical emitter 1044a, 1044b, 1044c may be configured to emit light of a different primary color. Advantageously, the offset areas 1402a, 1402b, 1402c may, in some embodiments, correspond to internal coupling optical elements of a waveguide. For example, the areas 1402a, 1402b, 1402c may correspond to the internal coupling optical elements 1022a, 1022b, 1022c of FIGS. 11A and 12, respectively. Thus, the offset orientation of the optical collimator 1302 and the optical emitters 1044a, 1044b, 1044c may advantageously provide a simple three-pupil projection system 1010 using a full-color emitting microdisplay.

[0284] As described herein, the optical collimator 1302 may also take the form of a nanolens. FIG. 27 illustrates an example of a side view of an individual light emitter 1044 of the light-emitting microdisplay 1030 with an upper layer array 1300 of the optical collimator 1302 that is a nanolens. As discussed herein, each of the individual light emitters 1044 may have an associated optical collimator 1302. The optical collimator 1302 redirects the light from the light emitter 1044, narrows the large-angle emission profile 1046 of the light emitter 1044, and outputs light with a narrowed angular emission profile 1047.

[0285] Continuing to refer to FIG. 27, in some embodiments, the optical collimator 1302 may be a grating structure. In some embodiments, the optical collimator 1302 may be a grating formed by alternating elongated discrete extensions (e.g., lines) of materials having different refractive indices. For example, the extensions of material 1306 may extend in and out of the page, be formed within the material of the substrate 1308, and thereby be separated. In some embodiments, the elongated extensions of material 1306 may have a sub-wavelength width and pitch (e.g., a width and pitch smaller than the wavelength of light, configured to receive from the light emitter 1044 associated with the optical collimator 1302). In some embodiments, the pitch 1304 may be 30 - 300 nm, the depth of the grating may be 10 - 1,000 nm, the refractive index of the material forming the substrate 1308 may be 1.5 - 3.5, and the refractive index of the material forming the grating feature 1306 may be 1.5 - 2.5 (and different from the refractive index of the material forming the substrate 1308).

[0286] The illustrated grating structure may be formed in a variety of ways. For example, the substrate 1308 may be etched or nanoimprinted to define trenches, and the trenches may be filled with a material having a different refractive index than the substrate 1308 to form the grating feature 1306.

[0287] Advantageously, the nanolens array can provide various advantages. For example, the light collection efficiency of the nanolenslets can be high, e.g., 80 - 95%, including 85 - 90%, and is accompanied by excellent reduction of the angular emission profile, e.g., reduction to 30 - 40° (from 180°). In addition, low levels of crosstalk can be achieved because each of the nanolens collimators 1302 is selected to act on light of a specific color and possibly a specific angle of incidence, while preferably providing a high extinction ratio (with respect to the wavelengths of light of other colors), and can have physical dimensions and properties (e.g., pitch, depth, refractive index of the material forming features 1306 and substrate 1308). In addition, the nanolens array can have a flat profile (e.g., formed on a flat substrate), which can facilitate integration with a microdisplay, which can be a flat panel, and can also facilitate manufacturing and provide high reproducibility and accuracy when forming the nanolens array. For example, highly reproducible groove formation and deposition processes may be used to form each nanolens. Further, these processes enable an ease and reproducibility that exceeds what is typically achieved when forming curved lenses with similar variations with respect to variations between the nanolenses of the array.

[0288] Referring now to FIG. 28, a perspective view of an embodiment of a light-emitting microdisplay 1030 is illustrated. It should be understood that the optical collimator array 1300 advantageously enables the light emitted from the microdisplay to be routed as desired. As a result, in some embodiments, the light emitters of a full-color microdisplay may be arranged as desired, for example, for ease of manufacture or implementation within the display device. In some embodiments, the light emitters 1044 may be arranged within rows or columns 1306a, 1306b, 1306c. Each row or column may include light emitters 1044 configured to emit light of the same primary color. In a display that utilizes three primary colors, there may be three groups of rows or columns, which are repeated across the microdisplay 1030. It should be understood that if more primary colors are utilized, each repeating group may have that number of rows or columns. For example, if four primary colors are utilized, each group may have four rows or four columns, and one row or one column is formed by light emitters configured to emit light of a single primary color.

[0289] In some embodiments, some rows or columns may be repeated to increase the number of light emitters of a particular primary color. For example, the light emitters of some primary colors may occupy multiple rows or columns. This may facilitate color balance and / or may be utilized to address differential degradation or reduction in light emission intensity over time.

[0290] Referring to FIGS. 27 and 28, in some embodiments, each light emitter 1044 may have an associated optical collimator 1302. In some other embodiments, each line 1306a, 1306b, 1306c of the plurality of light emitters 1044 may have a single associated optical collimator 1302. The single associated optical collimator 1302 may extend substantially across the entirety of the associated line 1306a, 1306b, or 1306c. In some other embodiments, the associated optical collimator 1302 may be extended and extend across a plurality of light emitters 1044 that form a part of the associated line 1306a, 1306b, or 1306c, and a plurality of similar optical collimators 1302 may be provided along each of the associated lines 1306a, 1306b, 1306c.

[0291] Continuing to refer to FIG. 28, each light emitter 1044 may be extended along a particular axis (e.g., along the y-axis as shown). That is, each light emitter has a length along a particular axis, and the length is longer than the width of the light emitter. Additionally, a set of light emitters configured to emit light of the same primary color may be arranged within a line 1306a, 1306b, or 1306c (e.g., a row or a column) that extends along an axis (e.g., the x-axis) that intersects (e.g., is orthogonal to) the extension axis of the light emitters 1044. Thus, in some embodiments, the light emitters 1044 of the same primary color form a line 1306a, 1306b, or 1306c of light emitters that extends along a first axis (e.g., the x-axis), and the individual light emitters 1044 within the line are extended along a second axis (e.g., the y-axis).

[0292] In contrast, a full-color microdisplay typically includes sub-pixels for each primary color, where the sub-pixels are arranged in a particular relatively closely packed spatial orientation within a group, and these groups are understood to be reproduced across the array. Each group of sub-pixels may form a pixel in the image. In some cases, the sub-pixels are elongated along an axis, and rows or columns of sub-pixels of the same primary color extend along that same axis. Such an arrangement allows the sub-pixels of each group to be located close together, and it should be understood that this can have advantages with respect to image quality and pixel density. However, in the arrangement illustrated in FIG. 28, sub-pixels of different primary colors are relatively far apart due to the elongated shape of the light emitters 1044. That is, the light emitter of line 1306a is relatively far from the light emitter of line 1306c because the elongated shape of the light emitter of line 1306b spaces the light emitters 1306a and 1306c farther from the light emitters in the vicinity of a given line of light emitters. This can be expected to provide unacceptably poor image quality if the image formed on the surface of the microdisplay 1030 is relayed directly to the user's eye, but the use of the light collimator array 1300 advantageously allows light of different colors to be routed as desired to form a high-quality image. For example, light of each primary color may be used to form a separate monochromatic image, which is then routed to an eyepiece such as the eyepiece 1020 (e.g., FIGS. 11A and 12-14) and combined therein.

[0293] Referring to FIGS. 27 and 28, in some embodiments, each of the light emitters 1044 may have an associated optical collimator 1302. In some other embodiments, each line 1306a, 1306b, 1306c of the light emitters 1044 may have a single associated optical collimator 1302. The single associated optical collimator 1302 may extend across substantially the entire associated line 1306a, 1306b, or 1306c. In some other embodiments, the associated optical collimator 1302 may extend and extend across a plurality of light emitters 1044 that form a part of the associated line 1306a, 1306b, or 1306c, and a plurality of similar optical collimators 1302 may be provided along each of the associated lines 1306a, 1306b, 1306c.

[0294] It should be understood that the optical collimator 1302 may be used to direct light along different optical paths to form a multi-pupil projection system. For example, the optical collimator 1302 may direct light of different primary colors to two or three areas, respectively, for internal light combining.

[0295] FIG. 29 illustrates an example of a wearable display system with the full-color emissive microdisplay 1030 of FIG. 28 used to form a multi-pupil projection system 1010. In the illustrated embodiment, the full-color emissive microdisplay 1030 emits light of three primary colors to form a three-pupil projection system 1010. The projection system 1010 has three exit pupils, through which image light 1032a, 1032b, 1032c of different primary colors propagates to the light internal combining optical elements 1022a, 1022b, 1022c offset to three sides of the eyepiece 1020, respectively. The eyepiece 1020 then relays the image light 1032a, 1032b, 1032c to the user's eye 210.

[0296] The light-emitting microdisplay 1030 includes an array of light emitters 1044, which may be subdivided into monochromatic light emitters 1044a, 1044b, 1044c, which emit image light 1032a, 1032b, 1032c, respectively. It should be understood that the light emitters 1044 emit image light with a wide-angle emission profile 1046. The image light propagates through an array of light collimators 1300, which reduces the angular emission profile to a narrowed angular emission profile 1047.

[0297] In addition, the array of light collimators 1300 is configured to redirect the image light (image lights 1032a, 1032b, 1032c) at an angle such that the image light is output to the projection optical system 1070 so that the image light propagates to appropriate internal coupling optical elements 1022a, 1022b, 1022c. For example, the array of light collimators 1300 is preferably configured to direct the image light 1032a to propagate through the projection optical system 1070 and be incident on the internal coupling optical element 1022a, direct the image light 1032b to propagate through the projection optical system 1070 and be incident on the internal coupling optical element 1022b, and direct the image light 1032c to propagate through the projection optical system 1070 and be incident on the internal coupling optical element 1022c.

[0298] The different light emitters 1044 may emit light of different wavelengths and may need to be redirected in different directions to reach the appropriate internal coupling optical elements. Thus, in some embodiments, the light collimators associated with the different light emitters 1044 may have different physical parameters (e.g., different pitches, different widths, etc.). Advantageously, the use of flat nanolenses as light collimators facilitates the formation of the light collimators by varying the physical properties across the array of light collimators 1300. As described herein, the nanolenses may be formed using patterning and deposition processes, which facilitate the formation of structures with different pitches, widths, etc. across the substrate.

[0299] Referring again to FIG. 24A, it should be understood that the illustrated display system shows a single-emitter type microdisplay and omits the optical combiner 1050 (FIGS. 11A and 12 - 13B). In embodiments that utilize the optical combiner 1050, the reflective surfaces 1052, 1054 (FIGS. 11A, 12 - 13B, and 30B) within the optical combiner 1050 are preferably specular reflectors, and it would be expected that the light from the light emitter 1044 would retain its large-angle emission profile after being reflected from the reflective surfaces 1052, 1054. Thus, the problem regarding wasted light shown in FIG. 24A also exists when the optical combiner 1050 is utilized.

[0300] Referring now to FIG. 30A, an example of a wearable display system is illustrated that includes a light-emitting type microdisplay and an array of associated light collimators. FIG. 30A shows additional details regarding the interaction between the light emitter 1044, the light collimator 1302, and the internal coupling optical elements of the eyepiece 1020. The display system includes a microdisplay 1030b, which in some embodiments may be a full-color microdisplay. In some other embodiments, the microdisplay 1030b may be a monochrome microdisplay, and additional monochrome microdisplays (not shown) may be provided on different sides of an optional optical combiner 1050 (as shown in FIG. 30C).

[0301] Continuing to refer to FIG. 30A, each microdisplay 1030b includes an array of light emitters 1044 that emit light, each with a wide-angle emission profile (e.g., a Lambertian angle emission profile). Each light emitter 1044 has an associated dedicated light collimator 1302, which effectively narrows the angle emission profile to a narrowed angle emission profile 1047. The light beam 1032b with the narrowed angle emission profile passes through the projection optical system 1070, which projects or converges those light beams onto the internal coupling optical element 1022b. It should be understood that the light beam 1032b has a certain cross-sectional shape and size 1047a. In some embodiments, the internal coupling optical element 1022b has a size and shape that substantially matches or is larger than the cross-sectional shape and size of the light beam 1032b when the beam 1032b is incident on the internal coupling optical element 1022b. Thus, in some embodiments, the size and shape of the internal coupling optical element 1022b may be selected based on the cross-sectional size and shape of the light beam 1032b when incident on the internal coupling optical element 1022b. In some other embodiments, other factors (re-bounce reduction or angles or fields of view assisted by the internal coupling optical element 1022b) may be utilized to determine the size and shape of the internal coupling optical element 1022b, and the light collimator 1302 is preferably configured (e.g., sized and shaped) to provide a light beam 1032b with a properly sized and shaped cross-section that is fully or almost fully encompassed by the size and shape of the internal coupling optical element 1022b. In some embodiments, the physical parameters for the light collimator 1302 and the internal coupling optical element 1022b may be mutually modified to provide highly efficient light utilization in conjunction with other desired functionality (e.g., re-bounce reduction, assistance for a desired field of view, etc.). Advantageously, the above-described light collimation provided by the light collimator 1302 and the matching of the cross-sectional size and shape of the light beam 1032b with the size and shape of the internal coupling optical element 1022b enable the internal coupling optical element 1022b to capture a large percentage of the incident light beam 1032b.The internally coupled light then propagates through waveguide 1020b and is externally coupled to eye 210.

[0302] As shown, microdisplay 1030b may include an array 1042 of light emitters 1044, each surrounded by a non-emitting area 1045 having a total width 1045w. Additionally, light emitter 1044 has a width W and a pitch P. In an array where light emitters 1044 are regularly spaced, each light emitter 1044 and the surrounding area 1045 form a unit cell having a width 1045w that may in fact be equal to pitch P.

[0303] In some embodiments, optical collimator 1302 is a microlens disposed directly on and surrounding the associated light emitter 1044. In some embodiments, the width of microlens 1302 is equal to 1045w such that neighboring microlenses 1302 are in substantial contact or directly contact each other. It should be understood that light from light emitter 1044 fills the associated microlens 1302 and may in fact expand the area encompassed by light emitter 1044. Advantageously, such a configuration reduces the perceptibility of area 1045, which otherwise would be a dark space visible to the user and not emit light. However, since microlens 1302 effectively expands the associated light emitter 1044 such that it extends across the entire area of microlens 1302, area 1045 may be masked.

[0304] Continuing to refer to FIG. 30A, the relative sizes of the light emitter 1044 and the optical collimator 1302 may be selected such that the light from the light emitter 1044 fills the optical collimator 1302 with which it is associated. For example, the light emitters 1044 may be sufficiently spaced such that a microlens collimator 1302 having a desired curvature can be formed extending across the individual ones of the light emitters 1044. Additionally, as described above, the size and shape of the internal coupling optical element 1022b are preferably selected to match or exceed the cross-sectional shape and size of the light beam 1032b when it is incident on the internal coupling optical element 1022b. As a result, in some embodiments, the width 1025 of the internal coupling optical element 1022b may be greater than or equal to the width of the microlens 1302 (which may have a width equal to 1045w or P). Preferably, the width 1025 exceeds the width of the microlens 1302 or 1045w or P to account for some spreading of the light beam 1032b. As discussed herein, the width 1025 may also be selected to reduce rebounding and may be shorter than the length of the internal coupling optical element 1022b (orthogonal to the width). In some embodiments, the width 1025 may extend along the same axis as the propagation direction of the internally coupled light 1032b through the waveguide 1020b before being externally coupled for propagation to the eye 210.

[0305] Referring now to FIG. 30B, an example of a light projection system 1010 is illustrated, with a plurality of light-emitting microdisplays 1030a, 1030b, 1030c and associated arrays 1300a, 1300b, 1300c of optical collimators. The angular emission profiles of the light emitted by the microdisplays 1030a, 1030b, 1030c are narrowed by the optical collimator arrays 1300a, 1300b, 1300c, thereby facilitating the focusing of a large percentage of the light that is emitted by the projection optical system 1070 after propagating through the optical combiner 1050. The projection optical system 1070 then directs the light towards an eyepiece, such as the eyepiece 1020 (e.g., FIGS. 11A and 12 - 14) (not shown).

[0306] FIG. 30C illustrates an embodiment of a wearable display system with a plurality of light-emitting microdisplays 1030a, 1030b, 1030c, each with an associated array 1300a, 1300b, 1300c of optical collimators. The illustrated display system includes a plurality of microdisplays 1030a, 1030b, 1030c to emit light with image information. As illustrated, the microdisplays 1030a, 1030b, 1030c may be micro-LED panels. In some embodiments, the microdisplay may be a monochromatic micro-LED panel and is configured to emit different primary colors, respectively. For example, the microdisplay 1030a may be configured to emit light 1032a that is red, the microdisplay 1030b may be configured to emit light 1032b that is green, and the microdisplay 1030c may be configured to emit light 1032c that is blue.

[0307] Each of the microdisplays 1030a, 1030b, 1030c may have an associated array 1300a, 1300b, 1300c of optical collimators, respectively. The optical collimator narrows the angular emission profile of the light 1032a, 1032b, 1032c from the light emitters of the associated microdisplay. In some embodiments, an individual light emitter has a dedicated associated optical collimator (as shown in FIG. 30A).

[0308] Continuing to refer to FIG. 30C, the arrays 1300a, 1300b, 1300c of the optical collimators are between the associated microdisplays 1030a, 1030b, 1030c and the optical combiner 1050, which may be an X-cube. As shown, the optical combiner 1050 has internal reflective surfaces 1052, 1054 to reflect the incident light out from the output surface of the optical combiner. In addition to narrowing the angular emission profile of the incident light, the arrays 1300a, 1300c of the optical collimators are configured to redirect the light from the associated microdisplays 1030a, 1030c at an angle appropriate for the light to impinge on the internal reflective surfaces 1052, 1054 of the optical combiner 1050 as the light propagates towards the associated light internal coupling optical elements 1022a, 1022c, respectively. In some embodiments, to redirect the light in a particular direction, the arrays 1300a, 1300c of the optical collimators may comprise microlenses or reflective wells, which may be asymmetric and / or the light emitters may be offset from the center with respect to the microlenses or reflective wells as disclosed herein.

[0309] Continuing to refer to FIG. 30C, a projection optical system 1070 (e.g., a projection lens) is disposed on the output surface of the optical combiner 1050 and receives the image light emitted from the optical combiner. The projection optical system 1070 may include a lens configured to converge or focus the image light onto the eyepiece lens 1020. As shown, the eyepiece lens 1020 may include a plurality of waveguides, each configured to internally and externally couple light of a specific color. For example, the waveguide 1020a may be configured to receive red light 1032a from the microdisplay 1030a, the waveguide 1020b may be configured to receive green light 1032b from the microdisplay 1030b, and the waveguide 1020c may be configured to receive blue light 1032c from the microdisplay 1030c. Each of the waveguides 1020a, 1020b, 1020c has associated light internal coupling optical elements 1022a, 1022b, 1022c, respectively, for internally coupling light therein. Additionally, as discussed herein, the waveguides 1020a, 1020b, 1020c may each correspond to the waveguides 670, 680, 690 of FIG. 9B and may each have an associated orthogonal pupil expander (OPE) and an exit pupil expander (EPE), which ultimately externally couple the light 1032a, 1032b, 1032c to the user.

[0310] As discussed herein, a wearable display system incorporating a microdisplay preferably outputs light with different amounts of wavefront divergence and is configured to provide comfortable accommodation-convergence / divergence motion matching for the user. These different amounts of wavefront divergence may be achieved using external coupling optical elements with different refractive powers. As discussed herein, the external coupling optical elements may be present on or within the waveguides of an eyepiece lens, such as the eyepiece lens 1020 (e.g., FIGS. 11A and 12-14). In some embodiments, a lens may be utilized to increase the wavefront divergence provided by the external coupling optical element, or in configurations where the external coupling optical element is configured to output collimated light, it may be used to provide the desired wavefront divergence.

[0311] Figures 31A and 31B illustrate an embodiment of the eyepiece lens 1020 having a lens for varying the wavefront divergence of light to a viewer. FIG. 31A illustrates the eyepiece lens 1020 having a waveguide structure 1032. In some embodiments, as discussed herein, light of all primary colors may be internally coupled into a single waveguide such that the waveguide structure 1032 includes only a single waveguide. This advantageously provides a compact eyepiece lens. In some other embodiments, the waveguide structure 1032 may be understood to include a plurality of waveguides (e.g., waveguides 1032a, 1032b, 1032c of FIGS. 11A and 12 - 13A), each configured to relay light of a single primary color to the user's eye.

[0312] In some embodiments, variable focus lens elements 1530, 1540 may be disposed on both sides of the waveguide structure 1032. The variable focus lens elements 1530, 1540 may be in the path of image light from the waveguide structure 1032 to the eye 210 and also in the path of light from the surrounding environment through the waveguide structure 1032 to the eye 210. The variable focus optical element 1530 may modulate the wavefront divergence of the image light output to the eye 210 by the waveguide structure 1032. It should be understood that the variable focus optical element 1530 may have a refractive power that can distort the view of the world from the eye 210. As a result, in some embodiments, a second variable focus optical element 1540 may be provided on the world side of the waveguide structure 1032. The second variable focus optical element 1540 may provide a refractive power opposite to that of the variable focus optical element 1530 (or opposite to the net refractive power of the variable focus optical element 1530 and the waveguide structure 1032 if the waveguide structure 1032 has a refractive power) such that the net refractive power of the variable focus lens elements 1530, 1540 and the waveguide structure 1032 is substantially zero.

[0313] Preferably, the refractive powers of the variable focus lens elements 1530, 1540 may be dynamically modified, for example, by applying an electrical signal thereto. In some embodiments, the variable focus lens elements 1530, 1540 may comprise a transmissive optical element such as a liquid crystal lens, an electroactive lens, a conventional refractive lens with movable elements, a mechanically deformable-based lens, an electrowetting lens, an elastomer lens, or a plurality of fluids with different refractive indices. By modifying the shape, refractive index, or other properties of the variable focus lens element, the wavefront of the incident light may be changed. In some embodiments, the variable focus lens elements 1530, 1540 may comprise a layer of liquid crystal sandwiched between two substrates. The substrates may comprise an optically transmissive material such as glass, plastic, acrylic, or the like.

[0314] In some embodiments, in addition to, or alternatively to, providing a variable amount of wavefront divergence to place virtual content on different depth planes, the variable focus lens elements 1530, 1540 and the waveguide structure 1032 may advantageously provide a net refractive power equal to the user's prescription refractive power for a corrective lens. Accordingly, the eyepiece lens 1020 may serve as a substitute for a lens used to correct refractive errors, including myopia, hyperopia, presbyopia, and spherical aberration. Further details regarding the use of variable focus lens elements as substitutes for corrective lenses may be found in U.S. Patent Application No. 15 / 481,255, filed Apr. 6, 2017, the entire disclosure of which is incorporated herein by reference.

[0315] Referring now to FIG. 31B, in some embodiments, the eyepiece 1020 may include static lens elements and not be variable. Similar to FIG. 31B, the waveguide structure 1032 may include a single waveguide (e.g., capable of relaying light of different colors) or a plurality of waveguides (e.g., each capable of relaying light of a single primary color). Similarly, the waveguide structure 1034 may include a single waveguide (e.g., capable of relaying light of different colors) or a plurality of waveguides (e.g., each capable of relaying light of a single primary color). One or both of the waveguide structures 1032, 1034 may have a refractive power and may output light with a particular amount of wavefront divergence or may simply output collimated light.

[0316] Continuing to refer to FIG. 31B, in some embodiments, the eyepiece 1020 may include static lens elements 1532, 1534, 1542. Each of these lens elements is disposed within the path of light from the surrounding environment through the waveguide structures 1032, 1034 into the eye 210. Additionally, the lens element 1532 is between the waveguide structure 1003 2 and the eye 210. The lens element 1532 corrects the wavefront divergence of the light output to the eye 210 by the waveguide structure 1032.

[0317] The lens element 1534 corrects the wavefront divergence of the light output to the eye 210 by the waveguide structure 1034. It should be understood that the light from the waveguide structure 1034 also passes through the lens element 1532. Thus, the wavefront divergence of the light output by the waveguide structure 1034 is corrected by both the lens element 1534 and the lens element 1532 (and the waveguide structure 1032 if the waveguide structure 1003 2 has a refractive power). In some embodiments, the lens elements 1532, 1534 and the waveguide structure 1032 provide a particular net refractive power for the light output from the waveguide structure 1034.

[0318] The illustrated embodiments provide two different levels of wavefront divergence, one for the light output from waveguide structure 1032 and the second for the light output by waveguide structure 1034. As a result, virtual objects can be placed on two different depth planes corresponding to the different levels of wavefront divergence. In some embodiments, additional levels of wavefront divergence, and thus additional depth planes, may be provided by adding additional lens elements along with additional waveguide structures between the additional waveguide structures and the eye 210, adding the additional waveguide structures between lens element 1532 and the eye 210. Further levels of wavefront divergence may be similarly added by adding additional waveguide structures and lens elements.

[0319] Continuing to refer to FIG. 31B, it should be understood that lens elements 1532, 1534 and waveguide structures 1032, 1034 provide a net refractive power that can distort the user's view of the world. As a result, lens element 1542 may be used to counteract the refractive power and distortion of ambient light. In some embodiments, the refractive power of lens element 1542 is set to nullify the aggregate refractive power provided by lens elements 1532, 1534 and waveguide structures 1032, 1034. In some other embodiments, the net refractive power of lens element 1542, lens elements 1532, 1534, and waveguide structures 1032, 1034 is equal to the user's prescription refractive power for corrective lenses. (Exemplary Light Projection System with a Light Emitting Microdisplay Providing Improved Resolution)

[0320] As described above, a display system (e.g., a wearable display system that presents AR or VR content) can reduce size, mass, and / or power consumption relative to systems that utilize various other display technologies by using one or more emissive microdisplays. For example, the display system may optionally utilize a threshold number of emissive microdisplays (e.g., three displays each including an array of light emitters such as micro LEDs). In this embodiment, each emissive microdisplay may be configured to generate light of a particular primary color. The generated light may be combined, as discussed herein, to provide the appearance of a full-color image. Various embodiments in which multiple emissive microdisplays are utilized are discussed above and also below with reference to FIGS. 36A-36B. As another example, the display system may optionally utilize a single emissive microdisplay. In this embodiment, the emissive microdisplay may include light emitters (e.g., micro LEDs) for each primary color.

[0321] Using one or more emissive microdisplays, the display systems described herein may be configured to output AR or VR content (“virtual content”) with a resolution that exceeds the resolution directly corresponding to the number of light emitters included within the emissive microdisplay. For example, the display system may utilize one or more actuators to effect movement or adjustment of one or more portions of an optical projection system configured to output light forming virtual content to a user. For example, the actuator may adjust a geometric position associated with the light emitter. As an example, as illustrated in FIG. 36B, the actuator may effect a change in the position of the emissive microdisplay panel. In this example, the micro-LED panel may be offset along one or two axes. As another example, as illustrated in FIG. 36A, the actuator may effect a change in the position of the location of a projection optical system (e.g., one or more projection lenses). As described herein, the projection optical system may route light generated by one or more micro-LED panels to one or more internal coupling optical elements such as an internal coupling grating (ICG). The internal coupling optical element may be configured to route the light to a user of the display system.

[0322] The adjustments described above may be utilized to cause the geometric position of the light emitter to assume a position within the inter-emitter region of the array. As described above, the inter-emitter region (e.g., region 1045 illustrated in FIG. 32A) may include regions of the emissive microdisplay that each contain one light emitter therein. The inter-emitter region may thus be defined based on one or more pixel pitches. For example, the inter-emitter region may be outlined by a first side having a length equal to the pixel pitch along a first axis and a second side having a length equal to the pixel pitch along a second axis.

[0323] FIG. 32A illustrates an embodiment of a light emitting microdisplay 1030 having an array 1042 of light emitters (e.g., light emitter 1044) separated by an inter-emitter region 1045. The light emitter 1044 may have an emitter size p and a pixel pitch Λ. As illustrated, the light emitter 1044 may have an emitter size p and a pixel pitch Λ that are substantially the same in the x and y directions. However, it should be understood that in some embodiments, the emitter size p and the pixel pitch Λ may be different in the x and y directions. Additionally, different light emitters of the array 1042 may have different sizes, shapes (e.g., rounded), compositions, etc. In the illustrated array 1042, the second light emitter above the top row of light emitters is indicated by a reference 1044' to facilitate the subsequent discussion herein.

[0324] Continuing to refer to FIG. 32A, in some embodiments, the magnitude of the pixel pitch Λ may be greater than the emitter size p. As discussed herein, the light emitting microdisplay 1030 may have a relatively low fill factor due to various physical and electrical constraints. For example, each light emitter 1044 may be disposed within an associated area 1049, and only a small portion of that area is occupied by the light emitter 1044. Most of the area 1049 is occupied by the inter-emitter region 1045. The area 1049 may be defined as extending from the pixel pitch from the ends of the associated light emitters 1044 along the x-axis and the pixel pitch from the ends of the associated light emitters 1044 along the y-axis. The low fill factor may undesirably limit the pixel density and the ultimate resolution of the image formed using the light emitting microdisplay 1030.

[0325] In some embodiments, the positions of the light emitters of array 1042 as seen by the user at a first time point are offset at a second time point to their original locations within emitter - to - emitter region 1045, thereby being able to display pixels corresponding to those locations within the image. Thus, a high - resolution image frame can be decomposed into lower - resolution sub - frames. The first sub - frame has pixels at locations corresponding to the first positions of the light emitters, the second sub - frame has pixels at locations corresponding to the second positions of the light emitters, the third sub - frame has pixels at locations corresponding to the third positions of the light emitters, and so on. Thus, the positions of the light emitters as seen by the user can be adjusted so as to effectively tile (e.g., substantially tile) the sub - frames of the high - resolution image. It should be understood that the sub - frames and the high - resolution image frame occupy substantially the same area as perceived by the user (e.g., are of substantially the same physical size). For example, the sub - frames are preferably 90%, 95%, 99%, or 100% of the size of the high - resolution image frame, except that they have a lower pixel density than the high - resolution image frame.

[0326] FIG. 32B illustrates an example of a method by which the emissive microdisplay 1030 of FIG. 32A can be configured to emulate a higher - fill - factor microdisplay through time - multiplexing and repositioning of the array or associated optics. As described above, the emissive microdisplay 1030 of FIG. 32A can be configured to form individual partial - resolution sub - frames in rapid succession and with an offset for the perceived position of each individual light emitter. In such embodiments, the user's visual system can merge the sub - frames together such that the user perceives a full - resolution frame.

[0327] Continuing to refer to FIG. 32B, the illustrated pixels 1044a - 1044c represent the location of the first light emitter 1044 (FIG. 32A) as seen by the user at different times. Additionally, the illustrated pixels 1044a’ - 1044c’ each represent the location of the second light emitter 1044’ (FIG. 32A) at the same time as the illustrated pixels 1044a - 1044c. The first light emitter 1044 at the first position may emit light for the pixel 1044a of the first sub - frame. This pixel may represent the first pixel 1044a of the frame in which the virtual content is rendered. The perceived position of the first light emitter 1044 may then be offset by a distance less than the pixel pitch Λ (e.g., Δx and / or Δy) along the offset axis, and the first light emitter 1044 may emit light for the pixel 1044b in the second sub - frame. As illustrated, the position of the first light emitter 1044 is offset by Δx along the x - axis. Pixel 1044b may thus become the second pixel 1044b of the frame in which the virtual content is rendered. As will be described below, the geometric position of the first light emitter 1044 may be offset via an actuator connected to the array 1042 (FIG. 32A). Thus, the first light emitter 1044 may be physically re - positioned within the three - dimensional space. The geometric position may also be offset via an actuator connected to the projection optical system through which the light from the first light emitter 1044 is routed. Thus, the first light emitter 1044 may remain within the same physical location, and its light may be offset by offsetting the position of the projection optical system with respect to the first light emitter 1044.

[0328] Continuing to refer to FIG. 32B, following the offset of the geometric position of the first light emitter 1044 for pixel 1044b, the first light emitter 1044 can again be offset (by Δx, as shown), and the first light emitter 1044 can emit light for pixel 1044c within a third sub-frame of the frame in which the virtual content is rendered. This pixel 1044c can represent the third pixel of the frame being rendered. This process can be repeated for a total of N sub-frames that together form a frame of the virtual content rendered at full resolution. Similarly, the remaining light emitters of array 1042 can emit light over a plurality of offset sub-frames for a plurality of pixels within the full frame. In the embodiment of FIG. 32B, this process is repeated for nine sub-frames such that the first emitter 1044 provides all nine pixels that fit within area 1049. Thus, array 1042 (FIG. 32A) can output virtual content with a resolution that is three times in the x-direction and three times in the y-direction compared to the resolution of array 1042.

[0329] In some embodiments, the perceived position of the array 1042 of light emitters may be updated in a substantially continuous movement. For example, the position of emitter 1044 may be continuously offset along the x direction up to the output of pixel 1044c. The display system (e.g., one or more processors or processing elements) described herein may determine the extent to which the position of emitter 1044 is offset during this continuous movement. The display system may be configured to determine the time at which light corresponding to a new pixel should be output. For example, the display system may identify that the position of emitter 1044 has reached the distance corresponding to pixel 1044b. The display system may then cause emitter 1044 to output light based on the image value associated with pixel 1044b in the second subframe. Utilizing such continuous adjustment of the geometric position may reduce the jerkiness associated with offsetting the geometric position. In some other embodiments, the geometric position may be offset in discrete steps. For example, emitter 1044 may output light corresponding to pixel 1044a. The geometric position of emitter 1044 may then be offset in discrete steps, paused, and may output light corresponding to pixel 1044b. Other light emitters of array 1042 may be offset along with emitter 1044 as well. For example, light emitter 1044’ may be offset in discrete steps and may provide pixels 1044a’, 1044b’, and 1044c’ at different discrete steps.

[0330] In some embodiments, the number of sub - frames N may be determined or limited by the physical properties of the array 1042. Exemplary properties may include the maximum frame rate (e.g., N is preferably not so large that the sub - frames cannot merge together within the user's visual system. All N of the sub - frames are preferably displayed over a duration that is less than the user's flicker fusion threshold, e.g., less than 1 / 60 second). Additional exemplary properties may include the emitter pitch Λ, the emitter size p, etc. As described above, the number of sub - frames N may be determined based on the number of positions within which the emitters of the array 1042 can fit within the inter - emitter region 1045. In the example of FIG. 32B, the first emitter 1044 can be placed at 9 distinctly different positions within the inter - emitter region 1045. Thus, up to 9 sub - frames can exist. If the emitter size p is larger and / or the emitter pitch Λ is smaller, the number of sub - frames N can be reduced. Similarly, if the emitter size p is smaller and / or the emitter pitch Λ is larger, the number of sub - frames N can be increased.

[0331] In some embodiments, N may be determined based on calculating the floor value of the emitter pitch Λ divided by the emitter size p. The calculated floor value may represent the number of times the emitter can be adjusted or moved in each direction. For example, if Λ = 2.5 microns and p = 0.8 microns, N would be equal to 3. Thus, 9 sub - frames (e.g., 3×3) can exist. It should be understood that this determination may be adjusted depending on whether the emitter pitch Λ and / or the emitter size p vary along the x and y directions. For example, the emitter pitch X Λ X and the emitter pitch Y Λ Y may exist. In this example, N can thus vary based on the direction. The number of sub - frames can be determined as N X ×N Y as. (Exemplary Emissive Microdisplay for Forming a Foveated Image)

[0332] Another potentially desirable feature in VR, AR, and MR applications is foveated imaging (also simply referred to as foveation), in which the resolution of the displayed image varies across the image. In particular, VR, AR, and MR may include an eye-tracking system that determines where the user is looking. Generally, based on the limitations of the human visual system to detect less detail in portions of the visual field that are away from the user's fixation point, it may not be desirable to present full-resolution content (e.g., content at full rendering resolution) at the periphery of the user's vision. Peripheral full-resolution content can consume excessive processing power during rendering and may have excessive power consumption when displayed by a display system. In other words, significant advantages can be achieved from the perspective of reduced processing load and display power consumption by reducing the resolution of content away from the user's fixation point and delivering the highest-resolution image content only to the portion of the visual field that the user is looking at, e.g., only at the fixation point and directly adjacent thereto. It should be understood that the fixation point corresponds to the portion of the visual field that is focused on the user's fovea, and thus the eye has relatively high sensitivity to detail in this portion of the visual field.

[0333] Foveation may also be based on factors other than the fixation point. As an example, the content creator may specify that certain content, such as text, be displayed at full resolution even when the user's eyes are diverted from the content. As another example, the content creator may specify that foveation should be active only under certain conditions. As yet another example, foveation may be a user-selectable setting or may be automatically enabled as a result of low battery conditions. In at least some embodiments, foveation delivers the highest resolution only to the portion of the image within the portion of the visual field on which the user is fixating (e.g., represented by the foveal region A in FIG. 33), while delivering a lower resolution image to the peripheral portion (e.g., represented by the peripheral region B in FIG. 33), thereby conserving display resources (data, pixels, bandwidth, computations).

[0334] FIGS. 32C, 32D, and 33 illustrate examples of the configuration of the array 1042 of FIG. 32A for providing foveated images 1130, 1140, and 1200, respectively.

[0335] FIG. 32C illustrates an example of a foveated image 1130 formed by a light-emitting microdisplay such as the light-emitting microdisplay 1030 of FIG. 32A. The array 1042 of FIG. 32A can be configured to provide two levels of resolution for the image 1130 on which virtual content is rendered. In particular, the light emitter array 1042 can provide full resolution (or relatively high resolution) within the foveal region 1132 (the portion of the image expected to be focused on the user's fovea), and partial resolution (relatively low resolution) within a second region 1134. The location of the foveal region 1132 can be determined according to the location of the user's fixation point. For example, a gaze detection scheme may be employed. The gaze detection scheme can track the user's eyes. As an example, the pupil may be identified within each eye. A vector can extend from each identified pupil, and the intersection of the vectors in three-dimensional space can be determined. This intersection can represent the user's fixation point. The foveal region 1132 can correspond to the portion of the rendered image 1130 that is within a threshold angular distance of this fixation point. Additional details regarding foveation and detection of the user's fixation point can be found in U.S. Patent Application Publication No. 2018 / 0275410, the entire disclosure of which is incorporated herein by reference.

[0336] While providing high resolution for the foveal region 1132 and maintaining a lower resolution for the second region 1134, the light emitters 1044 of the array 1042 (FIG. 32A) may be updated differently. Referring again to the embodiment of FIG. 32B where the emitter is updated N (e.g., 9) times for a full resolution frame of virtual content, the portion of the light emitter 1044 corresponding to the pixels within the foveal region 1132 may be updated N times. In contrast, the remaining portion of the light emitter 1044 corresponding to the pixels within the second region 1134 may be updated less than N times (e.g., 1, 2, 3, etc.) per frame being rendered. As an example, the perceived position of the light emitter 1044 may be offset as described herein. With respect to the light emitter corresponding to the foveal region 1132, the light emitter 1044 may be updated for each offset within the perceived light emitter position. For example, these emitters may generate light corresponding to the updated pixel values included for each emitter position. The updated pixel values may represent pixel values included within individual sub - frames of a full resolution frame of virtual content. With respect to the emitters included within the second region 1134, the emitters may not be updated for each offset. For example, these emitters may generate light corresponding to the same pixel value for two or more geometric positions, or may simply not output light. As a result, these emitters may skip (e.g., not present) pixel values included within one or more of the sub - frames.

[0337] The second region 1134 is illustrated in FIG. 32C to have the resolution of the physical array of emitters included within the array 1042; however, this is merely one option. Optionally, the partial resolution within the second region 1134 may have a resolution lower than that of the physical array 1042. For example, some of the emitters within the region 1134 may be deactivated. Alternatively, the second region may have a resolution higher than that of the physical array 1042. For example, the perceived position of the light emitters corresponding to the region 1134 may be offset to display pixels for multiple sub-frames per image frame 1130, but the number of pixels displayed within the region 1134 per rendered frame is less than that within the foveal region 1132.

[0338] FIG. 32D illustrates an embodiment of a light-emitting microdisplay, such as the light-emitting microdisplay 1030 of FIG. 32A, configured to form a foveated image with three or more levels of resolution within an image. With respect to the illustrated displayed image 1140, the array 1042 of FIG. 32A may be configured to provide three levels of resolution within each display image 1140, where a first region 1142 has full resolution, a second region 1144 has intermediate resolution, and a third region 1146 has low resolution. In general, the array 1042 may be configured to implement foveation with any desired number of regions of different resolutions, and the resolution at any location within the display may be arbitrarily selected (e.g., by selecting the number of pixels of the total sub-frames utilized for each light emitter location of the array). For example, with respect to the pixels within the first region 1142, the corresponding light emitter may present pixel information for each sub-frame, with respect to the second region 1144, the corresponding light emitter may present pixel information for fewer sub-frames, and with respect to the third region 1146, the corresponding light emitter may present pixel information for even fewer sub-frames. In some embodiments, it may be desirable to have a smooth transition between the high-resolution and low-resolution regions. Such a transition can be accomplished by gradually reducing the number of sub-frames in which the individual light emitters present information, as discussed above.

[0339] Referring to FIGS. 32C and 32D, the foveal regions (e.g., regions 1132 and 1142) and the transition region 1144 are shown as rectangles (squares) for ease of illustration, but it should be understood that these regions can take on any shape. For example, these regions may have a circular, star-shaped, oval, or other shape. FIG. 33 illustrates another embodiment of a foveated image provided by an emissive microdisplay. As shown in FIG. 33, the highest resolution may be provided only to the foveal portion 1202, represented by region A1202, which may have a circular shape. Outside the foveal portion of the field of view (e.g., within region B1204), the resolution of the displayed image is reduced, thereby reducing the processing load associated with rendering and reducing the display power consumption.

[0340] In each of FIGS. 32C, 32D, and 33, the location of the high-resolution portion of the foveated image may be determined according to the user's eye posture or line-of-sight direction (e.g., as determined by an eye-tracking system including components such as the camera assembly 630 of FIG. 6). As an example, the high-resolution portions (e.g., the foveal region 1132, the first region 1142, and the region A1202) may be approximately at the center of the foveated image when the user is looking straight ahead, and the high-resolution portion may shift to the left when the user is looking left. Thus, the user may be presented with a relatively high-resolution image along the direction of the user's line of sight (e.g., to the fixation point), while the user is presented with a lower resolution for portions of the image within the peripheral vision. (Exemplary movement of the emissive microdisplay and / or the display optics)

[0341] As discussed herein, the position of the pixels to be displayed may be shifted by shifting the physical position of a part of an optical projection system such as, for example, the light emitter 1044 (FIG. 32A) and / or the projection optical system 1070 (FIGS. 11A, 12 - 14, 24A - 24B, 26A - 26C, and 29 - 30C). Also, as discussed herein, the physical position may be shifted using an actuator that is mechanically connected to the part to be shifted. It should be understood that the position of the light emitter may be shifted, for example, by shifting an array containing the light emitter.

[0342] In some embodiments, these shifts may be performed in discrete steps. For example, the light emitter and / or the projection optical system may be stationary or substantially stationary while they emit light and form the pixels of an individual subframe. The position of the light emitter and / or the projection optical system may then be shifted between the presentation of different subframes.

[0343] In some embodiments, the light emitter and / or the projection optical system may be continuously moved between subframes, with or without a reduction in speed, while the light emitter displays or projects an individual subframe. Such continuous movement may advantageously be simpler to implement than precisely starting and stopping the movement of the light emitter and / or the projection optical system in small steps. In either case, the result is that the relatively low - resolution and low - fill - factor array 1042 (FIG. 32A) emulates a relatively high - resolution and high - fill - factor array.

[0344] FIG. 34 illustrates various exemplary paths of movement of a portion of a light-emitting microdisplay to shift the position of the pixels being displayed. For example, as described herein, the movement may be performed using an actuator that is connected to one or more light-emitting microdisplays or to one or more projection optical systems. The actuator may move the light-emitting microdisplay and / or the projection optical system in a plane along the illustrated paths. In FIG. 34, each numbered position is a position at which light for pixels of different subframes is emitted, and thus each numbered position may be understood to be associated with different subpixels. Preferably, one loop of the various paths of movement is completed within the flicker fusion threshold and returned to the initial position.

[0345] In some embodiments, the movement may be performed through the use of two actuators. For example, a first actuator may adjust movement in a first direction (e.g., the x-direction), and a second actuator may adjust movement in a second direction (e.g., the y-direction). In some embodiments, the first actuator and the second actuator may operate orthogonally (e.g., 180 degrees out of phase with respect to each other). As an example, the first actuator may perform a cosine motion while the second actuator may perform a sine motion, and the two motions are combined to define a circle. As a result, in some embodiments, it should be understood that the various actuators herein (e.g., actuator 1504, 1504a-c, etc.) are each an aggregated structure that includes two component actuators that provide movement along a particular axis.

[0346] Continuing to refer to FIG. 34, in movement pattern 1300, the geometric position moves back and forth (e.g., oscillates) between two points, and thus can provide a perceived pixel resolution that is twice the basic resolution of the array. The basic resolution, as discussed herein, will be understood to be the resolution provided by the array without shifting the array. In movement pattern 1302, the path of movement can define a triangular shape that can increase the basic resolution of array 1042 by up to three times. In movement pattern 1304, the path of movement can define a rectangular pattern, and thus can increase the basic resolution of array 1042 by up to four times. In movement pattern 1306, the geometric position moves within a rectangular pattern, and thus can increase the basic resolution of array 1042 by up to six times. It should be understood that different sub-frames are not necessarily presented at each numbered position, and as a result, as described above, the increase in resolution can be a multiple "up to" a certain number.

[0347] In some embodiments, continuing to refer to FIG. 34, it should be understood that other additional sub-frames may be presented on each section of the various illustrated paths. For example, on the section from position 1 to position 2 of movement path 1300, one or more sub-frames may be presented at different points on that path between positions 1 and 2. In such an arrangement, the increase in resolution can be a multiple of at least the associated multiple described above for each movement pattern.

[0348] Referring now to FIGS. 35A and 35B, an embodiment of a method by which displacement of a light emitter and a projection optical system can change the position of a pixel to be displayed is illustrated. As shown in FIG. 35A, displacing an object point (e.g., an individual light emitter) along a line on plane 1400 by an amount δ, which may represent displacing an array along that line, changes the direction of the light rays transmitted through projection optical system 1070 from a first direction α1 1404 to a second direction α2 1406. This change in direction will then displace the pixels provided by the illustrated light emitter because the position of the light emitter is offset. The direction of the light rays transmitted through projection optical system 1070 can have a nearly one-to-one correspondence with the position of the object point. Thus, the emissive microdisplay can be displaced along one or more axes to displace the location of the pixels to be displayed.

[0349] As shown in FIG. 35B, displacing projection optical system 1070 by an amount δ also changes the direction of the transmitted light rays from a first position α1 1404 to a second position α2 1406. It should be understood that the displacement of projection optical system 1070 along one or more axes may be based on physical characteristics associated with projection optical system 1070. For example, the range by which projection optical system 1070 is adjusted upwardly may depend on the physical characteristics of projection optical system 1070 and the effect of projection optical system 1070 on the optical path of light. Exemplary characteristics can include focal length, lens type, refractive index, radius of curvature, and the like.

[0350] Accordingly, the techniques described herein for improving the resolution of a light-emitting microdisplay can be accomplished via displacement of a projection optical system or other optical components between the light-emitting microdisplay and the user. Further, as described above with respect to FIGS. 9A-9E, a full-color light-emitting microdisplay may instead employ three light-emitting microdisplays, each having a different color (e.g., a red array, a green array, and a blue array), the light from which is optically combined and then projected through a common projection optical system. In such an embodiment, instead of a displacement actuator (or set of actuators) for each light-emitting microdisplay, only a single displacement actuator (or set of actuators) may be required to displace the common optical system, as described herein, such that implementation of a controlled displacement of the common optical system may be simpler. (Exemplary Light-Emitting Microdisplay System)

[0351] As discussed herein, various portions of a light projection system can be moved to provide a desired shift in the position of the pixels being displayed, and this movement can be accomplished using actuators that are mechanically coupled to the portion to be moved.

[0352] FIG. 36A illustrates an example of a wearable display system having an optical projection system with an actuator coupled to the projection optical system. Both the optical projection system 1010 and the actuator 1504 may be referred to as the projection system 1500. It should be understood that the optical projection system 1010 of the projection system 1500 may take any of the various configurations disclosed herein (e.g., as illustrated and discussed with respect to FIGS. 11A, 12-14, 24A). A microlens, a microreflector, or a grating may be used as the light emitter of the projection system 1500 (e.g., as illustrated in FIGS. 24B, 26A-26C, and 29-30C), and the microlens, the microreflector, or the grating is preferably configured to provide an effective pixel size less than the pixel pitch in order to facilitate the position shift described herein for a sufficiently sparse array. In addition, as discussed herein, the actuator 1504, or the actuators 1504a-1504c, may each include two actuators that can be moved along different axes, or may be otherwise represented.

[0353] Continuing to refer to FIG. 36A, an example of the actuator is a piezoelectric actuator. The actuator 1504 may adjust the position of the projection optical system 1070 along one or more axes on a plane (e.g., a plane parallel to the plane on which the eyepiece lens 1020 is disposed) as described herein. For example, the actuator 1504 may move the projection optical system 1070 along two intersecting axes on its plane (e.g., using a two-dimensional piezoelectric motor). As illustrated, the projection optical system outputs light from the light-emitting microdisplays 1030a-1030c to the internal coupling optical elements 1022a-1022c of the waveguide of the eyepiece lens 1020.

[0354] The light projection system 1010 may utilize monochromatic light emitting microdisplays 1030a, 1030b, 1030c, each configured to output a different primary color. An optical combiner 1050, such as a dichroic x-cube, may redirect the light emitted from the light emitting microdisplays 1030a - 1030c to the projection optical system 1070 as described above.

[0355] In some embodiments, the projection optical system 1070 is configured to receive image light from the light emitting microdisplays 1030a - 1030c, and the actuator 1504 is configured to move the projection optical system 1070, which in turn deflects the image light output by the light projection system 1500. Accordingly, the pixels presented by the array can be perceived as being adjusted in location, for example, tiled across the subframe between emitters as described herein, and the light emitting microdisplays 1030a - 1030c can output light corresponding to a plurality of subframes. These subframes can be presented rapidly and continuously (within the flicker fusion threshold) such that the user can perceive them as being presented simultaneously within the full resolution frame of the virtual content.

[0356] In some embodiments, one or more of the emissive microdisplays 1030a - 1030c are independently movable relative to the other of the emissive microdisplay, the optical combiner 1050, and the projection optics 1070. In such embodiments, each independently movable microdisplay may have an associated independently movable actuator 1504. FIG. 36B illustrates an example of a wearable display system having an optical projection system 1500 with a plurality of actuators 1504a - 1504c coupled to different emissive microdisplays 1030a - 1030c, respectively. The actuators 1504a - 1504c can thus deflect the associated primary color emissive microdisplays 1030a - 1030c. This embodiment may enable the same subframe to be output such that each primary color emissive microdisplay 1030a - 1030c is deflected to an individual position and the subframes overlap within the user's eye. In some embodiments, the primary color emissive microdisplays 1030a - 1030c may be deflected along different paths.

[0357] Referring again to FIGS. 36A - 36B, the actuator 1504 or 1504a - 1504c may be controlled via one or more processing elements included within the display system. Additionally, the output of image light by the emissive microdisplays 1030a - 1030c may be synchronized with the movement of the actuator 1504 or 1504a - 1504c. For example, the emissive microdisplays 1030a - 1030c may output light based on a signal or command indicating to the actuator 1504 or 1504a - 1504c that a portion of the light projection system 1500 being moved by the actuator is offset to one or more positions. This signal or command may be utilized for a continuous movement pattern or a discrete movement pattern as described above. The signal or command may be generated by a display system such as one or more processors or processing elements. In some embodiments, the light projection system 1500 is part of the display system 60 (FIG. 9E), and the control elements for the actuator 1504 and the emissive microdisplays 1030a - 1030c may be part of the processing module 140 or 150 (FIG. 9E).

[0358] In some embodiments, actuator 1504 or 1504a - 1504c may continuously move a mechanically coupled portion of the light projection system 1500, for example, according to the movement pattern illustrated in FIG. 34, and the light - emitting microdisplays 1030a - 1030c may periodically generate light. With respect to the continuous movement pattern, the light - emitting microdisplays 1030a - 1030c may be synchronized with a signal (e.g., a clock signal) also utilized by actuator 1504 and / or 1504a - 1504c, thereby presenting sub - frames at their desired locations. For example, actuator 1504 may deflect the projection optics 1070 according to a known movement pattern (e.g., a known speed based on a clock signal). Thus, the light - emitting microdisplays 1030a - 1030c may utilize a signal to identify the extent along the movement pattern over which the projection optics 1070 is being moved. The light - emitting microdisplays 1030a - 1030c may then output light corresponding to a new sub - frame, for example, when the projection optics 1070 is at a position associated with the new sub - frame.

[0359] In some embodiments, as discussed herein, time - division multiplexing may be utilized for microdisplays 1030a, 1030b, 1030c. For example, different ones of the light - emitting microdisplays 1030a, 1030b, 1030c may be activated at different times to generate different primary - color images.

[0360] In some embodiments, actuators 1504a - 1504c may be moved to complete at least one movement loop (e.g., the loops of movement paths 1300 - 1306 in FIG. 34), and only a single one of the light - emitting microdisplays 1030a - 1030c generates a sub - frame of a single primary color during that loop. In some embodiments, after completing one loop, a sub - frame of a second primary color is generated by a second one of the microdisplays during a second loop of actuator movement, and after completing that second loop, a sub - frame of a third primary color is generated by a third one of the microdisplays during a third loop of actuator movement. Thus, each loop of actuator movement generates a set of tiled sub - frames for a total of three sets of loops and sub - frames when there are three primary colors (the number of sets is equal to the number of primary colors). Preferably, the completion of each set of sub - frames of a primary color is completed within the flicker fusion threshold.

[0361] The eyepiece 1020 is illustrated in FIGS. 36A - 36B as including a stack of waveguides, but it should be understood that the eyepiece 1020 may, in some embodiments, include a single waveguide as disclosed herein. FIG. 37A illustrates an example of a wearable display system having a light - projection system 1500 with an eyepiece 1020 having a single waveguide 1020a. The single - waveguide eyepiece 1020 shown may be similar to that illustrated and discussed with respect to FIGS. 13B, 14, 30A, and 31A - 31B.

[0362] In addition, as discussed herein, a single microdisplay may emit light of two or more (e.g., all) primary colors (e.g., emitting red, green, and blue light). For example, FIG. 14 illustrates an example of a wearable display system with a single full-color emissive microdisplay 1030b that can emit light of each primary color. In some embodiments, such a microdisplay and / or associated projection optics may be offset to present different pixels of an image, as discussed herein.

[0363] FIG. 37B illustrates an example of a wearable display system having a light projection system 1010 in which a single array of light emitters outputs light of one or more different primary colors. As shown, the light projection system 1010 may include a light-emitting microdisplay 1030b that includes an array 3742 of light emitters 3744. In some embodiments, the array 3742 may be configured to emit light of all primary colors utilized by the display system to form a full-color image. For example, the microdisplay 1030b may include an array of light emitters, each of which generates red, green, and blue light, respectively.

[0364] Preferably, for each primary color for a given pixel, it is emitted from an overlapping area of the array 3742, which can advantageously facilitate the offsets described herein for providing different pixels of an image. For example, the light emitters 3744 may each be understood to include a stack of constituent light generators, each of which is configured to emit light of a different associated primary color. The microdisplay 1030b may, in some embodiments, include coaxial red, green, and blue stacked constituent light generators.

[0365] Advantageously, continuing to refer to FIG. 37B, by emitting each primary color, the microdisplay 1030b can thus avoid the use of an optical combiner such as the optical combiner 1050 described herein. Light from the microdisplay 1030b (e.g., multi-component light) can be routed through the projection optical system 1070 to the eyepiece lens 1020. As described above with respect to at least FIG. 36A, the actuator 1504 may adjust the position of the projection optical system and form different pixels of the image. In some embodiments, a single waveguide eyepiece lens 1020 may be used to receive light (e.g., via an internal coupling optical element 1122a configured to internally couple the incident light of each primary color).

[0366] FIG. 37B illustrates the actuator 1504 that adjusts the position of the projection optical system 1070, but the actuator 1504 may, in addition or alternatively, be attached to the microdisplay 1030b to adjust the position of the microdisplay 1030b, offset the microdisplay 1030b, and provide different pixels as discussed herein. FIG. 37C is similar to that of FIG. 37B but illustrates a wearable display system where the actuator 1504 is attached to the microdisplay 1030b instead of the projection optical system 1070.

[0367] As described above, in some embodiments where different microdisplays are utilized to generate light of different primary colors, the projection system may use an optical combiner to combine the differently colored light generated separately. For example, an x-cube may be employed to combine the light from different microdisplays 1030a - 1030c (FIG. 36B). The combined light may be routed through the projection optical system 1070 and directed to an eyepiece lens 1020 that includes one or more waveguides.

[0368] In some embodiments, as illustrated in FIGS. 38A - 38D, even when different microdisplays are utilized to generate light of different primary colors, the optical combiner may be omitted from the projection system 1500. For example, the microdisplays 1030a - 1030c may each route light to the eyepiece 1020 through dedicated associated ones of the projection optics 1070a - 1070c. As illustrated, the microdisplay 1030a has an associated projection optic 1070a that focuses light onto an associated internal coupling optic 1022a, the microdisplay 1030b has an associated projection optic 1070b that focuses light onto an associated internal coupling optic 1022b, and the microdisplay 1030c has an associated projection optic 1070c that focuses light onto an associated internal coupling optic 1022c.

[0369] It should be understood that in embodiments where the optical combiner 1500 is not used, several exemplary advantages can be achieved. As an example, when the intervening optical combiner 1500 is omitted, the microdisplays 1030a - 1030c can be placed closer to the projection optics 1070a - 1070c, so improved light collection can exist. As a result, higher light utilization efficiency and image brightness can be achieved. As another example, the projection system 1500 can be simplified and tuned to a particular primary color of light. For example, the optical system design for each individual projection optic 1070a - 1070C can be calibrated separately for the light of each primary color generated by the microdisplays 1030a - 1030c. Thus, the projection system 1500 can avoid the need for achromatization of the projection optics.

[0370] As another exemplary advantage, as illustrated in FIG. 38A, the light from each of the projection optical systems 1070a - 1070c can advantageously be more specifically focused onto the respective associated internal coupling optical elements 1022a - 1022c. With respect to FIGS. 36A - 36B, the combined light is routed onto the eyepiece 1020 via the projection optical system 1070. As shown, the light may be internally coupled through different internal coupling optical elements 1022a - 1022c. In the embodiment of FIGS. 36A - 36B, the eyepiece 1020 includes three example waveguides that internally couple the individual primary colors generated by the microdisplays 1030a - 1030c. However, it should be understood that each primary color does not have to be precisely focused onto the respective internal coupling elements 1022a - 1022c of the eyepiece 1020. As a non - limiting example, FIGS. 36A - 36B illustrate combined light that is focused at a certain depth between the internal coupling elements 1022b and 1022c.

[0371] In contrast, the embodiment of FIGS. 38A - 38B allows for more precise focusing of each primary color onto the respective internal coupling elements 1022a - 1022c. The projection optical systems 1070a - 1070c for each primary color may be configured to precisely focus the light onto the respective internal coupling elements 1022a - 1022c. In some embodiments, this precise focusing may improve the image quality by providing a clearly focused image for each primary color.

[0372] FIG. 38A illustrates an embodiment of a light projection system 1500 without an optical combiner (e.g., the optical combiner 1050 described above). In the illustrated embodiment, three microdisplays 1030a - 1030c provide light (e.g., primary color light) to individual projection optical systems 1070a - 1070c. The projection optical systems 1070a - 1070c may be connected to or otherwise positioned along a connection element 3802. The connection element 3802 may be adjusted in position by an actuator 1504. Thus, the actuator 1504 may adjust the position of the projection optical systems 1070a - 1070c that form an integrated structure. Light from each microdisplay 1030a - 1030c may be routed through the projection optical systems 1070a - 1070c and focused onto individual internal coupling elements 1022a - 1022c included within the eyepiece 1020.

[0373] FIG. 38A illustrates an actuator 1504 that adjusts the position of the projection optical systems 1070a - 1070c via a connection element 3802. In some embodiments, each projection optical system 1070a - 1070c may include its own dedicated actuator. For example, the projection system 1500 may include three actuators to adjust the positions of the three individual projection optical systems 1070a - 1070c.

[0374] FIG. 38B illustrates another embodiment of a wearable display system having a light projection system without an optical combiner. In some embodiments, the microdisplays 1030a - 1030c may form a single integrated unit. For example, the microdisplays 1030a - 1030c may be mounted on a single backplane 3804. In some embodiments, the backplane 3804 may be a silicon backplane and may include electrical components for the microdisplays 1030a - 1030c. Similar to FIG. 38A, the illustrated actuator 1504 may adjust the position of the connection element 3802.

[0375] It should be understood that the actuators in FIGS. 38A - B may be attached to and configured to move the microdisplays 1030a - 1030c instead of the projection optical systems 1070a - 1070c. For example, FIG. 38C shows a wearable display system that would otherwise be similar to the wearable display system of FIG. 38A, but with the actuator 1504, which would otherwise be attached to the projection optical systems 1070a - 1070c, omitted, and each of the microdisplays 1030a - 1030c having associated actuators 1504a - 1504c instead. In some embodiments where the microdisplays 1030a - 1030c are all integrated (e.g., two or more of the microdisplays are physically connected, for example, by sharing a common backplane), a single actuator 1504 may be utilized to change the position of the physically connected microdisplays 1030a - 1030c. For example, FIG. 38D shows a wearable display system that would otherwise be similar to the wearable display system of FIG. 38B, but with the actuator 1504, which would otherwise be attached to the projection optical systems 1070a - 1070c, omitted, and the actuator 1504 attached instead to the physically coupled microdisplays 1030a - 1030c so as to move these microdisplays together.

[0376] In the above description with respect to at least FIGS. 36A - 38D, one or more actuators are described as causing displacement or movement of different components of the wearable display system. In each of these figures, one or more actuators are illustrated as moving the same type of component (e.g., a microdisplay or projection optics) for ease of illustration and discussion, but in some embodiments, the actuators may be provided to adjust the positions of two or more types of components (e.g., in the same display system, the actuators may be attached and configured to adjust the positions of both the microdisplay and the projection optics illustrated in these figures).

[0377] For example, FIG. 36A illustrates an actuator 1504 for adjusting the position of the projection optics 1070, and FIG. 36B illustrates actuators 1504a - 1504c for adjusting the positions of the microdisplays 1030a - 1030c, respectively. In some embodiments, the wearable display system may include both actuator 1504 and actuators 1504a - 1504c, where actuator 1504 is configured to adjust the position of the projection optics 1070 while actuators 1504a - 1504c are configured to adjust the positions of the microdisplays 1030a - 1030c. For example, the position of the projection optics 1070 may be adjusted simultaneously with the positions of the microdisplays 1030a - 1030c. As another example, the positions of the projection optics 1070 and the microdisplays 1030a - 1030c may be adjusted at different times (e.g., sequentially).

[0378] Similarly, referring to FIGS. 37B and 37C, in some embodiments, both the microdisplay 1030b and the projection optics 1070 may each have an associated actuator 1504 for moving the microdisplay 1030b and the projection optics 1070 (e.g., for moving these components simultaneously or at different times). In some embodiments, the same display system may include and use both an actuator 1504 (illustrated in FIG. 38A) and actuators 1504a - 1504c (illustrated in FIG. 38C) to adjust the positions of the projection optics 1070a - 1070c and the microdisplays 1030a - 1030c, respectively. Referring to FIGS. 38B and 38D, in some embodiments, the same display system may include the projection optics 1070a - 1070c with a first associated actuator 1504, and the combined microdisplays 1030a - 1030c may each have a second associated actuator 1504. (Exemplary flowchart)

[0379] FIG. 39 illustrates a flowchart of an exemplary process for outputting sub - frames of a frame in which virtual content is rendered. For convenience, the process will be described as being performed by a display system having one or more processors (e.g., within the local processing and data module 140 or the remote processing module 150 of FIG. 9E).

[0380] In block 3902, the display system obtains a frame in which virtual content is rendered. As described above, the display system may generate a frame of virtual content for presentation to the user. For example, the local processor and data module 140 may include one or more graphics processing elements. The module 140 may then generate a frame in which virtual content is rendered.

[0381] As described with respect to FIGS. 32 - 36, the frame to be rendered may be rendered at a resolution (e.g., pixel density) that is at least partially higher than that of the light emitters (e.g., micro LEDs) included within the microdisplay of the light projection system. Each sub - frame may be formed based on the light generated by the light emitters. Each set of sub - frames for forming a full - resolution frame may be output continuously at high speed (e.g., within the flicker fusion t...

Claims

1. A head mounted display system, comprising: a support structure configured to mount on a user's head; an optical projection system supported by the support structure, the optical projection system comprising: a microdisplay comprising an array of light emitters associated with a base resolution, the array of light emitters configured to output light forming a frame of virtual content; A projection optical system; One or more actuators; an optical projection system comprising: an eyepiece supported by the support structure, the eyepiece configured to receive light from the optical projection system and direct the received light to the user; one or more processors, the one or more processors comprising: receiving a rendered frame of virtual content, the rendered frame comprising a foveal region of the user's eye associated with a full resolution, the full resolution being higher than the base resolution, the foveal region being within a threshold angular distance of the user's fovea determined by a location of the user's fixation point identified by an eye gaze detection scheme, the rendered frame comprising a non-foveal region of the user's eye outside the foveal region, at least a portion of the non-foveal region associated with a partial resolution, the partial resolution being higher than the base resolution but lower than the full resolution; causing an emissive microdisplay projector to output light forming a first sub-frame of the rendered frame that includes within the foveal region, the first sub-frame and the rendered frame being substantially the same size; displacing, via the one or more actuators, one or more portions of the optical projection system to adjust a position associated with a light emitter light output from the optical projection system after causing the emissive microdisplay projector to output light forming the first sub-frame of the rendered frame; after shifting one or more portions of the optical projection system, causing the optical projection system to output light forming at least a second subframe and a third subframe of the rendered frame including within the foveal region, the second subframe and the third subframe each overlapping with the first subframe and being shifted relative to the first subframe, and output light from only a portion of light emitters of the emissive microdisplay projector is updated to improve resolution within the foveal region relative to the non-foveal region and improve resolution within the non-foveal region relative to the base resolution; one or more processors configured to A head mounted display system comprising:

2. The head-mounted display system of claim 1, wherein each emissive microdisplay array has an associated emitter size, the emitter size being less than a pixel pitch, and a total number of subframes of the rendered frame is determined based on a size associated with the pixel pitch and the emitter size.

3. A head-mounted display system as described in claim 2, wherein the one or more processors are configured to cause the light projection system to continuously output light that forms the total number of sub-frames.

4. A head-mounted display system as described in claim 3, wherein the one or more processors are configured to time multiplex the rendered frames by causing the one or more actuators to shift a portion of the optical projection system on a subframe-by-subframe basis.

5. A head-mounted display system as described in claim 4, wherein the one or more processors are configured to cause the one or more actuators to shift portions of the optical projection system such that geometric locations associated with the array of optical emitters are tiled within individual inter-emitter regions.

6. The head-mounted display system of claim 1, wherein the one or more processors are configured to cause the one or more actuators to shift a portion of the optical projection system according to a movement pattern, the movement pattern being a continuous movement pattern.

7. The head-mounted display system of claim 1, wherein the first sub-frame, the second sub-frame and the third sub-frame each comprise pixels associated with a distinct portion of the rendered frame.

8. A head-mounted display system as described in claim 1, wherein the optical projection system comprises a plurality of arrays of optical emitters.

9. A head-mounted display system as described in claim 1, wherein the one or more actuators are configured to displace the projection optical system.

10. A head-mounted display system as described in claim 1, wherein the one or more actuators are piezoelectric motors.

11. A head-mounted display system as described in claim 1, wherein the one or more actuators displace the emissive microdisplay projector along two axes.

12. A head-mounted display system as described in claim 1, wherein the array of light emitters is configured to emit light of multiple primary colors.

13. A head-mounted display system as described in claim 12, wherein each light emitter comprises a stack of constituent light generators, each constituent light generator emitting light of a different color.

14. The eyepiece comprises a waveguide assembly comprising one or more waveguides, each waveguide comprising: an incoupling optical element configured to incoupling light from the microdisplay into the waveguide; an outcoupling optical element configured to outcouple the incoupling light out of the waveguide; The head mounted display system of claim 1 .

15. A head mounted display system comprising: a support structure configured to mount on a user's head; an optical projection system supported by the support structure, the optical projection system comprising: a microdisplay comprising an array of light emitters associated with a base resolution, the array of light emitters configured to output light forming a frame of virtual content; A projection optical system; an X-cube prism, wherein each of the arrays of light emitters faces a different side of the X-cube prism; One or more actuators; an optical projection system comprising: an eyepiece supported by the support structure, the eyepiece configured to receive light from the optical projection system and direct the received light to the user; one or more processors, the one or more processors comprising: receiving a rendered frame of virtual content, the rendered frame comprising a foveal region of the user's eye associated with a full resolution, the full resolution being higher than the base resolution, the foveal region being within a threshold angular distance of the user's fovea determined by a location of the user's fixation point identified by an eye gaze detection scheme, the rendered frame comprising a non-foveal region of the user's eye outside the foveal region, at least a portion of the non-foveal region associated with a partial resolution, the partial resolution being higher than the base resolution but lower than the full resolution; causing an emissive microdisplay projector to output light forming a first sub-frame of the rendered frame that includes within the foveal region, the first sub-frame and the rendered frame being substantially the same size; displacing one or more portions of the optical projection system via the one or more actuators to adjust a position associated with an optical emitter light output from the optical projection system; causing the light projection system to output light forming at least a second subframe and a third subframe of the rendered frame including within the foveal region, the second subframe and the third subframe respectively overlapping with and shifted relative to the first subframe, and output light from only a portion of light emitters of the emissive microdisplay projector being updated to improve resolution within the foveal region relative to the non-foveal region and improve resolution within the non-foveal region relative to the base resolution; one or more processors configured to A head mounted display system comprising:

Images