Multi-resolution displays for head-mounted display systems

Abstract

To provide a favorable multi-resolution display for head-mounted display systems.SOLUTION: This disclosure describes a head-mounted display with a display assembly configured to display content to most or all of a user's field of view. The display assembly can be configured to display content in far-peripheral regions of the user's field of view differently than content upon which a user can focus. For example, the spatial resolution, color resolution, refresh rate and intensity (i.e., brightness) can be adjusted to save resources and / or to bring attention to virtual content positioned within a far-peripheral region. In some embodiments, these changes can save processing resources without detracting from the user's overall experience.SELECTED DRAWING: Figure 3E

Description

[Background technology]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims priority to U.S. Provisional Patent Application No. 62 / 423,162, filed November 16, 2016, entitled "Multi-Resolution Display Assembly for Head-Mounted Display Systems," U.S. Provisional Patent Application No. 62 / 475,012, filed March 22, 2017, entitled "High Resolution High Field of View Display," and U.S. Provisional Patent Application No. 62 / 539,934, filed August 1, 2017, entitled "High Resolution High Field of View Display."

[0002] BACKGROUND OF THE INVENTION Virtual and augmented reality systems generally include displays that project light into a user's eyes. Unfortunately, these systems are not designed to project content along the periphery of a user's field of view or beyond a small central portion of the user's field of view due to the output angle limitations of available display technology. This can reduce the level of immersion felt by users of these systems that might otherwise be possible when content is delivered from angles that extend to the periphery of the user's field of view. For this reason, a mechanism for stimulating the periphery of a user's field of view is desirable. Summary of the Invention [Means for solving the problem]

[0003] (Summary of the Invention) This disclosure describes wearable devices configured to present immersive virtual, augmented, and mixed reality content to a user. In some embodiments, a head-mounted display with a wraparound display assembly is provided and configured to display content in most or all of the user's field of view. The display assembly can be configured to display content differently in the far peripheral region of the user's field of view than content on which the user may focus. For example, spatial or angular resolution, color resolution, refresh rate, and intensity (i.e., brightness) can be adjusted to conserve resources and / or draw attention to virtual content located within the far peripheral region. In some embodiments, these changes can conserve processing resources without compromising the user's overall experience.

[0004] The present disclosure describes a head-mounted display assembly that includes: a first display, a second display at least partially surrounding the first display, and a mounting member configured to couple the first and second displays to a user's head, wherein the second display has a greater curvature than the first display.

[0005] A wearable display device is disclosed, comprising: a frame including mounting members configured to secure the display device to a user's head; and a display assembly coupled to the frame, the display assembly comprising a primary display and a peripheral display disposed along a periphery of the primary display.

[0006] A display for a head-mounted display device is disclosed, the display including: a first region having a first resolution, a second region at least partially surrounding the first region and having a second resolution substantially lower than the first resolution, and a transition region between the first and second regions having a variable resolution, the side of the transition region adjacent to the first region being lower than the side of the transition region adjacent to the second region.

[0007] Numerous advantages over conventional techniques are achieved by the methods of the present invention. For example, embodiments of the present invention provide a more immersive experience than head-mounted displays that do not target the far-peripheral region of a user's field of vision. Furthermore, because the human eye has difficulty discerning high-resolution spatial and color images in the peripheral region of a user's field of vision, lower-cost peripheral displays can be used to target the far-peripheral region. For this reason, the present invention enables a more immersive experience without substantially increasing the overall cost of the head-mounted display.

[0008] Additionally, portions of the wearable frame that would by default simply function as obstructions can now be surfaces for light display and modulation. These previously obstructive structures can be made aesthetically pleasing or interactive. These previously obstructive structures can also be made "invisible" to the viewer by matching the displayed content to the scene behind the structure / wearable. The present specification also provides, for example, the following items: (Item 1) 1. A wearable display device, comprising: a frame including a mounting member configured to secure the wearable display device to a user's head; a display assembly coupled to the frame; and Equipped with The display assembly includes: The primary display; a peripheral display disposed along the periphery of the primary display; and A wearable display device comprising: (Item 2) Item 1. A wearable display device as described in item 1, wherein the primary display outputs an angular representation of virtual content into the user's eyes and the peripheral display generates a spatial representation of the virtual content. (Item 3) Item 1. A wearable display device as described in item 1, wherein a first region of the peripheral display shows content at a higher spatial resolution than a second region of the peripheral display, the first region being closer to the user's field of view than the second region. (Item 4) Item 4. The wearable display device of item 3, wherein pixels are spaced closer together in the first region than in the second region. (Item 5) Item 1. The wearable display device of item 1, wherein the primary display is coupled to and overlays at least a portion of the peripheral display. (Item 6) Item 1. The wearable display device of item 1, wherein the primary display comprises a first primary display associated with the user's left eye and a second primary display associated with the user's right eye, both of the first and second primary displays being coupled to the peripheral display layer. (Item 7) Item 1. The display assembly of item 1, wherein the primary display and the peripheral display utilize different display technologies. (Item 8) 1. A display assembly for a head mounted display device, the display assembly comprising: a first region having a first resolution; a second region at least partially surrounding the first region and having a second resolution substantially lower than the first resolution; a transition region between the first region and the second region having a variable resolution; and Equipped with A display assembly, wherein the variable resolution is lower on a first side of the transition region adjacent to the first region than on a second side of the transition region adjacent to the second region. (Item 9) Item 9. The display assembly of item 8, wherein the first region is sized to cover the ocular field of view of a user's eye. (Item 10) Item 10. The display assembly of item 8, wherein the second region renders virtual content in grayscale and the first region renders virtual content in color. (Item 11) Item 10. The display assembly of item 8, wherein the spatial resolution of the first region exceeds the spatial resolution of the second region. (Item 12) Item 9. The display assembly of item 8, wherein the second region is a transparent organic light-emitting diode (OLED) display. (Item 13) Item 10. The display assembly of item 8, further comprising a picoprojector configured to illuminate the first area by projecting content onto an inner or outer surface of the display assembly. (Item 14) Item 9. The display assembly of item 8, wherein the second region is sized to fill a gap between the periphery of the first region and the user's face when the head-mounted display device is worn. (Item 15) Item 9. The display assembly of item 8, wherein the second region has a greater curvature than the first region. (Item 16) 1. A wearable display device, comprising: The frame and a projector coupled to the frame; Display Assembly and Equipped with The display assembly includes: a primary display coupled to the frame and configured to output an angular representation of the virtual content to the user's eye; a peripheral display at least partially surrounding the primary display, the peripheral display configured to output a spatial representation of the virtual content; and A wearable display device comprising: (Item 17) a mounting member configured to mount the wearable display device to a user and rotate relative to the frame; Item 17. The wearable display device of item 16, wherein the peripheral display is configured to bend and flex to accommodate rotation of the mounting member relative to the frame. (Item 18) Item 17. A wearable display device as described in Item 16, wherein a portion of the peripheral display is positioned between a portion of the mounting member and the user of the head-mounted display when the head-mounted display is worn by the user. (Item 19) Item 17. The wearable display device of item 16, further comprising a camera configured to monitor the user's surroundings, and the peripheral display configured to display the user's surroundings on one or more optically opaque components of the wearable display device. (Item 20) Item 17. The wearable display device of item 16, wherein the peripheral display is optically coupled to one or more optical components positioned along at least one side of the main display.

[0009] These and other embodiments of the present invention, along with many of its advantages and features, are described in more detail in conjunction with the following text and accompanying figures.

[0010] The present disclosure will be readily understood by the following detailed description in conjunction with the accompanying drawings, in which like reference numerals indicate like structural elements, and in which: [Brief explanation of the drawings]

[0011] [Figure 1A] 1A-1C show several different wearable display embodiments, according to some embodiments. [Figure 1B] 1A-1C show several different wearable display embodiments, according to some embodiments. [Figure 1C] 1A-1C show several different wearable display embodiments, according to some embodiments. [Figure 1D] 1D-1E show how the primary display can be positioned on either the exterior-facing or interior-facing surface of the peripheral display. [Figure 1E] 1D-1E show how the primary display can be positioned on either the exterior-facing or interior-facing surface of the peripheral display. [Figure 1F] FIG. 1F shows how a peripheral display can extend between the waveguides of one or more primary displays. [Figure 1G] FIG. 1G shows how a peripheral display can surround two primary displays. [Figure 1H] Figures 1H-1M show various primary and peripheral display configurations. [Figure 1I] Figures 1H-1M show various primary and peripheral display configurations. [Figure 1J] Figures 1H-1M show various primary and peripheral display configurations. [Figure 1K] Figures 1H-1M show various primary and peripheral display configurations. [Figure 1L] Figures 1H-1M show various primary and peripheral display configurations. [Figure 1M]Figures 1H-1M show various primary and peripheral display configurations. [Figure 2A] FIG. 2A shows an exemplary monocular field of view for the human eye. [Figure 2B] FIG. 2B illustrates an exemplary wearable display device configured to provide virtual content over an area suitable to cover a user's field of view, according to some embodiments. [Figure 2C] FIG. 2C shows the field of view and oculomotor field of view overlaid on one of the primary displays depicted in FIG. 2B. [Figure 2D] FIG. 2D illustrates an example embodiment of an augmented reality system configured to provide virtual content to a user. [Figure 2E] FIG. 2E diagrammatically illustrates light paths within an exemplary viewing optical assembly (VOA) that may be used to present a digital or virtual image to a viewer, according to an embodiment of the present invention. [Figure 3A] FIG. 3A illustrates how a peripheral display may conform to the contours of a user's face, according to some embodiments. [Figure 3B] FIG. 3B compares the radii of curvature of the peripheral and primary displays depicted in FIG. 3A, according to some embodiments. [Figure 3C] 3C-3D show top views of various primary and peripheral displays incorporated within a wearable device, according to some embodiments. [Figure 3D] 3C-3D show top views of various primary and peripheral displays incorporated within a wearable device, according to some embodiments. [Figure 3E] FIG. 3E shows the interior-facing surface of a wearable or head-mounted display device, according to some embodiments. [Figure 3F] FIG. 3F shows a flowchart illustrating a method in which a peripheral display represents virtual content progressing along a path, according to some embodiments. [Figure 4A]FIG. 4A shows a perspective view of an exemplary wearable display device without a peripheral display, according to some embodiments. [Figure 4B] FIG. 4B illustrates how a peripheral display may be combined with the wearable display device depicted in FIG. 4A, according to some embodiments. [Figure 5A] FIG. 5A shows a wearable display device including two multi-region displays joined by a bridge, according to some embodiments. [Figure 5B] FIG. 5B shows a wearable display device having a display with multiple display regions, according to some embodiments. [Figure 5C] FIG. 5C shows a multi-resolution display 570 similar to the displays depicted in FIGS. 5A and 5B, according to some embodiments. [Figure 6] Figures 6-7 show display components associated with specific display technologies. [Figure 7] Figures 6-7 show display components associated with specific display technologies. [Figure 8A] 8A-8C schematically illustrate display systems according to some other embodiments of the present invention. [Figure 8B] 8A-8C schematically illustrate display systems according to some other embodiments of the present invention. [Figure 8C] 8A-8C schematically illustrate display systems according to some other embodiments of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0012] Detailed Description of Specific Embodiments Representative applications of the methods and apparatus according to the present application are described in this section. These examples are provided solely to add context and aid in understanding the described embodiments. Thus, it will be apparent to one skilled in the art that the described embodiments may be practiced without some or all of these specific details. In other instances, well-known process steps have not been described in detail to avoid unnecessarily obscuring the described embodiments. Other applications are possible, and the following examples should not be construed as limiting.

[0013] Head-mounted or wearable display devices can be configured to provide an immersive user experience by projecting virtual content directly into the user's eyes. Unfortunately, the displays associated with these types of devices generally do not extend to cover the user's entire field of view. While a user's ability to focus on an object is limited to approximately 30-50 degrees off-axis, most users' eyes are capable of detecting content, especially fast-moving content, more than 100 degrees off-axis in some directions. For this reason, to create a truly immersive experience, the display must be configured to cover the periphery of the user's field of vision.

[0014] One solution to this problem is to incorporate a peripheral display for displaying content outside the user's ocular field of view in the peripheral region of the user's visual field. The ocular field consists of the portion of the user's visual field on which the user can directly focus. Because the peripheral display shows content outside the user's ocular field of view, the need for seamless blending or transitioning content from the peripheral display to the primary display is minimal. Furthermore, because the user's visual acuity is substantially reduced in the peripheral region, the peripheral display can be activated in a reduced-vision mode that saves power and / or processing power. For example, the peripheral display can display content with lower spatial or angular resolution, lower color resolution, different intensity, and / or a lower refresh rate. In some embodiments, portions of the display may not be capable of displaying high spatial, angular, and / or color resolution images, for example, due to reduced pixel density. In addition to allowing wearable devices to operate at lower power levels, these reduced-vision display modes allow hardware costs associated with peripheral displays to be substantially reduced by not requiring the peripheral displays to be capable of generating high-resolution images at high refresh rates. In some embodiments, the peripheral display may take the form of a transparent OLED (organic light-emitting diode) display. A transparent OLED may include an array of pixels distributed across a transparent and flexible substrate. In some embodiments, the substrate may be formed from a polymer composite. In other embodiments, the peripheral display may take the form of a picoprojector, projecting content onto the interior and / or exterior surfaces of the wearable display device.

[0015] Another solution involves using a customized display that covers the user's entire field of view. The customized display can be designed to display content at decreasing spatial and color resolution toward the periphery of the display. In some embodiments, the resolution can gradually decrease toward the periphery of the display. In some embodiments, the resolution change can be based on the current position of the user's eyes. For example, if an eye tracking sensor determines that the user's eyes are focused toward one side of the display, the opposite side of the display can be configured to display a correspondingly lower resolution.

[0016] These and other embodiments are discussed below with reference to Figures 1A-8C. However, those skilled in the art will readily appreciate that the detailed description given herein with respect to these figures is for illustrative purposes only and should not be construed as limiting.

[0017] 1A shows a wearable display device 100 including a high-resolution primary display 102 and a lower-resolution peripheral display 104 that surrounds the primary display 102. In some embodiments, the peripheral display 104 can be positioned to conform to the interior-facing surface of a temple arm 106. Note that the size of the primary display 102 can be adjusted to match the average ocular field of view for a user wearing the wearable display device 100. Selection of a display technology for the peripheral display 104 that includes a flexible substrate material that can bend and flex with the temple arm 106 can enable the peripheral display 104 to conform to at least a portion of the interior-facing surface of the temple arm 106. In some embodiments, the flexible substrate material can be sufficiently flexible to accommodate the temple arm 106 being folded against the display for storage.

[0018] FIG. 1B illustrates a wearable display device 110 having a single primary display 112 covering the ocular field of view for both eyes of a user operating the wearable display device 110. In some embodiments, the primary display 112 can utilize a different display technology than the peripheral display 114. For example, the primary display 112 can take the form of a brightfield display device, which can include one or more waveguides configured to project a brightfield onto the user's retina. The output of the brightfield display is an angular representation of the content and can be configured to project variable angular resolution. U.S. Application Nos. 14 / 707,000, 14 / 555,585, and / or 15 / 182,511 all provide detailed examples of brightfield display devices presented to a user as the primary display. The peripheral display 114 can take the form of a screen-based display device, which can include a "screen" (e.g., LCD, OLED, projector+projection screen, CRT, etc.) on which content is displayed. The output of this type of device is a spatial representation of the content as presented on a screen. The primary display 112 and peripheral display 114 can be coupled to the user's ears by temples 116.

[0019] In some embodiments, both the primary and peripheral displays can be transparent, allowing the outside world to be visible in areas where digital content is not actively displayed. FIG. 1C shows a wearable display device 120 having two separate primary displays 122 and two separate peripheral displays 124. The primary displays 122 can be configured to cooperatively cover the ocular field of view of a user's eyes, while the peripheral displays 124 can cooperatively cover any portion of the field of view not covered by the primary displays 122. Temples 126 represent attachment members suitable for engaging the user's ears, and a bridge 128 joins the two separate primary displays 122 together.

[0020] 1D-1E show cross-sectional views of various configurations of wearable display device 130. FIG. 1D illustrates how a front portion of wearable display device 130 can take the form of peripheral display device 132. In this manner, peripheral display device 132 can serve as a protective cover for primary displays 134 and 136. Primary displays 134 and 136 are depicted as including multiple distinct layers representing different waveguides for directing different wavelengths of light toward the user. In some embodiments, primary displays 134 and 136 can be adhered or otherwise attached to the surface of peripheral display 132. For example, such a surface of peripheral display 132 can be a continuous sheet or piece of material extending beyond the periphery of primary display 134 and primary display 136 to provide peripheral display functionality. Peripheral display 132 and primary displays 134, 136 can be transparent so that the user can perceive the outside world in addition to any virtual content generated by peripheral display 132 and primary displays 134, 136. In some embodiments, portions of peripheral display device 132 overlapping primary displays 134 and 136 can be configured not to display content, preventing the displays from displaying the same content. In some embodiments, peripheral display 132 can be configured to display content at startup while primary displays 134 and 136 undergo a warm-up cycle. Following initialization of primary displays 134 and 136, portions of peripheral display 132 overlapping primary displays 134 and 136 may be disabled. In some embodiments, peripheral display 132 may take over for primary displays 134 and 136 when interactive or high-resolution content is not actively being displayed.For example, when a user enters a configuration menu and the displayed content is limited to text or a simple menu structure, allowing one or more portions of peripheral display 132 that overlap primary displays 134 and 136 to activate instead of primary displays 134 and 136 may help conserve power and reduce heat generation in embodiments in which primary displays 134 and 136 consume more power and / or generate more heat than peripheral display 132. For example, peripheral display 132 may take the form of a flexible, transparent OLED display, allowing it to consume less power than primary displays 134 and 136 when the primary displays are driven by a relatively energy-consuming light projector.

[0021] In some implementations, some or all of the peripheral displays 132 can be operated to present content in conjunction with the primary displays 134 and 136 to further enhance the user experience. For example, portions of the peripheral displays 132 attached to or otherwise overlaying the primary displays 134 and 136 may present a flashing white light while the primary displays 134 and 136 present virtual content resembling fire / flames to simulate an explosion for a user engaged in a mixed reality gameplay experience. In another example, portions of the peripheral displays 132 attached to or otherwise overlaying the primary displays 134 and 136 may present text and / or serve to highlight real-world objects in the user's field of view. Furthermore, by utilizing portions of the peripheral display 132 that are attached to or otherwise overlapping the primary displays 134 and 136 and portions of the peripheral display 132 that are not attached to the primary displays 134 and 136 (e.g., the area of ​​the peripheral display 132 between and / or surrounding the perimeters of the primary displays 134 and 136), the boundary between the two types of display devices may appear smoother to the user. In some examples, some or all of the functionality of the portions of the peripheral display 132 that are attached to or otherwise overlapping the primary displays 134 and 136 as described herein with reference to FIG. 1D may extend into portions of the peripheral display that overlap (with respect to the user's field of view) one or more primary displays, as described in further detail below with reference to FIGS. 1I and 1K-1M.

[0022] 1E shows wearable display device 130 with an alternative configuration in which primary displays 134 and 136 may be positioned in front of peripheral displays 132. In such a configuration, primary displays 134 and 136 may include a protective cover layer 138 that protects primary displays 134 and 136 from damage. Wearable display device 140 may be operated in a similar manner to wearable display device 130, allowing peripheral displays 132 to take over operation from primary displays 134 and 136 in some circumstances.

[0023] Similarly, in some embodiments, the peripheral display can extend through a central portion of the primary displays 134 and 136, as shown in FIG. 1F. In some embodiments, the peripheral display 132 can act as a spacer to accentuate the distance between a first portion of the display and a second portion of the display. This distance can help light emitted from a portion of the primary display on the exterior-facing surface of the peripheral display appear to originate farther away than portions located along the interior-facing surface of the peripheral display.

[0024] 1G shows a front view of wearable display device 130. The front view of wearable display device 130 demonstrates how peripheral display 132 can be adjacent to and surround both primary displays 134 and 136. In some embodiments, the periphery of primary displays 134 and 136 can have reduced spatial, angular, and / or color resolution to blend with the lower resolution data being displayed on peripheral display 132. U.S. Provisional Patent Applications Nos. 62 / 475,012 and 62 / 539,934, both entitled "High Resolution High Field of View Display," from which this application claims priority, describe various ways in which the resolution of projection-based display systems can be configured with variable angular resolution.

[0025] 1H-1K show side views of various wearable display devices 140, 150, and 160. Wearable display device 140 includes a visor component 142, which provides a rigid substrate to which primary display 144 and peripheral display 146 can be coupled. The visor component can be optically neutral, but it can also be configured to create slight magnification or demagnification of objects within the visor's field of view. In some embodiments, the visor can include a polarizing layer and / or a tinted layer, which can be useful during outdoor use. Peripheral display 146 can extend from the edge of visor component 152 to the periphery of primary display 154. The displays can be attached to each other in many ways. For example, peripheral display 156 can be adhesively bonded to primary display 154. In some embodiments, an optically transparent frame can be positioned between visor component 152 and peripheral display 156 to help maintain the shape of peripheral display 156. FIG. 1I illustrates how peripheral display 152 may be adhered to an interior-facing surface of visor component 154. In this manner, visor component 154 can be configured to set the shape and position of peripheral display 162. FIG. 1J illustrates wearable display device 160 and how peripheral display 162 may be adhered to a peripheral portion of visor component 164. In some embodiments, peripheral display 162 can be attached to a recessed area defined by visor component 164. In this manner, peripheral display 162 need be sized to fill only the portion of the user's field of view that extends outside of primary display 166.

[0026] 1K shows a cross-sectional side view of wearable display device 170 and how primary display 172 may be surrounded by transparent curved optical element 174, which provides peripheral support for peripheral display 176. In some embodiments, the refractive index of curved optical element 174 can be adjusted to minimize distortion of light emitted by peripheral display 176. In some embodiments, transparent curved optical element 174 can take the form of a transparent frame used to support and position various other components associated with wearable display device 170. For example, in some embodiments, a waveguide configured to transmit light into primary display 172 can extend through an opening or channel defined by transparent curved optical element 174.

[0027] 1L-1M illustrate how a peripheral display may wrap around the edges of a primary display device and utilize various optics to direct light emitted from the peripheral display toward a reflector that redirects the light back into the eyes of a user of the wearable display device. FIG. 1L illustrates freeform optics 182 surrounding a primary display 184. The freeform optics 182 may include at least partially reflective surfaces 182 configured to redirect light 184 emitted by a peripheral display 186 back toward the user's eyes. In this manner, the freeform optics 182 can extend the effective size of the active display of a wearable display device 180 without requiring the peripheral display to extend to the very edge of the device. An alternative embodiment is depicted by a wearable display device 190, which may instead include a prism 192 having a triangular cross-section positioned along the periphery of a primary display 194. The prism 192 may redirect light 196 emitted by a peripheral display 186 that wraps around the edges of the primary display 184.

[0028] FIG. 2A shows a visual field diagram that depicts the perimeter of an exemplary monocular visual field 202 for a human eye in two-dimensional angular space. As shown in FIG. 2A, the temporo-nasal and inferior-superior axes of the visual field diagram serve to define the two-dimensional angular space into which the perimeter of the monocular visual field 202 is mapped. In this manner, the visual field diagram of FIG. 2A may be considered an equivalent or similar to a “Goldmann” visual field map or plot for a human eye. As indicated by the depicted arrangement of the temporo-nasal and inferior-superior axes, the visual field diagram shown in FIG. 2A represents a visual field diagram for a human left eye. While the visual field may vary slightly from person to person, the depicted visual field approximates what most people can see with their left eye. It follows that a visual field diagram depicting the perimeter of an exemplary monocular visual field for a right eye may resemble a version of the visual field diagram of FIG. 2A in which the temporo-nasal axis and the perimeter of the monocular visual field 202 are mirrored relative to the inferior-superior axis. The field of view diagram of FIG. 2A further depicts the periphery of an exemplary ocular field of view 204 for a human eye, which represents a portion of the monocular field of view 202 in angular space at which a person may fixate. In addition, the field of view diagram of FIG. 2A also depicts the periphery of an exemplary foveal field of view 206 for a human eye, which represents a portion of the monocular field of view 202 in angular space in the direct field of view of the fovea of ​​the human eye at a given time. As depicted, a person's foveal field of view 206 can move anywhere within the ocular field of view 204. The portion of the monocular field of view 202 outside the foveal field of view 206 in angular space may be referred to herein as the peripheral region of the person's field of view. Because the human eye's ability to distinguish high levels of detail outside the foveal field of view 206 is very limited, displaying reduced resolution images outside the foveal field of view 206 is unlikely to be noticeable and can allow for substantial savings in power consumption for processing components responsible for generating content for the display.

[0029] FIG. 2B shows an exemplary wearable display device 250 configured to provide virtual content over an area suitable for covering a user's field of view as depicted in FIG. 2A. Wearable display device 250 includes a primary display 252 supported by a frame 254. Frame 254 can be attached to the user's head using attachment members in the form of temple arms 106. In some embodiments, the image quality displayed by wearable display device 250 can be gradually reduced in either or both primary display 252 and peripheral display 256 such that areas near and within the ocular field of view have higher quality (e.g., higher spatial and / or color resolution) than areas near the edges of primary display 252. In some embodiments, the periphery of primary display 252 can be configured to match the quality or image characteristics of peripheral display 256. In some embodiments, the reduction in image quality can be accomplished by varying the spatial resolution, color bit depth, and / or refresh rate of primary display 252. For example, the color bit depth may be reduced from 12 bits to 5 or 6 bits, reducing both the required processing power and the peripheral display complexity. In some embodiments, the color bit depth can be reduced so that only grayscale or black and white content is displayed.

[0030] 2C shows field of view 202 and ocular field of view 204 overlaid on one of primary displays 252. FIG. 2C illustrates how primary display 252 cooperates with peripheral display 256, which covers ocular field of view 204 and covers most of field of view 202 for a user of wearable display device 250. While primary display 252 is shown to cover all of ocular field of view 204, the periphery of primary display 252 can be configured to optimize system resources by reducing the resolution of any portion of primary display 252 that does not actively cover ocular field of view 204. In some embodiments, sensors associated with wearable display device 250 can be configured to identify the position of the wearable display in conjunction with the eyes of a user of the wearable display to identify areas of primary display 252 that are not presenting content within ocular field of view 204. Because eye position may vary due to the shape of the head of a user of the wearable display device 250, an oversized primary display 252 may be useful in allowing the primary display 252 to cover the entire ocular field of view for a wide cross-section of the user. In some embodiments, an alignment mechanism may also help ensure proper ocular display positioning. For example, the alignment mechanism may take the form of an adjustable bridge and temples that may be used to accommodate different facial features by ensuring that the user's ocular field of view is covered by the primary display 252 and the user's peripheral vision is substantially covered by the peripheral display 256. To help achieve this alignment, the peripheral display 256 may have an asymmetric shape configured to match the shape of the user's peripheral vision 204, as depicted. In some embodiments, the user's ability to observe real-world content surrounding the wearable display device 204 may be obstructed by components that support the operation of the wearable display device. The peripheral display 256 may be configured to overlay content over those portions of the peripheral display that overlay the obstructing components.In some embodiments, real-world content can be displayed along the interior-facing surfaces of temple 106 using images acquired from world cameras positioned along the exterior-facing surfaces of temple 106.

[0031] Referring now to FIG. 2D , an exemplary embodiment of an AR system configured to provide virtual content to a user will now be described. In some embodiments, the AR system of FIG. 2D may represent the system to which the wearable display device 250 of FIG. 2B belongs. The AR system of FIG. 2D uses a stacked light directing optical element assembly 200 and generally includes an image generation processor 210, a light source 220, a controller 230, a spatial light modulator (“SLM”) 240, an injection optical system 260, and at least one set of stacked eyepiece layers or light directing optical elements (“LOEs,” e.g., planar waveguides) 200 that function as a multiple planar focusing system. The system may also include an eye tracking subsystem 270. It should be understood that other embodiments may have multiple sets of stacked LOEs 200, but the following disclosure will focus on the exemplary embodiment of FIG. 2D .

[0032] The image generation processor 210 is configured to generate virtual content for display to a user. The image generation processor may convert images or video associated with the virtual content into a format that can be projected to the user in 3-D. For example, in generating 3-D content, the virtual content may need to be formatted so that portions of certain images are displayed at specific depth planes, while others are displayed at other depth planes. In one embodiment, the images may all be generated at specific depth planes. In another embodiment, the image generation processor may be programmed to provide slightly different images to the right and left eyes so that, when viewed together, the virtual content appears coherent and comfortable to the user's eyes.

[0033] The image generation processor 210 may further include memory 212, a GPU 214, a CPU 216, and other circuitry for image generation and processing. The image generation processor 210 may be programmed with desired virtual content to be presented to a user of the AR system of FIG. 2D . It should be understood that in some embodiments, the image generation processor 210 may be stored within a wearable AR system. In other embodiments, the image generation processor 210 and other circuitry may be stored within a belt pack coupled to the wearable optics. The image generation processor 210 is operably coupled to a light source 220 that projects light associated with the desired virtual content and one or more spatial light modulators (described below).

[0034] The light source 220 is compact and has high resolution. The light source 220 includes multiple spatially separated sub-light sources 222 operably coupled to a controller 230 (described below). For example, the light source 220 may include color-specific LEDs and lasers arranged in various geometric configurations. Alternatively, the light source 220 may include similarly colored LEDs or lasers, each coupled to a specific region of the display's field of view. In another embodiment, the light source 220 may comprise a broad-area emitter, such as an incandescent or fluorescent lamp, with a mask overlay for segmenting the emission area and position. Although the sub-light sources 222 are directly connected to the AR system of FIG. 2D in FIG. 2D , the sub-light sources 222 may be connected to the system via optical fibers (not shown), as long as the distal ends of the optical fibers (away from the sub-light sources 222) are spatially separated from each other. The system may also include a concentrator (not shown) configured to collimate the light from the light source 220.

[0035] In various exemplary embodiments, the SLM 240 can be reflective (e.g., DLP DMD, MEMS mirror system, LCOS, or FLCOS), transmissive (e.g., LCD), or emissive (e.g., FSD or OLED). The type of spatial light modulator (e.g., speed, size, etc.) can be selected to improve the creation of the 3-D perception. While DLP DMDs operating at higher refresh rates can be easily incorporated into stationary AR systems, wearable AR systems typically use DLPs of smaller size and power. The power of the DLP changes the way the 3-D depth plane / focal plane is created. The image generation processor 210 is operably coupled to the SLM 240, which encodes the light from the light source 220 with the desired virtual content. The light from the light source 220 can be encoded with image information as it reflects off the SLM 240, as it emits from the SLM 240, or as it passes through the SLM 240.

[0036] Referring back to FIG. 2D , the AR system also includes an input optical system 260 configured to direct light from the light source 220 (i.e., the plurality of spatially separated sub-light sources 222) and the SLM 240 toward the LOE assembly 200. The input optical system 260 may include one or more lenses configured to direct the light into the LOE assembly 200. The input optical system 260 is configured to form adjacent, spatially separated, different pupils (at the respective foci of the beams exiting the input optical system 260) in the LOE 200 corresponding to the spatially separated, different beams from the sub-light sources 222 of the light source 220. The input optical system 260 is configured such that the pupils are spatially displaced from one another. In some embodiments, the input optical system 260 is configured to spatially displace the beams only in the X and Y directions. In such embodiments, the pupils are formed in a single X, Y plane. In other embodiments, the input optical system 260 is configured to spatially displace the beams in the X, Y, and Z directions.

[0037] The spatial separation of the light beams forms different beams and pupils, which allows for the placement of internal coupling gratings in different beam paths so that each internal coupling grating is primarily addressed (e.g., intersected or impinged) by only one different beam (or group of beams). This, in turn, facilitates the injection of spatially separated light beams into each LOE 200 of the LOE assembly 200 while minimizing the injection of other light beams from other sub-light sources 222 of the plurality (i.e., crosstalk). A light beam from a particular sub-light source 222 enters its respective LOE 200 through an internal coupling grating (not shown) on the LOE 200. The internal coupling grating of each LOE 200 is configured to interact with the spatially separated light beams from the multiple sub-light sources 222 such that each spatially separated light beam intersects only the internal coupling grating of one LOE 200. Thus, each spatially separated light beam is primarily incident on one LOE 200. Thus, image data encoded by SLM 240 onto the light beams from each of sub-light sources 222 can be efficiently propagated along a single LOE 200 for delivery to a user's eye.

[0038] Each LOE 200 is then configured to project an image or sub-image onto the user's retina that appears to originate from a desired depth plane or FOV angular location. Each of the multiple LOEs 200 and sub-light sources 222 can thus selectively project images (synchronously encoded by SLM 240 under control of controller 230) that appear to originate from different depth planes or positions in space. By sequentially projecting images using each of the multiple LOEs 200 and sub-light sources 222 at a sufficiently high frame rate (e.g., 360 Hz for six depth planes at an effective full-volume frame rate of 60 Hz), the system of FIG. 2D can generate 3-D images of virtual objects at different depth planes that appear to exist simultaneously within the 3-D image.

[0039] The controller 230 communicates with and is operatively coupled to the image generation processor 210, the light source 220 (sub-light source 222), and the SLM 240, and coordinates the synchronized display of images by instructing the SLM 240 to encode the light beam from the sub-light source 222 with appropriate image information from the image generation processor 210.

[0040] The AR system also includes an optional eye tracking subsystem 270 configured to track the user's eyes and determine the user's focus. In one embodiment, only a portion of the sub-light sources 222 may be activated to illuminate a portion of the LOE 200 based on input from the eye tracking subsystem, as discussed below. Based on input from the eye tracking subsystem 270, one or more sub-light sources 222 corresponding to a particular LOE 200 may be activated so that an image is generated at a desired depth plane consistent with the user's focus / accommodation. For example, if the user's eyes are parallel to each other, the AR system of FIG. 2D may activate sub-light sources 222 corresponding to LOEs 200 configured to deliver collimated light to the user's eyes so that images appear to originate from optical infinity. In another example, if the eye tracking subsystem 270 determines that the user's focus is one meter away, sub-light sources 222 corresponding to LOEs 200 configured to be approximately focused within that range may be activated instead. It should be appreciated that in this particular embodiment, only one group of sub-light sources 222 is activated at any given time, while the other sub-light sources 220 are deactivated to conserve power.

[0041] FIG. 2E schematically illustrates light paths within an exemplary viewing optical assembly (VOA) that may be used to present digital or virtual images to a viewer, according to an embodiment of the present invention. In some embodiments, the VOA may be incorporated into a system similar to wearable display device 250 as depicted in FIG. 2B. The VOA includes projector 201 and eyepiece 200 that may be worn around the viewer's eye. Eyepiece 200 may correspond, for example, to LOE 200 as described above with reference to FIG. 2D. In some embodiments, projector 201 may include a group of red LEDs, a group of green LEDs, and a group of blue LEDs. For example, projector 201 may include two red LEDs, two green LEDs, and two blue LEDs, according to an embodiment. In some examples, projector 201 and its components (e.g., LED light source, reflective collimator, LCoS SLM, and projector relay) as depicted in FIG. 2E may represent or provide the functionality of one or more of light source 220, sub-light source 222, SLM 240, and injection optical system 260 as described above with reference to FIG. 2D. Eyepiece 200 may include one or more eyepiece layers, each of which may represent one of LOEs 200 as described above with reference to FIG. 2D. Each eyepiece layer of eyepiece 200 may be configured to project an image or sub-image onto the retina of a viewer's eye that appears to originate from a respective desired depth plane or FOV angular position.

[0042] In one embodiment, the eyepiece 200 includes three eyepiece layers, one eyepiece for each of the three primary colors, i.e., red, green, and blue. For example, in this embodiment, each eyepiece layer of the eyepiece 200 may be configured to deliver collimated light to the eye that appears to originate from an optical infinity depth plane (0 diopters). In another embodiment, the eyepiece 200 may include six eyepiece layers, i.e., a set of eyepiece layers for each of the three primary colors configured to form virtual images at one depth plane and another set of eyepiece layers for each of the three primary colors configured to form virtual images at another depth plane. For example, in this embodiment, each ocular lens layer in a set of ocular lens layers of the eyepiece 200 may be configured to deliver collimated light to the eye that appears to originate from an optical infinity depth plane (0 diopters), while each ocular lens layer in another set of ocular lens layers of the eyepiece 200 may be configured to deliver collimated light to the eye that appears to originate from a distance of 2 meters (0.5 diopters). In other embodiments, the eyepiece 200 may include three or more ocular lens layers, one for each of three primary colors, for three or more different depth planes. For example, in such an embodiment, each of yet another set of ocular lens layers may be configured to deliver collimated light that appears to originate from a distance of 1 meter (1 diopter).

[0043] Each eyepiece layer comprises a planar waveguide and may include an internal coupling grating 207, an orthogonal pupil expander (OPE) region 208, and an exit pupil expander (EPE) region 209. Further details about internal coupling gratings, orthogonal pupil expansion, and exit pupil expansion are described in U.S. patent application Ser. Nos. 14 / 555,585 and 14 / 726,424. Still referring to FIG. 2E , projector 201 projects image light onto internal coupling grating 207 in eyepiece layer 200. Internal coupling grating 207 couples the image light from projector 201 into a waveguide, which propagates it in a direction toward OPE region 208. The waveguide propagates the image light horizontally by total internal reflection (TIR). The OPE region 208 of the eyepiece lens layer 200 includes diffractive elements that couple a portion of the image light propagating in the waveguide and redirect it toward the EPE region 209. More specifically, the collimated light propagates horizontally (i.e., relative to the view of FIG. 2E ) along the waveguide via TIR, repeatedly intersecting with the diffractive elements of the OPE region 208. In some examples, the diffractive elements of the OPE region 208 have relatively low diffraction efficiency. This means that at each intersection with a diffractive element in the OPE region 208, a percentage of the light (e.g., 10%) is diffracted vertically downward toward the EPE region 209, and via TIR, a percentage of the light continues on its original trajectory horizontally along the waveguide. Thus, at each intersection with a diffractive element in the OPE region 208, additional light is diffracted downward toward the EPE region 209. By splitting the incident light into multiple out-coupling sets, the exit pupil of the light is horizontally expanded by the diffractive elements of the OPE region 208. The expanded light coupled out of the OPE region 208 enters the EPE region 209 .

[0044] The EPE region 209 of the eyepiece lens layer 200 also includes diffractive elements that couple a portion of the image light propagating within the waveguide and redirect it toward the viewer's eye. Light entering the EPE region 209 propagates perpendicularly (i.e., relative to the view of FIG. 2E ) along the waveguide due to TIR. At each intersection between the propagating light and the diffractive elements of the EPE region 209, a percentage of the light is diffracted toward an adjacent surface of the waveguide, allowing the light to escape TIR, emerge from the surface of the waveguide, and propagate toward the viewer's eye. In this manner, the image projected by the projector 201 can be viewed by the viewer's eye. In some embodiments, the diffractive elements of the EPE region 209 can be designed or configured to have a phase profile that is the sum of a linear diffraction grating lens and a radially symmetric diffractive lens. The radially symmetric lens sides of the diffractive elements of EPE region 209 additionally impart a focal level to the diffracted light, both shaping the optical wavefronts of the individual beams (e.g., imparting curvature) and steering the beams to angles that match the designed focal level. Each beam of light outcoupled by the diffractive elements of EPE region 209 may geometrically extend to a respective focal point located in front of the viewer and may be imparted with a convex wavefront profile with a radius centered at the respective focal point to generate an image or virtual object at a given focal plane.

[0045] Descriptions of such viewing optical assemblies and other similar setups are further provided in U.S. patent application Ser. Nos. 14 / 331,218, 15 / 146,296, and 14 / 555,585. It follows that in some embodiments, an exemplary VOA may include and / or take the form of one or more components described in any of the patent applications mentioned above with reference to FIG. 2E.

[0046] FIG. 3A illustrates how the peripheral display 256 can conform to the contours of the user's 300's face. In some embodiments, the peripheral display 256 can have a curvature greater than that of the primary display 252 such that the peripheral display 256 can contact the user's 300's face without requiring a substantial curvature of the higher-resolution primary display 204. Contact between the peripheral display 256 and the user's face effectively allows the peripheral display 256 to project the content 302 along with any external light 304 reaching the user's 300's eyes 305 from above or below the primary display 204. In some embodiments, the peripheral display 256 can be configured to deform to conform to the user's 300's face. Additionally, the primary display 252 can also undergo deformation to accommodate certain contours of the user's 300's face. In some embodiments, the mechanical coupling between the peripheral display 256 and the primary display 252 can be configured to accommodate rotation of the peripheral display 256 relative to the primary display 252. For example, a flexible or elastomeric bond that accommodates rotation can couple primary display 252 to peripheral display 256. The interior-facing surface of peripheral display 256 can include padding or sealing elements to increase comfort for user 300 while wearing wearable display device 250. In other embodiments, peripheral display 256 can extend vertically more than depicted from primary display 204 so as not to contact user 300's face while the user is wearing wearable display device 250.

[0047] 3B shows how radius of curvature R1 for primary display 252 is substantially larger than radius of curvature R2 and radius of curvature R3. Because curvature is inversely proportional to radius of curvature, primary display 252 has a much smaller curvature than peripheral display 256. FIG. 3B also illustrates how radius of curvature R2 can differ from radius of curvature R3. The difference in curvature can be further varied as peripheral display 256 bends, flexes, and adapts to the shape of user 300's face.

[0048] FIG. 3C shows a top view of wearable device 250 worn on a user's head. As depicted, wearable device 250 can include a visor 306 having a primary viewing port corresponding to the surface on which primary display 252 is mounted. Visor 306 can also include walls extending from the viewing port toward the user's face at the top, bottom, and lateral sides of the viewing port. In some embodiments, the walls can protrude from the viewing port at a substantially perpendicular angle. Peripheral display 256 can then be adhered to the interior- or exterior-facing surfaces of the walls so that an image can be overlaid on light entering through any one of the walls. In some embodiments, peripheral display 256 can also cover the portion of the primary viewing port that is not covered by primary display 252. It should be noted that although the wearable device 250 is not depicted as extending to the user's head in some embodiments, the walls of the visor 306 can be configured to be in full contact with the user's face, allowing most, if not all, of the user's peripheral vision to be covered by the peripheral display 256.

[0049] FIG. 3D shows how the peripheral display 256 can be incorporated into the wearable device 250 in a more limited manner. The peripheral display 256 can be embodied by two flexible displays extending from a portion of each temple 106 to the interior-facing surface of the visor 306. The flexible nature of the peripheral display 256 can then accommodate the folding of the temples 106 into the visor 306. In this way, the lateral periphery of the user's peripheral vision can be covered without reducing the storage capability of the wearable device 250. In some embodiments, the peripheral display 256 can extend into a portion of the visor 306. For example, the portion of the visor 306 not covered by the main display 252 can be covered by an additional portion of the peripheral display 256. In some embodiments, the peripheral display 256 can be a single display extending from one temple 106 to the other temple 106.

[0050] FIG. 3E shows the interior-facing surface of wearable or head-mounted display device 250. Wearable display device 250 includes a frame or visor 254 that is pivotally coupled to temple arms 106 by hinges 308. As depicted, frame 254 supports primary display 252 and provides a surface to which peripheral display 256 may be attached. Peripheral display 256 is shaded for emphasis. In particular, FIG. 3E illustrates how peripheral display 256 may display virtual content 309 as the virtual content 309 repeatedly enters and exits locations in space, causing peripheral display 256 to render a representation of the virtual content 309.

[0051] FIG. 3F shows a flowchart illustrating a method in which the peripheral display 256 presents virtual content 309 progressing along a path. The dashed line indicates the path of the virtual content 309. Because the virtual content 309 follows a path through three-dimensional space, the wearable display device 250 will not always present the virtual content 309. Segment 310 represents a portion of the path that occurs outside the field of view of the head-mounted display. Segment 312 is a portion of the path corresponding to the virtual content 309 initially located at a location where the peripheral display 256 is responsible for displaying the virtual content 309. In some embodiments, the peripheral display 256 can be configured to present the virtual content 309 at a higher intensity level and / or refresh rate to help the user become more quickly aware of the virtual content 309. For example, because peripheral vision is typically more effective at tracking fast-moving objects, a higher refresh rate may help the user identify objects represented by the peripheral display 256. In some embodiments, the peripheral display 256 may, at least initially, depict the virtual content 309 as a brightly colored blob or flashing light in segment 312 to help direct the user's attention to the incident content. In some embodiments, the peripheral portion of the peripheral display 256 may be illuminated in a predetermined manner to alert the user that a particular event has occurred. For example, a fast flashing light may indicate that an incident augmented reality object is entering the user's field of view, while a slow pulsing blue orb may indicate the receipt of a text or in-game message.

[0052] In segment 314, as the virtual content 309 approaches closer to the primary display 252, a clear view of the outside world may be blocked by the frame 254 when the frame 254 is optically opaque. In some embodiments, a portion of the peripheral display 256 positioned in front of the frame 254 can be configured to display real-world content gathered by a camera mounted on the wearable device, presenting the user with a view that is virtually unobstructed by the frame 254. In this manner, the real-world content can be blended with the virtual content 309 to create a virtual representation of virtual and real-world content. In some embodiments, the real-world content reproduced by the peripheral display 256 can be based, at least in part, on the measured intensity and color of ambient light present in the surrounding environment. Such an implementation can create an unrestricted view and a greater sense of immersion without having to incorporate a video feed from an additional camera. Any lack of detail from constructing the view in this manner will largely go unnoticed, given that the user cannot directly focus on that portion of their field of view. Note that overlaying real-world images onto mask frame 254 is an optional operation, and in some embodiments it may be more desirable to either not show any content at all during segment 314 and highlight the presence of frame 254, or simply show the virtual content as one progresses across segment 314. Once the virtual content reaches segment 314, peripheral display 256 may begin to display the virtual content 309 in greater detail as the person's ability to perceive higher resolution increases.

[0053] At segment 316, the primary display 252 takes over displaying the virtual content 309. Because the peripheral display 258 and primary display 252 are contiguous, the virtual content 309 can remain in view persistently as it transitions from the peripheral display 256 to the primary display 252. At segment 318, the peripheral display 256 resumes displaying the virtual content 309, blending it with a background image that masks the frame 254 from the user's view. Note that, as with segment 314, displaying the background real-world image can be an optional step. At segment 320, the peripheral display 258 creates a representation of only the virtual content 309, and at segment 322, the peripheral display 256 ceases displaying the virtual content 309.

[0054] FIG. 4A shows a perspective view of an exemplary wearable display device 400 without a peripheral display. The wearable display device 400 includes primary displays 252. Each of the primary displays 252 can include an eye tracking sensor 402 configured to track eye movement of a user of the wearable display device 400. In some embodiments, the resolution of the image rendered by the primary displays 252 can be adjusted to account for the user's eye movement as determined by the eye tracking sensor 402. For example, the resolution can vary across the surface of the primary displays 252 so that processing power can be devoted to providing high resolution to only those areas focused on by the user's eyes. Other areas can be rendered at a lower resolution. The wearable display device 400 also includes a projector assembly 404, which is integrated into the temple arm 106. The projector assembly 404 can include a projector that shines light through diffractive optics, which is then reflected through the primary displays 252 into the user's eyes. The wearable display device 400 may also include camera assemblies 406. Each of the camera assemblies 406 may include several camera modules 408 for observing and characterizing the environment surrounding the wearable display device 400. Characterizing the environment may be important for a number of reasons, including, for example, for combining virtual content with real-world objects in the environment. For example, being able to identify items such as chairs using the camera modules may allow a virtual character to sit on one of the real-world chairs instead of having to generate a virtual chair or give the appearance of sitting in the air. In some embodiments, the wearable display device 400 may include one or more camera modules 408 with depth-sensing sensors to synchronize with the depth of the virtual content displayed by the primary display 204. Similar to the projector assembly 404, the camera assemblies 406 may be combined with the temple arm 106.

[0055] 4B shows how peripheral displays 256 may be incorporated into wearable display device 400. As depicted, peripheral displays 256 may be positioned along the periphery of each of primary displays 252. Peripheral displays 256 may also extend between primary displays 252 to prevent any lack of coverage above bridge 410. In some embodiments, temple regions 412 of peripheral displays 256 may extend farther from primary displays 252 than the rest of peripheral displays 256. Temple regions 412 may be configured to display content to make projector assembly 404 and camera assembly 406 less noticeable from the user's peripheral vision. This may help the user feel more immersed in the surrounding virtual and / or real-world content.

[0056] FIG. 5A shows a wearable display device 500 including two displays 502 joined by a bridge 504. In particular, FIG. 5A illustrates how the display 502 can have two distinct regions configured to display content differently. The high-vision region 506 can transition to a low-vision region 508 in a transition region 510, as indicated by the protruding star pattern. The change in vision can be accomplished in many different ways. In some embodiments, the low-vision region can have the same number of pixels as the high-vision region and simply display content at a lower resolution. For example, four pixels in the low-vision region 508 can display the same value, such that the low-vision region 508 has a spatial resolution that is one-quarter of the spatial resolution of the high-vision region 506. In other embodiments, the spacing between pixels in the low-vision region 508 can exceed that in the high-vision region 506. In some embodiments, the pixels in the low-vision region 508 can be larger than those in the high-vision region 506 due to the additional space provided by the greater pixel spacing. Transition region 510 may have pixels that are gradually spaced farther apart, creating a more uniform transition between regions 506 and 508. Note that high-vision region 506 and low-vision region 508 can have many different variations, not limited to differences in spatial perception. For example, low-vision region 508 may display fewer colors than high-vision region 506, refresh at a different rate than high-vision region 506, and further display virtual content at a different intensity level (i.e., brightness) than high-vision region 506.

[0057] FIG. 5B shows a wearable display device 550 having a display 552 with multiple regions 554, 556, and 558. Region 554 can be designed to correspond to the human eye's ability to distinguish color and spatial resolution. Because the center of the eye has the highest density of cones, which have the best ability to distinguish detail and color, region 554 can be configured to emit the highest resolution and truest color reproduction. Region 556 can be configured to display virtual content at a relatively lower spatial and / or color resolution. In some embodiments, region 556 can be positioned along the boundary of a user's ocular field of view of the wearable display device 550. For this reason, the difference between region 556 and region 554 can be implemented over a transition zone between regions 554 and 556 so that the change in resolution is not obvious or distracting to the user of wearable display device 550. Similarly, region 558 can cover a portion of the user's field of view corresponding to far peripheral vision. Region 558 can be configured to display virtual content at an even lower resolution than region 556. For example, region 558 can be configured to display virtual content in grayscale.

[0058] 5C shows a display 570 similar to displays 502 and 552. The distribution of pixels 572 can vary across display 570. In particular, pixels 572 are shown to have a lower density in a peripheral region 574 and a higher density in a central region 576. By configuring display 570 in this manner, the spatial resolution of any image displayed by display 570 can be gradually reduced as virtual content moves from central region 576 into peripheral region 574 of display 570.

[0059] 6 and 7 detail display technologies that may be used with primary displays, such as primary displays 102, 112, 122, 134, 136, 144, 166, 172, 184, 252, 506, 554, and 576. In some embodiments, peripheral displays may also utilize this type of display technology. The displays may or may not incorporate eye tracking devices to further optimize the locations where high and low resolution images are displayed.

[0060] In FIG. 6 , the viewer's eye 602 is oriented relative to the eyepiece 600 in a first manner so that the viewer may view the eyepiece 600 in a relatively straight-on direction. The orientation of the viewer's eye 602 in FIG. 6 may be the same as or similar to the orientation of the viewer's eye 302 as described above with reference to FIGS. 3A-3B , for example, and may be determined by the AR system using sensing components and / or one or more of the techniques described herein. Thus, at the stage depicted in FIG. 6 , the AR system may employ a head-tracking and foveated-tracking rendering viewpoint in a relative position and orientation. The FOV of the foveated-tracking rendering viewpoint employed by the AR system may encompass the virtual object 612, for example, but may not encompass either of the virtual objects 611 and 613. 6, it follows that the AR system may render virtual object 612 in fine detail as it would be captured from the perspective of a foveated tracking virtual camera, and may render virtual objects 611 and 613 in lower detail as they would be captured from the perspective of a head-tracking virtual camera. In addition, the AR system may project light representing such renderings of virtual objects 611, 612, and 613 through eyepiece 600 onto the retina of viewer's eye 602. In some embodiments, the AR system may render virtual object 612 in lower detail as it would be captured from the perspective of a head-tracking virtual camera.

[0061] 6 also illustrates an exemplary bright field 630A that is outcoupled by the eyepiece 600 and projected onto the retina of the viewer's eye 602. The bright field 630A may include various angular light components that represent one or more of the above-described renderings of virtual objects 611, 612, and 613. For example, the angular light components of the bright field 630A representing virtual object 611 as it would be captured from the perspective of a head-tracked virtual camera may include those to be projected onto the retina of the viewer's eye 602 at angles ranging from −α to −β angular units relative to the viewer's eye 602, and the angular light components of the bright field 630A representing virtual object 613 as it would be captured from the perspective of a head-tracked virtual camera may include those to be projected onto the retina of the viewer's eye 602 at angles ranging from ε to ζ angular units relative to the viewer's eye 602. Similarly, the angular light components of bright field 630A representing virtual object 612 as it would be captured from the perspective of a foveal-tracking virtual camera may include those to be projected onto the fovea of ​​the viewer's eye 602 at angles ranging from −γ to δ angular units relative to the viewer's eye 602. Thus, the components of bright field 630A representing virtual object 612 (i.e., components to be projected at angles ranging from −γ to δ angular units relative to the viewer's eye 602) may be more densely distributed in angular space than the components of bright field 630A representing virtual objects 611 or 613 (i.e., components to be projected at angles ranging from −α to −β or ε to ζ angular units relative to the viewer's eye 602). In this manner, the resolution at which virtual object 612 may be rendered and presented to the viewer may be higher than the resolution at which virtual objects 611 or 613 may be rendered and presented to the viewer.

[0062] In Figure 7, the viewer's eye 602 is oriented relative to the eyepiece 600 in a second manner that is different from the first manner in which the viewer's eye 602 is oriented relative to the eyepiece 600 in Figure 6. The orientation of the viewer's eye 602 in Figure 7 may be determined by the AR system using sensing components and / or one or more of the techniques described herein. Thus, at the stage depicted in Figure 7, the AR system may adopt a head-tracking and foveated-tracking rendering viewpoint in a relative position and orientation similar to those of the head-tracking and foveated-tracking rendering viewpoint. In the particular example of Figure 7, the FOV of the foveated-tracking rendering viewpoint adopted by the AR system may, for example, encompass virtual object 613 but may not encompass either of virtual objects 611 and 612. 7, it follows that the AR system may render virtual object 613 in high definition as it would be captured from the perspective of a foveated tracking virtual camera, and may render virtual objects 611 and 612 in lower definition as they would be captured from the perspective of a head-tracking virtual camera. In addition, the AR system may project light representing such renderings of virtual objects 611, 612, and 613 through eyepiece 600 onto the retina of viewer's eye 602. In some embodiments, the AR system may render virtual object 613 in lower definition as it would be captured from the perspective of a head-tracking virtual camera.

[0063] 7 also illustrates an exemplary bright field 630B that is outcoupled by the eyepiece 600 and projected onto the retina of the viewer's eye 602. The bright field 630B may include various angular light components that represent one or more of the above-described renderings of virtual objects 611, 612, and 613. For example, the angular light components of the bright field 630B representing virtual object 611 as it is captured from the perspective of a head-tracked virtual camera may include those to be projected onto the retina of the viewer's eye 602 at angles ranging from −α to −β angular units relative to the viewer's eye 602, and the angular light components of the bright field 630B representing virtual object 612 as it is captured from the perspective of a head-tracked virtual camera may include those to be projected onto the retina of the viewer's eye 602 at angles ranging from −γ to δ angular units relative to the viewer's eye 602. Similarly, the angular light components of bright field 630B representing virtual object 613 as it is captured from the perspective of the foveal tracking virtual camera may include those to be projected onto the fovea of ​​the viewer's eye 602 at angles ranging from ε to ζ angular units relative to the viewer's eye 602. Thus, the components of bright field 630B representing virtual object 613 (i.e., components to be projected at angles ranging from ε to ζ angular units relative to the viewer's eye 602) may be more densely distributed in angular space than the components of bright field 630A representing virtual objects 611 or 612 (i.e., components to be projected at angles ranging from −α to −β or −γ to δ angular units relative to the viewer's eye 602). In this way, the resolution at which virtual object 613 can be rendered and presented to the viewer may be higher than the resolution at which virtual objects 611 or 612 can be rendered and presented to the viewer. In fact, from the stage of Figure 6 to the stage of Figure 7, the AR system described herein with reference thereto effectively redirects the viewpoint so that the virtual content can then be viewed in high resolution according to the change in line of sight of the viewer's eye 602 between stages.

[0064] 8A-8C schematically illustrate a display system 800 according to some other embodiments of the present invention. The display system 800 includes an image source 810, a beam splitter 830, a first optical lens 842, a second optical lens 844, a third optical lens 845, a fourth optical lens 846, a fifth optical lens 847, a sixth optical lens 848, a scanning mirror 860, a polarizer 880, and a switching polarization rotator 890. These components enable a projector to input light from multiple image sources into the display and help generate a composite image on the display that includes images with variable resolution.

[0065] 8A-8C illustrate display system 800 in each of three different stages. In each of the three stages, image source 810, which may be coupled to the temple of a wearable display device, can output a range of angular brightfield components representing the virtual content as captured from the perspective of a head-tracking virtual camera and a range of angular brightfield components representing the virtual content as captured from the perspective of a foveated tracking virtual camera. The two sets of angular brightfield components may be, for example, time-division multiplexed, polarization-division multiplexed, wavelength-division multiplexed, etc. Thus, the angular bright field component associated with the head-tracking virtual camera can be redirected upward by the polarizing beam splitter 830 along a first optical path through the first and second optical lenses 842 and 844, and the angular bright field component associated with the foveal tracking virtual camera can pass through the polarizing beam splitter 830 along a second optical path through the third and fourth optical lenses 845 and 846 towards the scanning mirror 860 and be reflected upward through the fifth and sixth optical lenses 847 and 848.

[0066] The virtual content represented by the angular brightfield component associated with the head-tracking virtual camera may be rendered at a relatively low resolution upstream from image source 810, while the virtual content represented by the angular brightfield component associated with the foveated tracking virtual camera may be rendered at a relatively high resolution upstream from image source 810. And, as shown in Figures 8A-8C, each of display systems 800 may be configured to output the angular brightfield component associated with the head-tracking rendering viewpoint and the angular brightfield component associated with the foveated tracking rendering viewpoint as high-FOV and low-FOV brightfields, respectively. In each of Figures 8A-8C, the brightfield component propagating along the first optical path is output by display system 800 as a relatively wide cone of light 852.

[0067] At the stage depicted in FIG. 8A , scanning mirror 860 is in a first position. Accordingly, it can be seen that the bright field component passing through polarizing beam splitter 830 and propagating along the second optical path is output by display system 800 as a relatively narrow cone of light 854A that spans a substantially central region of angular space. Within the context of the examples described above with reference to FIGS. 6-7 , display system 800 may position scanning mirror 860 in the first position shown in FIG. 8A when, for example, the user's eye is directed in a manner similar to that of viewer's eye 602 in FIG. 6 . In this manner, light component 854A may represent virtual content, such as virtual object 612, in a relatively central region of the rendering space. In addition to the examples of FIGS. 6-7 , the relatively wide cone of light 852 may include virtual content, such as virtual objects 611 and 613, in regions offset from the center of the rendering space. In some examples, the relatively wide cone of light 852 may include additional light components that represent the same virtual content as represented by light component 854A, but at a lower resolution.

[0068] At the stage depicted in FIG. 8B , scanning mirror 860 is in a second position different from the first position. Accordingly, it can be seen that the bright field component passing through polarizing beam splitter 830 and propagating along the second optical path is output by display system 800 as a relatively narrow cone of light 854B spanning a substantially off-center region of angular space. Within the context of the example described above with reference to FIGS. 6-7 , display system 800 may, for example, position scanning mirror 860 in the second position shown in FIG. 8B when the user's eye is directed in a manner similar to that of viewer's eye 602 while the viewer is viewing virtual object 611. In this manner, light component 854B may represent virtual content, such as virtual object 611, in a relatively off-center region of rendering space. 6-7, the relatively wide cone of light 852 may include virtual content, such as virtual object 613, in other off-center regions of the rendering space, and virtual content, such as virtual object 612, in a central region of the rendering space. In some examples, the relatively wide cone of light 852 may further include light components that represent the same virtual content as represented by light component 854B, but at a lower resolution.

[0069] At the stage depicted in FIG. 8C , scanning mirror 860 is in a third position different from the first and second positions. Accordingly, it can be seen that the bright field component passing through polarizing beam splitter 830 and propagating along the second optical path is output by display system 800 as a relatively narrow cone of light 854C spanning another, different, substantially off-center region of angular space. Within the context of the example described above with reference to FIGS. 6-7 , display system 800 may, for example, position scanning mirror 860 in the second position shown in FIG. 8C when the user's eye is directed in a manner similar to that of viewer's eye 602 in FIG. 7 . In this manner, light component 854C may represent virtual content, such as virtual object 613, in another, relatively off-center region of the rendering space. 6-7, the relatively wide cone of light 852 may include virtual content, such as virtual object 611, in an off-center region of the rendering space described above with reference to FIG. 8B, and virtual content, such as virtual object 612, in a central region of the rendering space. In some examples, the relatively wide cone of light 852 may further include a light component that represents the same virtual content as represented by light component 854C, but at a lower resolution.

[0070] Various aspects, embodiments, implementations, or features of the described embodiments can be used separately or in any combination. Various aspects of the described embodiments can be implemented by software, hardware, or a combination of hardware and software. The described embodiments can also be embodied as computer-readable code on a computer-readable medium for controlling manufacturing operations or for controlling a production line. A computer-readable medium is any data storage device that can store data that can then be read by a computer system. Examples of computer-readable media include read-only memory, random-access memory, CD-ROMs, HDDs, DVDs, magnetic tape, and optical data storage devices. The computer-readable medium can also be distributed over network-coupled computer systems so that the computer-readable code is stored and executed in a distributed fashion.

[0071] The foregoing description, for purposes of explanation, uses specific nomenclature to provide a thorough understanding of the described embodiments. However, it will be apparent to those skilled in the art that specific details are not required to practice the described embodiments. Thus, the foregoing descriptions of specific embodiments are presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the described embodiments to the precise forms disclosed. It will be apparent to those skilled in the art that numerous modifications and variations are possible in light of the above teachings.

Claims

1. 1. A display assembly for a head mounted display device, the head mounted display device comprising a frame including a front portion and at least one temple arm, the display assembly comprising: a first region having a first resolution, the first region being disposed within the front portion; a second region at least partially surrounding the first region and having a second resolution substantially lower than the first resolution, the second region extending from the front portion onto an interior-facing surface of the at least one temple arm; a transition region between the first and second regions having variable resolution; and Equipped with the variable resolution is higher on a first side of the transition region adjacent to the first region than on a second side of the transition region adjacent to the second region; the transition region includes pixels that are spaced progressively farther apart from one another moving from the first side of the transition region to the second side of the transition region; the head-mounted display device is an augmented reality device, and at least the first region of the display assembly is transparent to allow the outside world to be visible in areas where digital content is not actively displayed; Display assembly.

2. The display assembly of claim 1 , wherein the first region is configured to cover a portion of a field of view of a user's eye.

3. 10. The display assembly of claim 1, wherein the second region renders virtual content in grayscale and the first region renders virtual content in color.

4. The display assembly of claim 1 , wherein the second region comprises a transparent organic light emitting diode (OLED) display.

5. The display assembly of claim 1 , further comprising a projector configured to illuminate the first area by projecting content onto an interior or exterior surface of the display assembly.

6. 10. The display assembly of claim 1, wherein the second region is sized to fill a gap between a periphery of the first region and a user's face when the head-mounted display device is worn.

7. The display assembly of claim 1 , wherein the second region has a greater curvature than the first region.

Images