Virtual and augmented reality systems and methods having unequal numbers of component color images distributed across the depth plane - Patents.com

Abstract

To provide virtual reality and augmented reality imaging and visualization systems.SOLUTION: Images perceived to be substantially full color or multi-colored may be formed using component color images that are distributed in unequal numbers across a plurality of depth planes. The distribution of component color images across the depth planes may vary based on color. In some embodiments, a display system includes a stack of waveguides that each output light of a particular color, where some of the colors have fewer numbers of associated waveguides than other colors. The stack of waveguides may collectively include multiple pluralities (e.g., first and second pluralities) of waveguides, each configured to produce an image by outputting light corresponding to a particular color. The total number of waveguides in the second plurality of waveguides is less than the total number of waveguides in the first plurality of waveguides, and may be more than the total number of waveguides in a third plurality of waveguides, in embodiments where three component colors are utilized.SELECTED DRAWING: None

Description

[Technical field]

[0001] (Incorporating Related Applications) This application claims the benefit of priority under §119(e) of U.S. Provisional Application No. 62 / 275,987, filed January 7, 2016, the entire disclosure of which is incorporated herein by reference.

[0002] (Incorporating Related Applications) This application incorporates by reference the entire contents of each of the following patent applications: U.S. Application No. 14 / 641,376, filed March 7, 2015; U.S. Application No. 14 / 555,585, filed November 27, 2014; U.S. Application No. 14 / 690,401, filed April 18, 2015; U.S. Application No. 14 / 212,961, filed March 14, 2014; U.S. Application No. 14 / 331,218, filed July 14, 2014; and U.S. Provisional Application No. 62 / 156,809, filed May 4, 2015.

[0003] (Field) The present disclosure relates to virtual reality and augmented reality imaging and visualization systems. [Background technology]

[0004] Description of Related Art Modern computing and display technologies have facilitated the development of systems for so-called "virtual reality" or "augmented reality" experiences, in which digitally reproduced images or portions thereof are presented to a user in a manner that may appear or be perceived as real. Virtual reality or "VR" scenarios typically involve the presentation of digital or virtual image information without transparency to other actual real-world visual inputs. Augmented reality or "AR" scenarios typically involve the presentation of digital or virtual image information as an extension to the visualization of the real world around the user. For example, referring to FIG. 1, an augmented reality scene (1) is depicted in which a user of the AR technology sees a real-world park-like setting (1100) featuring people, trees, and buildings in the background, and a concrete platform (1120). In addition to these items, a user of the AR technology also perceives as "seeing" a robotic figure (1110) standing on the real-world platform (1120) and a flying, cartoon-like avatar character (1130) that appears to be an anthropomorphic bumblebee, although these elements (1130, 1110) do not exist in the real world. Because the human visual perception system is complex, creating VR or AR technology that facilitates comfortable, natural-feeling, and rich presentation of virtual image elements among other virtual or real-world image elements is challenging.

[0005] The systems and methods disclosed herein address various challenges associated with VR and AR technologies. Summary of the Invention [Means for solving the problem]

[0006] (summary) In some embodiments, a display system is provided. The system includes a stack of waveguides including a first plurality of waveguides each configured to generate an image by outputting light in a first wavelength range, and a second plurality of waveguides each configured to generate an image by outputting light in a second wavelength range. The first plurality of waveguides has a total number of waveguides greater than the total number of waveguides forming the second plurality of waveguides. The system may also include a third plurality of waveguides each configured to display an image by outputting light corresponding to a third wavelength range. The third plurality of waveguides may include a total number of waveguides different from the number of waveguides in the first or second plurality of waveguides. In some embodiments, the first wavelength range encompasses light of a green color, the second wavelength range encompasses light of a red color, and the third wavelength range encompasses light of a blue color.

[0007] In some other embodiments, a display system is provided. The display system includes a display configured to output a plurality of component color images, the component color images corresponding to a plurality of depth planes. The component color images of a first color are greater in total than the component color images of a second color. The component color images of the first color are formed by light having wavelengths within a first wavelength range, and the component color images of the second color are formed by light having wavelengths within a second wavelength range.

[0008] The system includes a stack of waveguides including a first plurality of waveguides each configured to generate an image by outputting light corresponding to a first color and a second plurality of waveguides each configured to generate an image by outputting light corresponding to a second color different from the first color. The system also includes a controller configured to introduce one or more of blurring and varying pixel size into the image based on content of the image to be generated by a waveguide of the first or second plurality of waveguides.

[0009] In another embodiment, a display system comprises one or more waveguides comprising a first plurality of diffractive optical elements configured to couple out light in a first wavelength range to generate a first plurality of component color images and a second plurality of diffractive optical elements configured to couple out light in a second wavelength range to generate a second plurality of component color images. An image of the first plurality of component color images is set on one of X maximum possible depth planes for the first plurality of diffractive optical elements and an image of the second plurality of component color images is set on one of Y maximum possible depth planes for the second plurality of diffractive optical elements, where X and Y are different integers.

[0010] In some other embodiments, a display system is provided. The system includes a camera for determining pupil size, a stack of waveguides, and a controller. The stack of waveguides includes a first plurality of waveguides each configured to generate an image by outputting light corresponding to a first color, and a second plurality of waveguides each configured to generate an image by outputting light corresponding to a second color different from the first color. The controller is configured to vary a characteristic of the output light based on the determined pupil size. The present invention provides, for example, the following: (Item 1) 1. A display system, the system comprising a stack of waveguides, the stack of waveguides comprising: a first plurality of waveguides, each configured to generate an image by outputting light in a first wavelength range; a second plurality of waveguides, each configured to generate an image by outputting light in a second wavelength range; and Equipped with The system wherein the first plurality of waveguides totals a greater number of waveguides than the second plurality of waveguides. (Item 2) Item 14. The system of item 1, further comprising a third plurality of waveguides, each configured to display an image by outputting light in a third wavelength range. (Item 3) 3. The system of claim 2, wherein the third plurality of waveguides totals a different number of waveguides than the first or second plurality of waveguides. (Item 4) 3. The system of claim 2, wherein the first, second, and third wavelength ranges correspond to different first, second, and third colors, respectively, and the third color is red. (Item 5) 5. The system of claim 4, wherein the second color is selected from the group consisting of blue and green, and the first color is the other color from the group consisting of blue and green. (Item 6) 6. The system of claim 5, wherein the third plurality of waveguides totals the same number of waveguides as one of the first or second plurality of waveguides. (Item 7) each waveguide of the first plurality of waveguides has an associated depth plane, each waveguide of the first plurality of waveguides configured to generate an image that appears to originate from that waveguide's associated depth plane, each waveguide of the first plurality of waveguides having an associated depth plane that is different from at least one other waveguide of the first plurality of waveguides; each waveguide of the second plurality of waveguides having an associated depth plane, each waveguide of the second plurality of waveguides being configured to generate an image that appears to originate from that waveguide's associated depth plane, each waveguide of the second plurality of waveguides having an associated depth plane that is different from at least one other waveguide of the second plurality of waveguides; each waveguide of the third plurality of waveguides has an associated depth plane, each waveguide of the third plurality of waveguides configured to generate an image that appears to originate from that waveguide's associated depth plane, each waveguide of the third plurality of waveguides having an associated depth plane that is different from at least one other waveguide of the third plurality of waveguides; Item 2. The system according to item 2. (Item 8) 8. The system of claim 7, wherein the spacing between depth planes is different for each of the first, second, and third plurality of waveguides. (Item 9) 8. The system of claim 7, wherein each waveguide of the first, second, and third plurality of waveguides comprises a light extraction optical element, and a configuration for the light extraction optical element varies depending on the depth plane associated with the waveguide. (Item 10) 10. The system of claim 9, wherein the light extraction optical element has an optical power to provide a diverging beam. (Item 11) Item 11. The system of item 10, wherein the light extraction optical element comprises a diffraction grating. (Item 12) Item 11. The system of item 10, wherein the light extraction optical elements for different depth planes have different refractive powers to provide different divergences of the exiting light for each depth plane. (Item 13) 3. The system of claim 2, wherein the waveguide stack is configured to output N colors, the waveguide stack comprising a plurality of N waveguides, each configured to display an image by outputting light corresponding to one of the N colors, each of the plurality of waveguides being in total a different number than others of the plurality of waveguides, and N being a positive integer. (Item 14) 2. The system of claim 1, further comprising an exit pupil associated with each waveguide of the first and second plurality of waveguides, wherein an exit pupil size for a waveguide of the first plurality of waveguides is larger than an exit pupil size for a waveguide of the second plurality of waveguides. (Item 15) 2. The system of claim 1, wherein each waveguide of the first and second pluralities of waveguides outputs light with an output beam size, the output beam size differing between waveguides of the first and second pluralities of waveguides. (Item 16) Item 1. The system of item 1, wherein each waveguide of the first and second plurality of waveguides outputs light with an exit beam size, and the exit beam size for a waveguide of the first plurality of waveguides is less than 5 mm. (Item 17) 2. The system of claim 1, wherein a waveguide of the first plurality of waveguides outputs light with a different output beam size than a waveguide of the second plurality of waveguides. (Item 18) 2. The system of claim 1, further comprising a third plurality of waveguides, each configured to generate an image by outputting light in a third wavelength range, wherein a waveguide of the third plurality of waveguides outputs light with a different exit beam size than a waveguide of the first or second plurality of waveguides. (Item 19) Item 1 , the system further comprising an image input device, each image input device configured to input image information into a corresponding input edge of each of the first and second plurality of waveguides. (Item 20) 20. The system of claim 19, wherein the image input device comprises a variable aperture for light to be input into corresponding input edges of the first and second plurality of waveguides. (Item 21) 22. The system of claim 1, wherein the system comprises a frame configured to be worn by a viewer, the waveguide stack being attached to the frame. Further comprising another stack of waveguides; 2. The system of claim 1, wherein the stack of waveguides is configured to direct image information to a first eye of a viewer and the other stack of waveguides is configured to direct image information to a second eye of the viewer. (Item 23) the stack of waveguides and the other stack of waveguides are configured to generate image sets at a plurality of depth planes; Progression of the distance between the depth planes, the total number of depth planes, Focal length for the depth plane and 23. The system of claim 22, wherein one or more of are different between the depth plane of the stack of waveguides and the depth plane of another stack of waveguides. (Item 24) Each of the waveguides is configured to propagate light therein by total internal reflection, each waveguide comprising: Item 1 , the system comprising a diffractive optical element configured to redirect light propagating within the waveguide out of the waveguide towards a viewer. (Item 25) Item 14. The system of item 1, further comprising an optical element or image processing module for introducing blur into the image information injected into the waveguide. (Item 26) 2. The system of claim 1, wherein the optical element or image processing module is configured to introduce different amounts of blur into the waveguides of the first and second plurality of waveguides. (Item 27) Item 1, further comprising a third plurality of waveguides, each configured to generate an image by outputting light in a third wavelength range, and the optical element or image processing module configured to introduce different amounts of blur into waveguides of the first, second, and third plurality of waveguides. (Item 28) 2. The system of claim 1, wherein a first pixel size for an image generated by the first plurality of waveguides is smaller than a second pixel size for an image generated by the second plurality of waveguides. (Item 29) 2. The system of claim 1, further comprising a third plurality of waveguides, wherein a third pixel size for an image generated by the third plurality of waveguides is different from the first or second pixel size. (Item 30) Item 10. The system of item 1, further comprising a controller configured to distribute, between two neighboring depth planes, the intensity of points between the neighboring depth planes. (Item 31) 1. A display system, comprising: a display configured to output a plurality of component color images, the component color images corresponding to a plurality of depth planes; A system in which component color images of a first color total greater than a component color image of a second color, the component color images of the first color being formed by light having wavelengths within a first wavelength range, and the component color images of the second color being formed by light having wavelengths within a second wavelength range. (Item 32) Item 32. The system of item 31, wherein the component color images of a third color are fewer in total than the component color images of the second color, and the component color images of the third color are formed by light having wavelengths within a third wavelength range. (Item 33) Item 32. The system of item 31, wherein the system comprises a plurality of waveguides, each waveguide configured to output a single component color image. (Item 34) Item 34. The system of item 33, wherein each waveguide is further configured to output a component color image for a single associated depth plane. (Item 35) Item 34. The system of item 33, wherein each waveguide comprises an optical element with a refractive power configured to provide the perception of a different depth plane. (Item 36) Item 32. The system of item 31, wherein the spacing between depth planes is different for each of the multiple depth planes for the first, second, and third color component color images. (Item 37) Item 32. The system of item 31, wherein an exit pupil is associated with the light forming each component color image, the exit pupil size varying with the color of the component color image. (Item 38) Item 32. The system of item 31, wherein the light forming each component color image has an exit beam size, the exit beam size being less than 5 mm. (Item 39) Item 39. The system of item 38, wherein the exit beam size varies with the color of the component color image. (Item 40) Item 32. The system of item 31, wherein the system comprises a frame configured to be worn by a viewer, the display being attached to the frame. (Item 41) the display is configured to output two sets of component color images, one for each eye; Progression of the distance between the depth planes, the total number of depth planes, Focal length for the depth plane and 42. The system of claim 31, wherein one or more of the following are different between each pair of depth planes. Item 32. The system of item 31, further comprising an optical element or image processing module for introducing blur into the component color images. (Item 43) Item 43. The system of item 42, wherein the optical element or image processing module is configured to introduce different amounts of blur into the different color component color images. (Item 44) Item 32. The system of item 31, wherein a first pixel size for a component color image of the first color is different from a second pixel size for a component color image of the second color. (Item 45) Item 45. The system of item 44, further comprising a component color image of a third color, wherein a third pixel size for the component color image of the third color is different from the first or second pixel size. (Item 46) Item 45. The system of item 44, wherein the component color image comprises four or more colors. (Item 47) Item 32. The system of item 31, further comprising a controller configured to distribute, between two neighboring depth planes, the luminance of points between the neighboring depth planes. (Item 48) 1. A display system, comprising: one or more waveguides, the one or more waveguides comprising: a first plurality of diffractive optical elements configured to outcouple light of a first wavelength range and generate a first plurality of component color images, each image being positioned at one of X maximum possible depth planes for the first plurality of diffractive optical elements; a second plurality of diffractive optical elements configured to outcouple light of a second wavelength range and generate a second plurality of component color images, each image positioned on one of Y maximum possible depth planes for the second plurality of diffractive optical elements; Equipped with X and Y are integers, and X and Y are distinct in the system. (Item 49) Item 49. The system of item 48, wherein the first and second pluralities of diffractive optical elements are part of a single waveguide. (Item 50) Item 50. The system of item 49, wherein a diffractive optical element of the first and second pluralities of diffractive optical elements is switchable between an optically diffractive state and a non-diffractive state. (Item 51) Item 50. The system of item 49, wherein the first and second pluralities of diffractive optical elements comprise liquid crystals. (Item 52) Item 49. The system of item 48, wherein a diffractive optical element of the first plurality of diffractive optical elements is part of one or more waveguides different from a diffractive optical element of the first plurality of diffractive optical elements, and a diffractive optical element of the first and second plurality of diffractive optical elements is switchable between an optical diffractive state and a non-diffractive state. (Item 53) Item 49. The system of item 48, further comprising a third plurality of diffractive optical elements configured to outcouple light of a third wavelength range and generate a third plurality of component color images, each image being positioned on one of Z maximum possible depth planes for the third plurality of diffractive optical elements. (Item 54) Item 54. The system of item 53, wherein Z is different from X and Y. (Item 55) Item 54. The system of item 53, wherein the first, second, and third wavelength ranges of light comprise red, green, and blue colored light, respectively. (Item 56) Item 49. The system of item 48, wherein a progression of spacing between depth planes for the first plurality of diffractive optical elements is different from a progression of spacing between depth planes for the second plurality of diffractive optical elements. (Item 57) Item 49. The system of item 48, wherein the system is configured to outcouple the light using different exit pupil sizes depending on the wavelength of the light being outcoupled. (Item 58) Item 49. The system of item 48, wherein the light forming each component color image has an exit beam size, the exit beam size being less than 5 mm. (Item 59) Item 59. The system of item 58, wherein the exit beam size varies with the color of the component color image. (Item 60) Item 49. The system of item 48, wherein the system comprises a frame configured to be worn by a viewer, the display being attached to the frame. (Item 61) the display system is configured to output two sets of component color images, one for each eye; Progression of the distance between the depth planes, the total number of depth planes, Focal length for the depth plane and Item 49. The system of item 48, wherein one or more of differ between each set of the depth planes. (Item 62) Item 49. The system of item 48, further comprising an optical element or image processing module for introducing blur into the component color images. (Item 63) Item 63. The system of item 62, wherein the optical element or image processing module is configured to introduce different amounts of blur into the different color component color images. (Item 64) Item 49. The system of item 48, wherein a first pixel size for the component color image generated using light in the first wavelength range is different from a second pixel size for the component color image generated using light in the second wavelength range. (Item 65) Item 65. The system of item 64, further comprising a component color image of a third color, wherein a third pixel size for the component color image of the third color is different from the first or second pixel size. (Item 66) Item 49. The system of item 48, further comprising a controller configured to distribute, between two neighboring depth planes, the luminance of points between the two neighboring depth planes. (Item 67) 1. A display system, comprising: a display configured to output a plurality of component color images, the component color images corresponding to a plurality of depth planes; Controller and Equipped with The controller determines, based on content of an image to be displayed for at least one of the depth planes, Blurry and Change in pixel size and into the image. (Item 68) The display comprises a stack of waveguides, the stack of waveguides comprising: a first plurality of waveguides, each configured to output light corresponding to a first color, thereby generating a component color image; a second plurality of waveguides, each configured to generate a component color image by outputting light corresponding to a second color different from the first color; Item 68. The display system of item 67, comprising: (Item 69) 1. A display system, comprising: a camera for determining pupil size; a display configured to output light and display a plurality of component color images, the component color images corresponding to a plurality of depth planes; a controller configured to vary a characteristic of the output light based on the determined pupil size; and A display system comprising: (Item 70) The display comprises: a first plurality of waveguides, each configured to generate a component image by outputting light corresponding to a first color; a second plurality of waveguides, each configured to generate a component image by outputting light corresponding to a second color different from the first color; 70. The display system of item 69, comprising: [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 illustrates a user's view of an augmented reality (AR) device. [Diagram 2] FIG. 2 illustrates an example of a wearable display system. [Diagram 3] FIG. 3 illustrates a conventional display system for simulating a three-dimensional image for a user. [Figure 4] FIG. 4 illustrates aspects of an approach for simulating a three-dimensional image using multiple depth planes. [Diagram 5] 5A-5C illustrate the relationship between the radius of curvature and the radius of focus. [Figure 6] FIG. 6 illustrates an example of a waveguide stack for outputting image information to a user. [Figure 7] FIG. 7 shows an example of an output beam output by a waveguide. [Figure 8] FIG. 8 illustrates an example of a waveguide stack in which each depth plane has three associated waveguides, each outputting a different color of light. [Figure 9] FIG. 9 illustrates an example of a waveguide stack in which different component colors have different numbers of associated waveguides. [Figure 10] FIG. 10 illustrates another example of a waveguide stack in which different component colors have different numbers of associated waveguides. [Figure 11] FIG. 11 illustrates an example of a display system configured to monitor pupil size. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] (Detailed Description) VR and AR experiences can be provided by a display system having a display in which images corresponding to multiple depth planes are provided to a viewer. The images can be different for each depth plane (e.g., providing slightly different presentations of a scene or object) and can be focused separately by the viewer's eyes. As a result, the user is provided with depth cues based on the accommodation of the eyes required to focus on different image features for a scene located on different depth planes and / or based on the observation of different image features on different depth planes that are out of focus. As discussed elsewhere herein, such depth cues provide a realistic perception of depth.

[0013] In some configurations, at each depth plane, a full-color image may be formed by overlaying component images, each having a particular component color. For example, a red, green, and blue image may each be output to form each full-color image. As a result, each depth plane may have multiple component color images associated with it. As a result, a large number of component color images may undesirably be required to provide full-color images for the desired number of depth planes.

[0014] It has been found that a realistic perception of depth and a substantially full-color image can be achieved without providing component color images for all component colors in all depth planes presented by the display. Without being limited by theory, it is believed that the depth of field for different colors can vary. In other words, the human eye may have different sensitivities when imaging different wavelengths of light, such that an image or feature that is clearly distinguishable as being on a different depth plane in one color may not be perceived as being on a different depth plane in another color. For example, two images on different depth planes may appear well-focused for one color, but out of focus for another color. Thus, the depth of field (or alternatively, the depth of focus) is believed to be different for different colors. In addition, as disclosed herein, it may be possible to manipulate the perceived depth of field for different colors, or the sensitivity of the eye to discriminating depth cues for different component colors. Such manipulation can reduce the number of depth planes that can be discriminated by the eye.

[0015] In some embodiments, an image to be perceived as substantially full color may be formed using component color images distributed in unequal numbers across multiple depth planes. For example, rather than providing an image of each component color for each depth plane presented by the display, the distribution of component color images across the depth planes may vary based on color. For example, the total number of images of a first component color distributed across the depth planes may be different from the total number of images of a second component color. In some embodiments, the total number of images of a third component color may be different from the total number of the first or second component colors. In some embodiments, at least one component color has a total number of images that differ from another component color. In some other embodiments, all component colors have a total number of images that differ from each other. In some embodiments, the total number of images of a particular component color is chosen based on the sensitivity of the eye to distinguish different depth planes using the wavelength of light corresponding to that color. For example, if the component colors are red, green, and blue, the total number of green component images may exceed the total number of red component images, and the total number of red component images may exceed the total number of blue component images. As a result, in some embodiments, the separation in depth between the green component images may be shorter than the red component images. This configuration may be feasible because the eye may have an increased depth of field or focus for red light compared to green light, thereby allowing the two red images to be in focus despite being separated by a depth that exceeds the depth of field or focus for a similarly located green image.

[0016] In some embodiments, image information may be projected or output by a stack of waveguides. In some embodiments, the display system includes waveguides that each output light of a particular color, with some colors having fewer associated waveguides than other colors, as discussed above. For example, the display system may include a stack of waveguides formed by a plurality of pluralities (e.g., a first and a second plurality) of waveguides. Each of the plurality of waveguides is configured to generate an image by outputting light corresponding to a particular color. The total number of waveguides in the second plurality of waveguides is less than the total number of waveguides in the first plurality of waveguides. In some embodiments where a full-color image may be formed by three or more component colors, there may be correspondingly more than two pluralities of waveguides (e.g., a first, second, and third plurality of waveguides).

[0017] In some embodiments, the first plurality of waveguides may be configured to output green light (i.e., light of a wavelength corresponding to the perceived color green), the second plurality of waveguides may be configured to output blue light (light of a wavelength corresponding to the perceived color green), and the third plurality of waveguides may be configured to output red light (light of a wavelength corresponding to the perceived color green). The second plurality of waveguides (to output blue light) may include a smaller total number of waveguides than the first or third plurality of waveguides (to output green and red light, respectively). Additionally, in some embodiments, the total number of waveguides in the third plurality of waveguides may be less than in the first plurality of waveguides, or vice versa.

[0018] In some embodiments, one or more of the waveguides may include optical elements configured to couple light out and form images for two or more depth planes. For example, the waveguides may include optical elements, such as diffractive optical elements, that are switchable between an "on" state configured to couple or redirect light out and an "off" state that does not appreciably couple light out. In some embodiments, a single waveguide, or an individual waveguide of a stack of waveguides, may include multiple sets of optical elements, each set configured to form a specific color image for a specific depth plane. Once one set of optical elements has coupled light out and formed a specific component color image at a specific depth plane, it may be switched off, and another set of optical elements may be turned on to form another specific component color image at a specific depth plane.

[0019] In some embodiments, a display may include at least one waveguide with optical elements configured to outcouple multiple different colors of light without any optical elements that are switchable between on and off states. For example, a single waveguide may include multiple sets of optical elements, each set configured to outcouple a different color of light. The different sets may have optical activity that is selective for one or more particular wavelengths of light, thereby allowing different component color images to be formed using a single waveguide.

[0020] It should be appreciated that the embodiments disclosed herein may provide one or more of the following advantages. For example, as light from the outside world and from several waveguides propagates through the stack of waveguides to reach the viewer, optical artifacts such as reflections and / or other distortions may occur as the light propagates through and interacts with the waveguides. As a result, reducing the number of waveguides may reduce these optical artifacts. Reducing the number of waveguides may also improve manufacturability and yield, and reduce device costs, for example, by reducing the number of parts utilized within the display system, thereby reducing the structural and electrical complexity of the display system. In addition, even if the number of waveguides is not reduced, reducing the number of component color images per depth plane may reduce the processing or computational load on the auxiliary image processing system, since the system will be required to process less image information (since there are fewer total component color images to process). This may also improve display system performance, for example, by reducing the commitment of system resources to process images, improving responsiveness, and reducing latency.

[0021] The embodiments disclosed herein generally include a display system. In some embodiments, the display system is wearable, which may advantageously provide a more immersive VR or AR experience. For example, a display including a waveguide, e.g., a stack of waveguides, for displaying multiple depth planes may be configured to be worn to be positioned in front of the eyes of a user or viewer. In some embodiments, multiple waveguides, e.g., a stack of two waveguides (one for each eye of the viewer), may be utilized to provide different images to each eye.

[0022] 2 illustrates an example of a wearable display system (80). The display system (80) includes a display (62) and various mechanical and electronic modules and systems for supporting the functionality of the display (62). The display (62) may be coupled to a frame (64) that is wearable by a display system user or viewer (60) and configured to position the display (62) directly in front of the user's (60) eye. In some embodiments, a speaker (66) is coupled to the frame (64) and positioned adjacent the user's ear canal (in some embodiments, another speaker, not shown, is positioned adjacent the user's other ear canal to provide stereo / shapeable sound control). The display (62) is operatively coupled (68), such as by wired or wireless connectivity, to a local data processing module (70), which may be mounted in a variety of configurations, such as fixedly attached to the frame (64), fixedly attached to a helmet or hat worn by the user, built into headphones, or otherwise removably attached to the user (60) (e.g., in a backpack or belt-mounted configuration).

[0023] The local processing and data module (70) may comprise a processor and digital memory, such as non-volatile memory (e.g., flash memory), both of which may be utilized to aid in processing, caching, and storing data. The data may include a) data captured from sensors (e.g., which may be operatively coupled to the frame (64) or otherwise attached to the user (60)), such as image capture devices (e.g., cameras), microphones, inertial measurement units, accelerometers, compasses, GPS units, wireless devices, and / or gyroscopes, and / or b) data acquired and / or processed using the remote processing module (72) and / or remote data repository (74), potentially for processing or retrieval and subsequent passage to the display (62). The local processing and data module (70) may be operatively coupled to the remote processing modules (72) and remote data repositories (74) by communications links (76, 78), such as via wired or wireless communication links, such that the remote modules (72, 74) are operatively coupled to each other and available as resources to the local processing and data module (70).

[0024] In some embodiments, the remote processing module (72) may comprise one or more processors configured to analyze and process the data and / or image information. In some embodiments, the remote data repository (74) may comprise a digital data storage facility that may be available through the Internet or other networking configuration in a "cloud" resource configuration. In some embodiments, all data is stored and all calculations are performed within the local processing and data module, allowing for fully autonomous use from the remote module.

[0025] The perception of an image as being "three-dimensional" or "3-D" may be achieved by providing a slightly different presentation of the image to each eye of a viewer. FIG. 3 illustrates a conventional display system for simulating a three-dimensional image for a user. Two different images (5 and 7), one for each eye (4 and 6), are output to the user. The images (5 and 7) are spaced from the eyes (4 and 6) by a distance (10) along an optical or z-axis parallel to the viewer's line of sight. The images (5 and 7) are flat, and the eyes (4 and 6) focus on the images by assuming a single accommodated state. Such a system relies on the human visual system to combine the images (5 and 7) to provide the perception of depth for the combined image.

[0026] However, it should be appreciated that the human visual system is more complex and providing a realistic perception of depth is more difficult. For example, many viewers of conventional "3-D" display systems find such systems uncomfortable or may not perceive any depth perception at all. Without being limited by theory, it is believed that a viewer of an object may perceive the object as "three-dimensional" through a combination of vergence and accommodation. Vergence movement of the two eyes relative to each other (i.e., the rolling movement of the pupils toward or away from each other to converge the gaze of the eyes and fixate on the object) is closely linked to the focusing (or "accommodation") of the eye's lens. Under normal conditions, changing the focus of the eye's lens, or accommodating the eye and changing the focus from one object to another at a different distance, will automatically produce a matching change in vergence to the same distance, under a relationship known as the "accommodation-vergence reflex." Similarly, changes in vergence will also induce matching changes in accommodation under normal conditions. As described herein, many stereoscopic or "3-D" display systems display a scene using slightly different presentations (and therefore slightly different images) to each eye so that a three-dimensional perspective is perceived by the human visual system. However, such systems are uncomfortable for many viewers because, among other things, they simply provide different presentations of the scene but go against the "accommodation-vergence reflex" in which the eyes view all image information in a single accommodated state. Display systems that provide better matching between accommodation and vergence can create a more realistic and comfortable simulation of a three-dimensional image.

[0027] FIG. 4 illustrates aspects of an approach for simulating a three-dimensional image using multiple depth planes. With reference to FIG. 4A, objects at various distances from the eyes (4 and 6) on the z-axis are accommodated by the eyes (4, 6) such that the objects are in focus. The eyes (4 and 6) focus on objects at different distances along the z-axis by adopting a particular accommodated state. As a result, a particular accommodated state may be said to be associated with a particular one of the depth planes (14) having an associated focal length such that an object or part of an object at a particular depth plane is in focus when the eyes are in an accommodated state for that depth plane. In some embodiments, a three-dimensional image may be simulated by providing a different presentation of an image for each of the eyes (4, 6) and also by providing a different presentation of an image corresponding to each of the depth planes. Although shown as separate for clarity of illustration, it should be understood that the fields of view of the eyes (4, 6) may overlap, for example, as the distance along the z-axis increases. Additionally, while shown to be flat for ease of illustration, it should be understood that the contour of the depth plane may be curved in physical space such that all features within the depth plane are in focus for the eye in a particular accommodated state.

[0028] The distance between an object and an eye (4 or 6) can also change the amount of divergence of light from the object as it is viewed by that eye. Figures 5A-5C illustrate the relationship between distance and divergence of light rays. The distances between the object and the eye (4) are represented by R1, R2, and R3, in order of decreasing distance. As shown in Figures 5A-5C, the light rays become more divergent as the distance to the object decreases. As the distance increases, the light rays become more collimated. In other words, the light field generated by a point (an object or part of an object) can be said to have a spherical wavefront curvature that is a function of the distance of the point from the user's eye. The curvature increases with decreasing distance between the object and the eye (4). As a result, at different depth planes, the divergence of the light rays is also different, and the degree of divergence increases with decreasing distance between the depth plane and the viewer's eye 4. Although only a single eye (4) is illustrated in Figures 5A-5C and other figures herein for clarity of illustration, it should be understood that discussion regarding eye (4) may apply to both eyes (4 and 6) of a viewer.

[0029] Without being limited by theory, it is believed that the human eye is typically capable of interpreting and providing depth perception through a finite number of depth planes, and as a result, a highly realistic simulation of perceived depth can be achieved by providing the eye with a different presentation of an image corresponding to each of these limited number of depth planes.

[0030] FIG. 6 illustrates an example of a waveguide stack for outputting image information to a user. The display system 1000 includes a stack of waveguides or a stacked waveguide assembly (178) that can be utilized to provide a three-dimensional perception to the eye / brain using multiple waveguides (182, 184, 186, 188, 190). In some embodiments, the display system (1000) is the system (80) of FIG. 2, and FIG. 6 shows several portions of the system (80) in more detail diagrammatically. For example, the waveguide assembly (178) can be integrated into the display (62) of FIG. 2.

[0031] Continuing with reference to FIG. 6, the waveguide assembly (178) may include multiple features (198, 196, 194, 192) between the waveguides. In some embodiments, the features (198, 196, 194, 192) may be lenses. The waveguides (182, 184, 186, 188, 190) and / or multiple lenses (198, 196, 194, 192) may be configured to transmit image information to the eye with various levels of wavefront curvature or beam divergence. Each waveguide level may be associated with a particular depth plane and may be configured to output image information corresponding to that depth plane. The image input devices (200, 202, 204, 206, 208) may be utilized to input image information into the waveguides (182, 184, 186, 188, 190), each of which may be configured to distribute incident light across each respective waveguide for output towards the eye 4, as described herein. Light exits output surfaces (300, 302, 304, 306, 308) of the image input devices (200, 202, 204, 206, 208) and is input into corresponding input edges (382, 384, 386, 388, 390) of the waveguides (182, 184, 186, 188, 190). In some embodiments, a single beam of light (e.g., a collimated beam) may be launched into each waveguide and output an entire field of cloned, collimated beams that are directed toward the eye (4) at a particular angle (and divergence) corresponding to the depth plane associated with the particular waveguide.

[0032] In some embodiments, the image input devices (200, 202, 204, 206, 208) are individual displays that each generate image information for input into a corresponding waveguide (182, 184, 186, 188, 190, respectively). In some other embodiments, the image input devices (200, 202, 204, 206, 208) are the output of a single multiplexed display, which may convey the image information to each of the image input devices (200, 202, 204, 206, 208) via, for example, one or more optical conduits (such as fiber optic cables).

[0033] The controller 210 controls the operation of the stacked waveguide assembly (178) and the image input devices (200, 202, 204, 206, 208). In some embodiments, the controller 210 includes programming (e.g., instructions in a non-transient medium) that coordinates the timing and provision of image information to the waveguides (182, 184, 186, 188, 190), for example, according to any of the various schemes disclosed herein. In some embodiments, the controller may be a single integrated device or a distributed system connected by wired or wireless communication channels. The controller 210 may be part of the processing module (70 or 72) (FIG. 2) in some embodiments.

[0034] The waveguides (182, 184, 186, 188, 190) may be configured to propagate light within each respective waveguide by total internal reflection (TIR). Each of the waveguides (182, 184, 186, 188, 190) may be planar or have another shape (e.g., curved) with major top and bottom surfaces and edges extending between the major top and bottom surfaces. In the illustrated configuration, each of the waveguides (182, 184, 186, 188, 190) may include light extraction optical elements (282, 284, 286, 288, 290) configured to extract light out of the waveguide by redirecting the light to propagate within each respective waveguide and outputting image information from the waveguide to the eye 4. The extracted light may also be referred to as outcoupled light, and the light extraction optical elements may also be referred to as outcoupling optical elements. The extracted beam of light is output by the waveguide where the light propagating within the waveguide strikes the light redirecting element. The light extraction optical elements (282, 284, 286, 288, 290) may be, for example, reflective and / or diffractive optical features. In some embodiments, for ease of explanation and clarity of drawings, the light extraction optical elements (282, 284, 286, 288, 290) are shown disposed on the bottom major surface of the waveguide (182, 184, 186, 188, 190), but may be disposed on the top and / or bottom major surfaces and / or directly within the volume of the waveguide (182, 184, 186, 188, 190). In some embodiments, the light extraction optical elements (282, 284, 286, 288, 290) may be attached to a transparent substrate and formed within a layer of material that forms the waveguides (182, 184, 186, 188, 190). In some other embodiments, the waveguides (182, 184, 186, 188, 190) may be a monolithic piece of material and the light extraction optical elements (282, 284, 286, 288, 290) may be formed on and / or within that piece of material.

[0035] With continued reference to FIG. 6, as discussed herein, each waveguide (182, 184, 186, 188, 190) is configured to output light and form an image corresponding to a particular depth plane. For example, the waveguide (182) closest to the eye may be configured to deliver collimated light to the eye (4) as it is launched into such waveguide (182). The collimated light may represent an optical infinity focal plane. The next upper waveguide (184) may be configured to send collimated light that passes through a first lens (192; e.g., a negative lens) before it can reach the eye (4). Such a first lens (192) may be configured to create a slight convex wavefront curvature such that the eye / brain interprets the light originating from the next upper waveguide (184) as originating from a first focal plane closer inward from optical infinity toward the eye (4). Similarly, the third upper waveguide (186) passes its output light through both the first (192) and second (194) lenses before reaching the eye (4). The combined refractive powers of the first (192) and second (194) lenses may be configured to create another incremental amount of wavefront curvature such that the eye / brain interprets the light emerging from the third waveguide (186) as originating from a second focal plane that is closer inward from optical infinity towards the person than that emerging from the next upper waveguide (184).

[0036] The other waveguide layers (188, 190) and lenses (196, 198) are similarly configured, with the highest waveguide (190) in the stack sending its output through all of the lenses between it and the eye for an aggregate focal power that represents the focal plane closest to the person. To compensate the stack of lenses (198, 196, 194, 192) when viewing / interpreting light originating from the world (144) on the other side of the stacked waveguide assembly (178), a compensating lens layer (180) may be placed on top of the stack to compensate for the aggregate power of the lower lens stacks (198, 196, 194, 192). Such a configuration provides as many perceived focal planes as there are available waveguide / lens pairs. Both the light extraction optical elements of the waveguides and the focusing sides of the lenses may be static (i.e., non-dynamic or electro-active). In some alternative embodiments, either or both may be dynamic, using electro-active features.

[0037] Continuing with reference to FIG. 6, the light extraction optical elements (282, 284, 286, 288, 290) can be configured to both redirect light out of their respective waveguides and output this light with the appropriate amount of divergence or collimation for the particular depth plane associated with the waveguide. As a result, waveguides with different associated depth planes can have different configurations of light extraction optical elements (282, 284, 286, 288, 290), which output light with different amounts of divergence depending on the associated depth plane. In some embodiments, as discussed herein, the light extraction optical elements (282, 284, 286, 288, 290) can be solid or surface features that can be configured to output light at a particular angle. For example, the light extraction optical elements (282, 284, 286, 288, 290) can be volume holograms, surface holograms, and / or diffraction gratings. Light extraction optical elements such as diffraction gratings are described in U.S. Patent Application Serial No. 14 / 641,376, filed March 7, 2015, which is incorporated herein by reference in its entirety. In some embodiments, the features (198, 196, 194, 192) may not be lenses. Rather, they may simply be spacers (e.g., cladding layers and / or structures for forming air gaps).

[0038] In some embodiments, the light extraction optical elements (282, 284, 286, 288, 290) are diffractive features, i.e., "diffractive optical elements" (also referred to herein as "DOEs"), that form a diffraction pattern. Preferably, the DOEs have a relatively low diffraction efficiency, such that only a portion of the light of the beam is deflected toward the eye (4) at each intersection of the DOE, and the remainder continues traveling through the waveguide via total internal reflection. The light carrying the image information is thus split into several related exit beams that exit the waveguide at multiple locations, resulting in a very uniform exit emission pattern toward the eye (4) for this particular collimated beam bouncing within the waveguide.

[0039] In some embodiments, one or more DOEs may be switchable between an "on" state in which they actively diffract and an "off" state in which they do not significantly diffract. For example, a switchable DOE may comprise a layer of polymer dispersed liquid crystal, in which the microdroplets comprise a diffractive pattern within a host medium, and the refractive index of the microdroplets can be switched to substantially match that of the host material (in which case the pattern does not significantly diffract incident light), or the microdroplets can be switched to a refractive index that does not match that of the host medium (in which case the pattern actively diffracts incident light).

[0040] FIG. 7 shows an example of an exit beam output by a waveguide. Although one waveguide is shown, it should be understood that other waveguides in the waveguide assembly (178) may function similarly and that the waveguide assembly (178) includes multiple waveguides. Light (400) is launched into the waveguide (182) at the input edge (382) of the waveguide (182) and propagates within the waveguide (182) by TIR. At the point where the light (400) strikes the DOE (282), a portion of the light exits the waveguide as an exit beam (402). The exit beams (402) are shown as approximately parallel, but as discussed herein, they may be redirected to propagate to the eye (4) at an angle (e.g., forming a diverging exit beam) depending on the depth plane associated with the waveguide (182). It should be understood that a nearly collimated exit beam may refer to a waveguide with light-extracting optics that couples out light and forms an image that appears to be set on the depth plane at a long distance (e.g., optical infinity) from the eye (4). Other waveguides or other sets of light-extracting optics may output a more divergent exit beam pattern that requires the eye (4) to accommodate to closer distances and focus on the retina, and that would be interpreted by the brain as light from a distance closer to the eye (4) than optical infinity.

[0041] FIG. 8 diagrammatically illustrates an example of a stacked waveguide assembly, with each depth plane having three associated waveguides, each outputting a different color of light. A full color image may be formed at each depth plane by overlaying images in each of multiple component colors, for example, three or more component colors. In some embodiments, the component colors include red, green, and blue. In some other embodiments, other colors, including magenta and cyan, may be used in conjunction with or replace one or more of the red, green, or blue colors. In some embodiments, each waveguide may be configured to output a particular component color, such that each depth plane may have multiple waveguides associated with it. Each depth plane may have three waveguides associated with it, for example, one to output red light, a second to output green light, and a third to output blue light. It should be understood that references throughout this disclosure to a given color of light will be understood to encompass one or more wavelengths of light within the range of wavelengths of light that are perceived by a viewer to be of that given color. For example, red light may include one or more wavelengths of light in the range of about 620-780 nm, green light may include one or more wavelengths of light in the range of about 492-577 nm, and blue light may include one or more wavelengths of light in the range of about 435-493 nm.

[0042] Continuing with reference to FIG. 8, depth planes 14a-14f are shown. In the illustrated embodiment, each depth plane has three component color images associated with it: a first image of a first color G, a second image of a second color R, and a third image of a third color B. As a convention herein, the number following each of these letters indicates the diopters (1 / m), i.e., the distance of the depth plane from the viewer, and each box in the figure represents an individual component color image. In some embodiments, G is the color green, R is the color red, and B is the color blue.

[0043] In some arrangements, each component color image may be output by a different waveguide in the stack of waveguides. For example, each depth plane may have three component color images associated with it, a first waveguide for outputting a first color G, a second waveguide for outputting a second color R, and a third waveguide for outputting a third color B. In an arrangement in which waveguides are used to output component color images, each box in the diagram may be understood to represent an individual waveguide.

[0044] The waveguides associated with each depth plane are shown adjacent to one another in this schematic diagram for ease of illustration, but it should be understood that in a physical device the waveguides may all be arranged in a stack, with one waveguide per level. The different depth planes are indicated in the diagram by different numbers for diopters following the letters G, R, and B. However, as discussed herein, it would be beneficial to reduce the total number of waveguides in the waveguide stack.

[0045] To form a full color image using the component colors, it is believed that each component color should be output to form each image. As a result, when using a display in which a dedicated waveguide outputs a particular component color, it is believed that at least one waveguide should be available to output each component color to form an image at a particular depth plane. However, it has been found that a reduced number of waveguides can be utilized while still providing good color resolution.

[0046] In some embodiments, a stack of waveguides, also referred to herein as a stacked waveguide assembly, includes an unequal number of waveguides for outputting light of different colors or wavelength ranges. FIG. 9 illustrates an example of a waveguide stack, where different component colors have different numbers of associated waveguides. The waveguide stack can be part of a waveguide assembly 178 (FIG. 6). The waveguide stack includes a first plurality of waveguides, all designated by the letter G, a second plurality of waveguides, all designated by the letter R, and a third plurality of waveguides, all designated by the letter B. For clarity of illustration, depth planes 14a-14f are shown, however, fewer or more depth planes are also contemplated. In addition, while three component colors are illustrated, fewer (e.g., two) or more (e.g., four or more) component colors are also contemplated.

[0047] Without being limited by theory, it has been found that forming a full-color image preferably utilizes image information for each component color of the image, but this does not require each depth plane to have an associated waveguide for every component color. Rather, it is believed that the eye has different sensitivities when focusing using different wavelengths or colors of light. For example, it is believed that the eye is very sensitive to focusing using green light, and has a shorter depth of focus and depth of field than, for example, red light. As a result, the eye may distinguish a greater number of more closely spaced depth planes separately at wavelengths for green light compared to red light. It is believed that the eye is less sensitive to focusing using red light, and even less sensitive to focusing using blue light. As a result, the range of distances at which the eye perceives an object as being in focus also varies with color, and similarly, the number of perceived depth planes also varies with color. That is, a lower sensitivity to blue light indicates that an image presented using the light will be perceived to be in focus over a larger depth range than an image presented using red or green light, and an image presented using red light will be perceived to be in focus over a larger depth range than an image presented using green light. As a result, as illustrated in Figure 9, it is possible to present an image that is perceived to have a good range of colors without outputting light for each component color at every depth plane with an associated waveguide.

[0048] Continuing with reference to FIG. 9, the first plurality of waveguides G includes a total number of waveguides that exceeds the second plurality of waveguides R. It should be understood that this difference alone beneficially reduces the total number of waveguides in the waveguide assembly compared to the configuration shown in FIG. 8. In some embodiments, the third plurality of waveguides B may have a total number of waveguides that differs from the first or second plurality of waveguides G or R. In some embodiments, the total number of waveguides corresponding to each component color may be selected based on the eye's sensitivity to concentrating light corresponding to those colors and the associated depth of field, as discussed herein. For example, the total number of waveguides for outputting green light may exceed that for outputting red light, which in turn exceeds that for outputting blue light.

[0049] In some embodiments, the waveguides for each component color may correspond to equally spaced depth planes. For example, as shown, waveguide G may be associated with depth planes that are 0.50 diopter spaced, while waveguide R is associated with depth planes that are 0.75 diopter spaced, and waveguide B is associated with depth planes that are 1.0 diopter spaced. It should be understood that in some embodiments, the depth planes for the waveguides are not necessarily identical, and therefore the image information output by them may also be slightly different and correspond to the particular depth plane associated with that waveguide.

[0050] In some other embodiments, the image information output by the waveguides associated with neighboring depth planes may be the same except for color. For example, each of the waveguides represented by G, 0.25; R, 0.375; and B, 0.5 may be configured to output image information that is identical except for color. For example, the controller 210 (FIG. 6) may be programmed to provide image information that is otherwise identical to these different waveguides. In such embodiments, the spacing between the depth planes is selected such that the waveguides that output similar image information are separated from each other by a shorter distance than the waveguides that output image information for the presentation of different images on another depth plane.

[0051] It should be understood that different colors may be most clearly focused on depth planes that are not equally spaced. In some embodiments, a particular depth plane associated with a particular waveguide and color may be selected based on the eye's ability to clearly focus a particular depth plane using the color of light output by that waveguide. A particular depth plane associated with a waveguide for a particular color of light and its location on the z-axis may be chosen based on the eye's depth of focus or depth of field for that color. As a result, in some embodiments, at least some of the waveguides of the first, second, and third plurality of waveguides are associated with depth planes that are selected to provide images that are highly focused using the color of light output by the corresponding waveguide. Additionally, in some embodiments, the waveguides of the first plurality of waveguides may be configured to provide images for non-uniformly spaced depth planes. That is, the depth planes associated with the first plurality of waveguides may be non-uniformly spaced. Similarly, the depth planes associated with the second and / or third plurality of waveguides may also be non-uniformly spaced. As used herein with reference to schematic plots such as FIG. 9, the spacing between two depth planes is the difference in diopters between those depth planes.

[0052] FIG. 10 illustrates another example of a waveguide stack, where different component colors have different numbers of associated waveguides. Compared to the configuration of FIG. 9, the configuration of FIG. 10 includes fewer waveguides and provides image information for fewer depth planes. As a result, the spacing between depth planes for different component colors is also different from that illustrated in FIG. 9. In some embodiments, this further reduction in the number of waveguides may be accompanied by a reduction in resolution in the z-axis or an increase in depth of field. As previously mentioned, the illustrated waveguide stack may be part of a waveguide assembly 178 (FIG. 6).

[0053] Although discussed with reference to waveguides, it should be understood that Figures 9 and 10 and the associated description may generally be understood to relate to component color images associated with depth planes, as discussed above with respect to Figure 8. For example, Figures 9 and 10 may be understood to illustrate component color images associated with different depth planes, with or without a one-to-one correspondence between the waveguides and the depth planes and / or colors. For example, the different images may be output directly to a viewer by a display, or relayed by alternative optical means without the use of a waveguide stack such as that illustrated in Figure 6. For example, in some embodiments, a light modulator, such as a spatial light modulator for generating an image, may be coupled to a variable focus element, with the different component color images or subframes being positioned at different focal lengths (and different depth planes) by the variable focus element. Further details regarding such spatial light modulators and variable focus elements are disclosed in U.S. Patent Application Serial No. 14 / 641,376, filed March 7, 2015 (incorporated by reference as described herein).

[0054] In some embodiments, multiple optical elements, such as DOEs configured to output light for different component color images and / or images set on multiple depth planes, may reside on the shared waveguide. Individual optical elements on the shared waveguide may be switched to outcouple light and generate a component color image set on a particular depth plane, and then switched off when an image for a particular depth plane is not desired. Different optical elements may be switched on and off at different times to output images corresponding to multiple different depth planes and / or different wavelength ranges, all from the shared waveguide. Additional details regarding optical elements on the shared waveguide may be found in U.S. Patent Application Serial No. 14 / 641,376, filed March 7, 2015. In certain embodiments, the switchable optical elements may allow the waveguide assembly (178) to include only a single waveguide (e.g., waveguide 182), omitting other waveguides (e.g., waveguides 184, 186, 180, and 190).

[0055] In some embodiments, the light extraction optical elements may be selective to a particular wavelength of light, and a set of multiple light extraction optical elements, each set selective to a different wavelength of light, may form part of the waveguide. In some embodiments, these optical elements are not switchable between on and off states. Light to form different component color images may be outcoupled at different times by temporally sequencing the image information provided to the waveguide. In some embodiments, these optical elements may be diffractive optical elements.

[0056] As previously mentioned, the number of waveguides in a waveguide stack (or the total number of component color images utilized across multiple depth planes) may be advantageously reduced by taking advantage of the eye's sensitivity to focus particular wavelengths or colors of light. In some other embodiments, this reduction in the number of waveguides or component color images (e.g., as illustrated in Figures 9 and 10) may also be facilitated by manipulating other properties of the display system in addition to, or as an alternative to, considerations regarding the eye's non-uniform sensitivity to focusing using different colors and the associated different depths of field for the eye.

[0057] In some embodiments, the number of depth planes perceived by the eye can be manipulated by increasing the eye's depth of field or depth of focus. For example, the exit pupil for one or more waveguides can be reduced to increase the depth of focus. It should be understood that light propagates through the display system (1000) out of the waveguide and into the eye (4). This light passes through an exit pupil as it propagates from the light source to the eye (4). In some embodiments, the exit pupil can be set by an aperture placed in the optical path between the output surface of the waveguide and the display for generating the image information to be injected into the waveguide. In some embodiments, the aperture can have a fixed size.

[0058] In some other embodiments, the aperture may have a dynamically variable size that may be set by the controller (210) (FIG. 6). For example, the variable aperture may comprise an electronically adjustable iris, a spatial light modulator, a shutter with concentric zones (e.g., liquid crystal shutter), etc. In some other embodiments, multiple light sources with different characteristics (e.g., different numerical apertures) may be configured to couple light into the same waveguide. By switching or alternating the illuminated light sources, the numerical aperture and beam diameter at the waveguide input area (which then propagates through the waveguide) may be changed. In some embodiments using a fiber scanning display (FSD) as the image source, the FSD may comprise multiple cores with different mode field diameters and numerical aperture values. By varying the illuminated cores, the beam diameter and therefore the size of the light beam coupled into and propagating through the waveguide may be changed. Examples of light sources having different characteristics are disclosed in U.S. Provisional Application No. 62 / 156,809, filed May 4, 2015, the entire disclosure of which is incorporated herein by reference.

[0059] In some embodiments, reducing the exit pupil for the waveguides corresponding to all the component colors can generally reduce the number of perceived depth planes. For example, referring to FIG. 8 where six depth planes associated with the waveguides are illustrated, it may be possible to reduce the total number of associated depth planes by reducing the exit pupil size. Thus, even if waveguides for all the component colors are provided for all the associated depth planes, a reduction in the total number of waveguides may be achieved. However, as discussed herein, the number of depth planes corresponding to some colors may be reduced because the eye may distinguish fewer depth planes for some colors than others. Thus, in some exemplary embodiments, it may be possible to move from the configuration of FIG. 9 to the further reduced configuration of FIG. 10 without a significant reduction in image quality or depth perception by reducing the depth of field for the viewer's eyes. In some embodiments, the exit pupil may differ depending on the color or wavelength of light output by the waveguide. For example, a waveguide for outputting some colors of light may have a smaller exit pupil than a waveguide outputting other colors of light.

[0060] It should be appreciated that the depth of field of a viewer's eye may also be altered by reducing the size of the eye's pupil. In some embodiments, this may be accomplished, for example, by increasing the brightness of the image output by the waveguide, thereby causing the viewer's pupil to constrict.

[0061] In some embodiments, the waveguide may be configured to output a light beam smaller than the size of the viewer's pupil (FIG. 7). An exit beam smaller than the viewer's pupil can provide a similar effect on depth of field as using a small exit pupil or shrinking the size of the viewer's pupil. In some embodiments, the width of the exit beam may vary and differ depending on the color or wavelength of light output by the waveguide. For example, a waveguide for outputting some colors of light (e.g., colors that the eye is more sensitive to focus on) may have a smaller exit beam width than a waveguide outputting other colors of light (e.g., colors that the eye is less sensitive to focus on).

[0062] With small beam sizes, it is possible that multiple similar beams may enter the eye, which may cause undesirable visual artifacts. In some embodiments, the light extraction optics of the waveguide may be configured to direct only a single beam of light to the eye at a time, for example, by appropriately spacing the locations of the exit beam. In some embodiments, the width of the exit beam may be set by injecting a collimated beam of light into the waveguide, the collimated beam having the desired width for the exit beam. Because the injected light beam is collimated, the width of the exit beam may be expected to remain substantially similar throughout the output area of ​​the waveguide. In some embodiments, the light extraction optics may be configured to interact with the injected light beam and provide an exit beam of a desired size. For example, the light extraction optics may be configured to modify (e.g., reduce) the width of the exit beam compared to the width of the injected light beam. For example, two light extraction optical elements in series (one to converge and one to re-collimate the beam) can be used to reduce the size of the output beam, or two light extraction optical elements can include one to diverge the beam and one to re-collimate (increase width). It should be understood that reducing the width will increase the depth of field. Similarly, spacing of the redirecting elements can also be used to space the output beams so that only one beam passes through the pupil of the eye. In addition, the spacing between the output beams can be controlled by controlling the frequency at which the collimated beam hits the light redirecting element as it propagates through the waveguide by TIR.

[0063] In some embodiments, the number of times a collimated beam of light hits a light redirecting element can be selected by choosing the angle at which the collimated beam is launched into the waveguide and / or by using a light turning feature that receives the launched light and directs the light through the waveguide at a desired angle. For example, a collimated beam that is launched into a waveguide and hits a major surface of the waveguide at a shallow angle (relative to the major surface) can be expected to propagate a longer distance before hitting the major surface again compared to light that is launched into the waveguide and hits the major surface at a smaller angle measured relative to the normal to the major surface. In some systems, the angle at which light is launched into the waveguide determines the pixel it represents. In such systems, an active optical element within the waveguide (e.g., a switchable DOE) can be used to deflect the beam back to its intended angle to represent the correct pixel in space.

[0064] It should be understood that the location and spacing of the exit beams can be adjusted using various other schemes. For example, a waveguide incorporating a volume phase hologram can be highly angle selective. Such a hologram can be configured to suppress outcoupling of some beams in some locations while selectively outcoupling other beams in other locations. In some other embodiments of a waveguide with switchable DOEs for outcoupling, the DOEs can be divided into locally addressable cells or regions such that some regions of the DOE can be turned off (preventing beams from outcoupling from that region) while other regions are turned on to outcouple one or more beams. In some embodiments, a scanning beam display such as a fiber scanning display can be coupled into a locally addressable waveguide and synchronized so that different DOE cells are activated. Different DOE cells can be activated as a function of the beam angle injected into the waveguide. In some other embodiments, a shutter array (or SLM) can be placed between the waveguide and the eye to selectively attenuate some beams while allowing others to pass.

[0065] In some embodiments, the output beam width is about 7 mm or less, about 5 mm or less, about 3 mm or less, 2 mm or less, 1 mm or less, or a range between any combination of these values. In some embodiments, the width of the output beam can be varied using a variable aperture in the optical path of the light between the light source and the input of the light into the waveguide. For example, a variable aperture can be provided in the image input device (200, 202, 204, 206, 208) (FIG. 6). For example, a variable aperture can be provided at the output surface (300, 302, 304, 306, 308) of the image input device (200, 202, 204, 206, 208).

[0066] The various approaches disclosed herein for increasing the depth of field may be advantageously applied to one or more multiple waveguides for outputting one or more specific colors, respectively, allowing the number of waveguides for those colors to be reduced. In some embodiments, these approaches for increasing the depth of field may be applied in conjunction with the difference in eye sensitivity to concentrate light of different colors, thereby further allowing the total number of waveguides to be reduced. For example, FIG. 9 may be viewed as a waveguide stack that utilizes the difference in eye sensitivity to different colors, and FIG. 10 may be viewed as a waveguide stack that utilizes an approach for increasing the depth of field with a reduced aperture size and also the difference in eye sensitivity to different colors.

[0067] In some embodiments, the approaches disclosed herein for increasing the depth of field may be applied to reduce the number of waveguides (or component color images) associated with a selected color, as desired. For example, a green light emitter for providing green colored image information to a waveguide may cost more than a red light emitter. The sensitivity of the eye to focus with green light would be expected to dictate that the number of waveguides for outputting green light would be greatest. However, by applying one or more of the approaches disclosed herein for increasing the depth of field to the waveguides for the green color (e.g., reducing the aperture size), the total number of those waveguides may be reduced (e.g., to a number less than the number of red waveguides), thereby reducing the number of green light emitters, which in turn reduces the cost of a system using these components. As a result, in some embodiments, the waveguides in the various multiple waveguides for outputting different colors may have different associated exit pupil sizes and / or different exit beam sizes. For example, the total number of waveguides in a particular plurality of waveguides may be directly related to the exit pupil size and / or beam size associated with the waveguides of the particular plurality of waveguides, in some embodiments, waveguides having a relatively larger associated exit pupil size have a greater total number than waveguides having a relatively smaller exit pupil size.

[0068] It should be appreciated that reducing the number of depth planes represented by the waveguides may cause images output by some waveguides to be out of focus compared to images of other colors for the same depth plane. In conjunction with or as an alternative to utilizing different sensitivities of the eye to focus different colors and / or manipulate the depth of field for some or all of the waveguides, the sensitivity of the eye to out of focus may also be reduced in some embodiments. Sensitivity may be reduced by intentionally introducing aberrations, such as blur, to images output by some or all of the waveguides in the waveguide stack. For example, blur may be introduced for one or more of the multiple waveguides having a relatively low total number. In some embodiments, for a display system using red-green-blue component colors, blur may be introduced in images output by a waveguide associated with a blue color that may have a lower total number of waveguides than the waveguides associated with the red or green colors. In some embodiments, the amount of blur introduced may vary with the color light output by the waveguide. For example, the amount of blur introduced may be directly related to the number of waveguides associated with a particular color. The fewer the number of waveguides for a particular color, the greater the amount of blur introduced into the images output by those waveguides. In some embodiments, blur may be introduced for each component color image, with the amount of blur varying with color. For example, component color images of colors to which the human eye is very sensitive may have a higher level of introduced blur than component color images of colors to which the eye is relatively less sensitive.

[0069] In some embodiments, blurring may be introduced by one or more optical elements and / or image processing electronics. For example, if image information is provided to the waveguide by in-coupling a beam of light into the waveguide (such as with an FSD), the focus of the beam to be coupled into the waveguide may be adjusted away from collimation. A collimated beam is coupled out of the waveguide to create a smallest spot for the beam to be viewed. A non-collimated beam injected into the waveguide would still generate pixels in the correct location, but the spot size would be larger and would add blurring. In some embodiments, optical elements are used (e.g., positioned at the light input surface of the waveguide) prior to in-coupling the light beam to adjust the focus of the light beam so as to adjust away from collimation, thereby introducing blurring into the image input devices (200, 202, 204, 206, 208) (FIG. 6) and / or image information received from the image input devices (200, 202, 204, 206, 208). In some other embodiments, the optical element for introducing blur may be located in other parts of the optical path between the display for generating the image and the output of the image information from the waveguide to the viewer's eye.

[0070] In addition to or as an alternative to the use of optical elements to introduce blur, blur may be introduced electronically, for example, in the image processing in processing modules (70) or (72) (FIG. 2). The electronic introduction of blur may allow the blur to be varied on a dynamic basis, for example, depending on the image content. For example, some images may be determined to suffer less loss of perceived sharpness, e.g., images with less detail or information. For such images, the amount of blur may be increased in some embodiments without significantly decreasing image quality.

[0071] In some embodiments, the sensitivity to defocus may be reduced by decreasing the xy resolution (e.g., pixel resolution) of the image output by the waveguide. For example, this may be effectively achieved by increasing the size of the individual pixels. In some embodiments, the pixel size may be increased by increasing the size of the optical fiber that provides the image information to the waveguide. For example, a scanning fiber display may include multiple fiber cores, each of which has a different mode field diameter and therefore a different effective spot size or pixel size. In some embodiments, the size of the pixel may vary depending on the color of the light output by the waveguides of the waveguide stack. For example, the fewer the number of waveguides for a particular color, the larger the size of the individual pixels for the images output by those waveguides (and the lower the xy resolution). In some embodiments, the difference in pixel size may be established using differently sized sub-pixels in the spatial light modulator (SLM) that generates the image. For example, blue sub-pixels may be physically larger than green sub-pixels, and / or fewer in number and / or more sparsely distributed across the SLM. Alternatively, or in addition, it should be appreciated that pixel resolution may be dynamically altered by the image processing module in some embodiments. For example, as with blurring, some images may be determined to suffer less from perceived loss of pixel resolution, e.g., images with less detail or information. As a result, pixel resolution may be reduced in some embodiments without a significant decrease in image quality.

[0072] In some embodiments, the number and distribution of depth planes and / or depths of field may be dynamically varied based on the pupil size of the viewer's eye. FIG. 11 illustrates an example of a display system configured to monitor pupil size. A camera (500) (e.g., a digital camera) captures an image of the eye (4) and determines the size of the pupil of the eye 4. In some embodiments, the camera (500) may be mounted on a frame (64) (FIG. 2) and may be in electrical communication with a processing module (70) and / or (72) that may process image information from the camera (500) and determine the pupil diameter of the eye of the user (60). In some embodiments, one camera (500) may be utilized for each eye and may determine the pupil size of each eye separately, thereby allowing the presentation of image information to each eye to be dynamically adjusted for that eye. In some other embodiments, the pupil diameter of only one eye (4) (e.g., using only a single camera (500) per pair of eyes) is determined and assumed to be similar for both eyes of the viewer (60).

[0073] As discussed herein, the depth of field may vary inversely with the viewer's pupil size. As a result, as the size of the viewer's eye pupil decreases, the depth of field increases, whereby one plane that is indistinguishable because its location exceeds the eye's focal depth may become distinguishable and appear more in focus with a reduced pupil size and a comparable increase in depth of field. Similarly, the number of spaced depth planes used to present different images to the viewer may also decrease with a reduced pupil size. For example, referring again to FIG. 8, a viewer may not be able to clearly perceive the details of both depth planes 14a and 14b at one pupil size without moving away from one depth plane and adjusting the eye's accommodation to the other depth plane. However, these two depth planes may be well focused simultaneously to the user at another pupil size without changing accommodation.

[0074] In some embodiments, the display system may vary the number of waveguides that receive image information based on a pupil size determination or in response to receiving an electrical signal indicative of a particular pupil size. For example, if the user's eye is unable to distinguish between two depth planes associated with two waveguides, the controller (210) (FIG. 6) may be configured or programmed to cease providing image information to one of these waveguides. Advantageously, this may reduce the processing burden on the system, thereby increasing the responsiveness of the system. In embodiments in which the DOE for a waveguide is switchable between an on state and an off state, the DOE may be switched to an off state when the waveguide receives image information.

[0075] As discussed herein, in some embodiments, it may be desirable to have the exit beam satisfy a condition that it has a diameter less than the diameter of the viewer's eye. However, meeting this condition may be difficult in light of the variability in the size of the viewer's pupil. In some embodiments, this condition is met over a wide range of pupil sizes by varying the size of the exit beam in response to a determination of the size of the viewer's pupil. For example, as the pupil size decreases, the size of the exit beam may also decrease. In some embodiments, the exit beam size may be varied using a variable aperture, as discussed herein.

[0076] Various exemplary embodiments of the present invention are described herein. These examples are referred to in a non-limiting sense. They are provided to illustrate the broader applicable aspects of the invention. Various modifications may be made to the invention described, and equivalents may be substituted without departing from the true spirit and scope of the invention.

[0077] For example, although individual waveguides are discussed in some embodiments as outputting one particular color of light, in some other embodiments, at least some of the waveguides may be configured to output two or more colors of light. For example, in some embodiments, a volumetric material having light extraction optical elements therein (e.g., a volume hologram) may be utilized to selectively redirect different wavelengths of light out of the waveguide. The use of waveguides to output different colors of light may allow fewer current waveguides to be used within the waveguide stack.

[0078] It should be appreciated that one goal of the spacing between depth planes is to provide a spacing at which objects appear to transition smoothly from near to far without a noticeable jump in perceived focus. A "depth blending" approach may be used to increase the smoothness of the visual transition between closer and farther parts of an object. For example, given a finite number of depth planes as disclosed herein, it is possible that some points on an object may be between two depth planes and image information about these points may be available. In some embodiments, these points may have their brightness distributed between two neighboring planes. For example, a point halfway between two depth planes in diopter space may have half its brightness in the first of the two planes and half in the second of the two planes. By distributing the brightness of points that are between immediately neighboring depth planes among their depth planes, the smoothness of the transition between depth planes may be increased.

[0079] In some embodiments, multiple waveguides in a waveguide stack may have the same associated depth plane. For example, multiple waveguides configured to output the same color may be set to the same depth plane. This can provide the advantage of forming a tiled image and providing an extended field of view at that depth plane.

[0080] As discussed herein, in some embodiments, the display may present the same number of similarly spaced depth planes with the same progression of focal distance to the left and right eyes of the viewer. In some other embodiments, one or more of the total number, separation between them, and focal distance associated with the depth planes presented to the left and right eyes of the viewer may be different. For example, a greater number of depth planes may be presented to the right eye than the left eye or vice versa, with different progression of focal distance between the right and left eye depth planes. Without being limited by theory, it is believed that stereoscopic vision may mask a reduced number of depth planes in one eye. The difference in depth plane number, separation, and / or focal distance may be present for all component colors or may be limited to one or more specific component colors, e.g., green. As an example, in some embodiments, one eye may be presented with a depth plane as shown in FIG. 9, while another eye may be presented with a depth plane as shown in FIG. 10.

[0081] Although advantageously illustrated as a wearable system, the waveguides and associated structures and modules disclosed herein may be applied to form non-wearable displays. For example, rather than being housed within a wearable frame (64) (FIG. 2), the display (62) may be attached to a stand, mount, or other structure that supports the display (62) and allows the display (62) to provide an image to the viewer (60) without being worn by the viewer (60).

[0082] In addition, many modifications may be made to adapt a particular situation, material, composition of matter, process, process acts, or steps to the objective, spirit, or scope of the present invention. Moreover, as will be understood by those skilled in the art, each of the individual variations described and illustrated herein has individual components and features that can be readily separated from or combined with the features of any of the other several embodiments without departing from the scope or spirit of the present invention. All such modifications are intended to be within the scope of the claims associated with this disclosure.

[0083] The present invention includes methods that may be performed using the subject devices. The methods may include the act of providing such a suitable device. Such providing may be performed by an end user. In other words, the act of "providing" merely requires the end user to obtain, access, access, position, configure, activate, power on, or otherwise act to provide the requisite device in the subject method. The methods described herein may be carried out in any order of the described events that is logically possible, as well as the described order of events.

[0084] Exemplary aspects of the invention have been described above, along with details regarding material selection and manufacturing. As for other details of the invention, these will be understood in conjunction with the patents and publications referenced above, and will generally be known or understood by those skilled in the art. The same may be true with respect to method-based aspects of the invention in terms of additional acts as typically or logically adopted.

[0085] In addition, although the present invention has been described with reference to several examples that optionally incorporate various features, the present invention is not limited to those described and indicated as being considered with respect to each variation of the present invention. Various modifications may be made to the invention described, and equivalents (whether described herein or not included for some simplicity) may be substituted without departing from the true spirit and scope of the present invention. In addition, when a range of values ​​is provided, it is to be understood that all intervening values ​​between the upper and lower limits of that range, and any other stated or intervening values ​​within that stated range, are encompassed within the present invention.

[0086] It is also contemplated that any optional feature of the described inventive variation may be described and claimed independently or in combination with any one or more of the features described herein. Reference to a singular item includes the possibility that the same item is present in plural. More specifically, as used herein and in the claims associated herewith, the singular forms "a," "an," "said," and "the" include plural referents unless otherwise specified. In other words, the use of articles allows for "at least one" of the subject item in the above description as well as in the claims associated herewith. Furthermore, it is noted that such claims may be drafted to exclude any optional element. Thus, this statement is intended to serve as a predicate for the use of exclusive terminology such as "solely," "only," and the like, or the use of "negative" limitations in connection with the recitation of claim elements.

[0087] Without using such exclusive terms, the term "comprising" in the claims associated with this disclosure shall be construed as allowing for the inclusion of any additional elements, regardless of whether a given number of elements are recited in such claims or whether the addition of features can be considered as a transformation of the nature of the elements recited in such claims. Except as specifically defined herein, all technical and scientific terms used herein are to be given the broadest possible commonly understood meaning while maintaining the validity of the claims.

[0088] The scope of the present invention is not to be limited by the illustrative and / or subject specification provided, but rather is to be limited only by the scope of the language of the claims associated with this disclosure.

Claims

1. 1. A display system, comprising: a display configured to output a plurality of component color images, the plurality of component color images corresponding to a plurality of depth planes; Based on the content of an image to be displayed in at least one of the plurality of depth planes, Blurry and Change in pixel size and into the plurality of component color images; the display comprises a stack of waveguides, the stack of waveguides comprising: a first plurality of waveguides, each configured to output light in a first wavelength range to generate a first component color image of the plurality of component color images, the first component color image being formed from a first component color; a second plurality of waveguides, each configured to output light in a second wavelength range to generate a second component color image of the plurality of component color images, the second component color image being formed from a second component color different from the first component color; wherein the first plurality of waveguides totals a different number of waveguides than the second plurality of waveguides.

2. The display system of claim 1 , wherein the controller is configured to vary an amount of blur introduced into the plurality of component color images based on a component color of the plurality of component color images.

3. The display system of claim 2 , wherein the controller is configured to provide a relatively large amount of blur to the plurality of component color images for a component color having a relatively large depth of field.

4. 10. The display system of claim 1, wherein the controller is configured to provide a relatively large pixel size for the plurality of component color images for a component color having a relatively large depth of field.

5. 3. The display system of claim 2, wherein the plurality of component color images comprises a red image, a green image, and a blue image, and the controller is configured to provide a greatest amount of blur to the blue image.

6. The display system of claim 5 , wherein the controller is configured to not provide any blurring to the green image.

7. The display system of claim 1 , wherein the controller is configured to vary pixel size by providing different component color images with different pixel resolutions.

8. 10. The display system of claim 1, wherein the display is a scanning fiber display comprising a plurality of optical fibers configured to output light of different component colors to form associated component color images.

9. 9. The display system of claim 8, wherein different optical fibers of the scanning fiber display are configured to output light of different associated component colors, the different optical fibers for the different component colors having different sizes to output light with different effective spot sizes corresponding to different pixel sizes.

10. 10. The display system of claim 9, wherein the smallest of the effective spot sizes corresponds to an optical fiber for outputting green light.

11. 10. The display system of claim 1, wherein the display is configured to selectively output light to form the plurality of component color images with different amounts of wavefront divergence corresponding to different depth planes.

12. The display system of claim 1 , wherein the display is a head-mounted display.

13. 13. The display system of claim 12, wherein the display comprises a waveguide configured to in-couple image information and out-couple the image information to a viewer's eye, the waveguide being one of a plurality of waveguides forming a stack of the waveguides.

14. 14. The display system of claim 13, wherein at least some of the waveguides in the stack of waveguides comprise light extraction optical elements configured to output light with a different amount of wavefront divergence than other waveguides in the stack of waveguides, the different amounts of wavefront divergence corresponding to different depth planes.

15. The display system of claim 14 , wherein the light extraction optical element comprises a diffraction grating.

16. 2. The display system of claim 1, wherein the stack of waveguides further includes a third plurality of waveguides, each configured to display a third component color image of the plurality of component color images by outputting light in a third wavelength range, the third component color image being formed from a third component color different from the first component color or the second component color, and the third plurality of waveguides totaling a number of waveguides different from the first plurality of waveguides or the second plurality of waveguides.

Images