Dynamic Fresnel Projector

Abstract

To provide a system for generating a multi-depth image sequence comprising a modulation array.SOLUTION: A modulation array comprising a plurality of light modulators may shift light incident upon the modulators by a number of degrees. The plurality of light modulators may shift light in concert according to a modulation shift pattern. The modulation shift pattern can be configured to focus incident light to a voxel or to form a 3-D image. One or more modulation shift patterns can be changed or cycled through to raster one or more image objects in one or more image depth planes.SELECTED DRAWING: None

Description

Background Art

[0001] (Citation of Related Applications) This application claims priority from U.S. Provisional Patent Application No. 62 / 276,099, filed on January 7, 2016, entitled "DYNAMIC FRESNEL PROJECTOR", Attorney Docket No. ML.30033.00. The entire content of the above patent application is hereby incorporated by reference into this specification.

[0002] Modern computing and display technologies have facilitated the development of systems for so-called "virtual reality" or "augmented reality" experiences, where digitally reproduced images or portions thereof are presented to a user and can be perceived as reality. Virtual reality ("VR") scenarios typically involve the presentation of digital or virtual image information without transparency to other actual real-world visual inputs. Augmented reality ("AR") scenarios typically involve the presentation of digital or virtual image information as an extension of the visualization of the actual world around the user.

[0003] For example, referring to FIG. 1A, an augmented reality scene 100 is depicted, and a user of an AR technology device views a real-world park-like setting 102 characterized by people, trees, buildings in the background, and a concrete platform 104. In addition to these items, the user of the AR technology also perceives that a robot image 106 standing on the real-world platform 104 and a flying comic-like avatar character 108 "appear to be visible", but these elements do not exist in the real world. In conclusion, the human visual perception system is very complex, and it is difficult to generate VR or AR technologies that facilitate a comfortable, natural, and rich presentation of virtual image elements among other virtual or real-world image elements.

[0004] Referring to FIG. 1B, generally, a stereoscopic wearable glasses 112 type configuration is being developed that features two displays (e.g., 114a, 114b) configured to display images with the presentation of slightly different elements such that a three-dimensional perspective drawing is perceived by the human visual system. Such a configuration has been found to be uncomfortable for many users due to a mismatch between the vergence movement and the accommodation that must be overcome to perceive the image in three dimensions. In fact, some users cannot tolerate the stereoscopic configuration. Additionally, in some imaging devices, one or more imaging modules 116 for processing the images (e.g., for displays 114a, 114b) can be integrated into the device. To keep the device "wearable", the imaging module 116 typically has to be small, and thus there can be a trade-off between image quality and device size. For example, a larger imaging module can simultaneously generate a higher quality image output on the displays (114a, 114b), making the glasses 112 difficult and / or heavy to handle.

[0005] Referring to FIG. 1C, a simplified cross-sectional view of a human eye 118 is depicted that features the cornea 120, iris 122, lens, i.e., the "crystalline lens" 124, sclera 126, choroid layer 128, macula 130, retina 132, and the optic nerve pathway 134 to the brain. The macula is the center of the retina and is used to view medium detail. At the center of the macula is the "fovea", which is used to view the finest detail. The fovea contains more visual cells (about 120 cones per degree of vision) than any other part of the retina.

[0006] The human visual system is not a passive sensor type of system. It is configured to actively scan the environment. In a manner somewhat similar to the scanning of an image using a flatbed scanner or the use of fingers to read Braille from paper, the eye's photoreceptor cells do not respond uniformly to a constant state of stimulation but emit signals in response to changes in stimulation. Therefore, movement is required to present photoreceptor cell information to the brain (like the movement of a linear scanner array across a piece of paper in a flatbed scanner or the movement of fingers across a word of Braille imprinted on paper).

[0007] In fact, experiments using substances such as cobra venom, which are utilized to paralyze the eye muscles, have shown that a human subject will experience blindness if positioned while their eyes are open and viewing a static scene with the eyes paralyzed by the venom. In other words, in the absence of changes in stimulation, photoreceptor cells do not provide input to the brain and blindness is experienced. This is thought to be at least one reason why normal human eyes are observed to move back and forth or make tiny movements in a horizontal movement called "microsaccades".

[0008] As mentioned above, the fovea of the retina contains the highest density of photoreceptor cells, and while humans typically have the understanding that they have high-resolution visualization capabilities throughout their visual field, generally humans actually only have a small high-resolution center that mechanically sweeps around repeatedly, along with a persistent memory of the high-resolution information most recently captured at the fovea.

[0009] In several similar manners, an eye focus distance control mechanism (the ciliary muscle operably coupled to the lens in such a manner that ciliary muscle relaxation causes tension in the ciliary zonular fibers, flattening the lens for a more distant focus distance, and ciliary muscle contraction causes relaxation of the ciliary zonular fibers, enabling the lens to assume a more rounded geometry for a closer focus distance) reciprocally micro-moves by about 1 / 4 to 1 / 2 diopter in order to cyclically induce a small amount of what is called "dioptric blur" both proximal and distal to the target focus distance. This is utilized by the brain's accommodation control circuitry as a cyclic negative feedback that helps to constantly correct the line of sight and keep the retinal image of a stationary object substantially in focus.

[0010] The brain's visualization center obtains useful perceptual information from the movements of both eyes and their components relative to each other. The binocular divergence movement of both eyes relative to each other (i.e., the rolling of the pupils towards or away from each other to converge the lines of sight of the eyes and fix on an object) is closely associated with the focusing (or "accommodation") of the lenses of the eyes. Under normal conditions, changing the focus of the lenses of the eyes, i.e., accommodating the eyes and focusing on objects at different distances, will automatically cause a coordinated change in the binocular divergence movement for the same distance under a relationship known as the "accommodation-binocular divergence movement reflex action". Similarly, a change in the binocular divergence movement will, under normal conditions, also induce a coordinated change in accommodation. Acting against this reflex action is known to cause eye fatigue, headaches, or other forms of discomfort to the user (as in most of the conventional stereoscopic AR or VR configurations).

[0011] The movement of the head that stores the eyes also has an important impact on the visualization of objects. Humans move their heads to visualize the world around them. In many cases, they are in a state of constantly repositioning and redirecting their heads towards the object of interest. Furthermore, most people prefer to move their heads when their line of sight needs to move more than about 20 degrees from the center to focus on a particular object (i.e., people typically do not prefer to look at things "from the corner of their eyes"). Humans typically scan or move their heads in relation to sound to improve audio signal capture and utilize the geometry of the ears relative to the head. The human visual system obtains excellent depth cues from what is called "head movement parallax", which is related to the relative movement of objects at different distances as a function of both head movement and the binocular disparity movement distance of the eyes (i.e., when a person moves their head laterally and maintains a fixed position relative to an object, items farther away from that object will move in the same direction as the head, and items in front of that object will move in the opposite direction of the head movement. These are very prominent cues to the spatial location of objects in the environment for a person and are perhaps as good as stereopsis). Head movement is, of course, also used to look around objects.

[0012] Furthermore, the movements of the head and eyes are coordinated by what is called the "vestibulo-ocular reflex", which stabilizes the image information on the retina during head rotation and thus keeps the object image information near the center of the retina. In response to head rotation, the eyes rotate reflexively and proportionally in the opposite direction to maintain a stable fixation on the object. As a result of this compensatory relationship, many humans can read a book while oscillating their head back and forth (interestingly, the same does not generally hold when the book is panned back and forth at the same speed while the head remains mostly stationary; that is, the person is likely unable to read the pages of the book. The vestibulo-ocular reflex is one of the head and eye movement coordinations and is generally not developed for hand movement). This paradigm can be important for augmented reality systems because the user's head movement can be relatively directly associated with eye movement and the system will preferably cooperate with this relationship immediately.

[0013] The systems and techniques described herein are coupled with the typical human visual configuration and are configured to address these challenges.

SUMMARY OF THE INVENTION

MEANS FOR SOLVING THE PROBLEM

[0014] In one embodiment, a system for generating a multi-depth image sequence includes one or more light sources and one or more modulation arrays having a plurality of light modulators. The one or more light sources are configured to project light onto at least some of the plurality of light modulators. The plurality of light modulators are configured to generate a multi-depth image sequence by focusing light onto voxels and rasterizing the plurality of voxels at different depths on one or more image depth planes.

[0015] In one or more embodiments, the plurality of light modulators are configured to shift the phase of light by several degrees. The plurality of light modulators may also be configured to shift the phase of light according to a modulation shift pattern.

[0016] In one or more embodiments, the one or more light sources include a green light source, a red light source, and a blue light source. The one or more modulation arrays may include a first modulation array configured to receive red light, a second modulation array configured to receive green light, and a third modulation array configured to receive blue light. The one or more modulation arrays may include a segmented modulation array, and the segmented modulation array includes a first area configured to receive red light, a second area configured to receive green light, and a third area configured to receive blue light.

[0017] In one or more embodiments, the system also includes a diffractive optical assembly having one or more waveguide layers, and the one or more waveguide layers correspond to different image depth planes. The system may also include a processor configured to direct the formation of a Fresnel pattern on a plurality of light modulators. The processor may be configured to direct the sequential formation of a plurality of Fresnel patterns on the plurality of light modulators. At least two of the plurality of Fresnel patterns may be configured to focus light on respective different voxels at different image depths.

[0018] In another embodiment, a system for generating a multi-depth image sequence includes a light source and a modulation array including a plurality of light modulators. The light source is configured to project light onto at least some of the plurality of light modulators. The plurality of light modulators are configured to focus respective portions of the light to generate a multi-depth image.

[0019] In one or more embodiments, the plurality of light modulators are also configured to generate a series of multi-depth images. The plurality of light modulators may also be configured to shift the phase of the light by several degrees. The plurality of light modulators may also be configured to shift the phase of the light according to a modulation shift pattern.

[0020] In one or more embodiments, the light source is a white light source. The modulation array can be a segmented modulation array including a first area configured to receive red light, a second area configured to receive green light, and a third area configured to receive blue light.

[0021] In one or more embodiments, the system also includes a diffractive optical assembly including one or more waveguide layers, and the one or more waveguide layers correspond to different image depth planes. The system can also include a processor configured to instruct the plurality of light modulators to form computer-generated holograms. The processor can be configured to instruct the plurality of light modulators to successively form a plurality of computer-generated holograms. The modulation array can be an 8-bit LCOS.

[0022] Other additional objects, features, and advantages of the present invention are described in the detailed description, drawings, and claims. For example, the present application provides the following items. (Item 1) A system for generating a multi-depth image sequence, the system comprising: one or more light sources; one or more modulation arrays including a plurality of light modulators; and the one or more light sources are configured to project light onto at least some of the plurality of light modulators, and the plurality of light modulators are configured to focus the light onto voxels and generate a multi-depth image sequence by rasterizing a plurality of voxels at different image depths. A system configured to perform. (Item 2) The system according to claim 1, wherein the plurality of light modulators are configured to shift the phase of the light by several degrees. (Item 3) The system according to item 1, wherein the plurality of optical modulators are configured to shift the phase of the light according to a modulation shift pattern. (Item 4) The system according to item 1, wherein the one or more light sources include a green light source, a red light source, and a blue light source. (Item 5) The system according to item 4, wherein the one or more modulation arrays include a first modulation array configured to receive red light, a second modulation array configured to receive green light, and a third modulation array configured to receive blue light. (Item 6) The system according to item 4, wherein the one or more modulation arrays include a segmented modulation array, and the segmented modulation array includes a first area configured to receive red light, a second area configured to receive green light, and a third area configured to receive blue light. (Item 7) The system according to item 1, further comprising a diffractive optical assembly including one or more waveguide layers, wherein the one or more waveguide layers correspond to different image depth planes. (Item 8) The system according to item 1, further comprising a processor configured to instruct to form a Fresnel pattern on the plurality of optical modulators. (Item 9) The system according to item 8, wherein the processor is configured to instruct to continuously form a plurality of Fresnel patterns on the plurality of optical modulators. (Item 10) The system according to item 9, wherein at least two of the plurality of Fresnel patterns are configured to concentrate the light on respective different voxels at different image depths. (Item 11) A system for generating a multi-depth image sequence, the system comprising: a light source, a modulation array including a plurality of optical modulators comprising, A system, wherein the light source is configured to project light onto at least some of the plurality of light modulators, and the plurality of light modulators are configured to concentrate respective portions of the light to generate a multi-depth image. (Item 12) The system according to claim 11, wherein the plurality of light modulators are also configured to generate a series of multi-depth images. (Item 13) The system according to claim 11, wherein the plurality of light modulators are configured to shift the phase of the light by several degrees. (Item 14) The system according to item 11, wherein the plurality of light modulators are configured to shift the phase of the light according to a modulation shift pattern. (Item 15) The system according to item 11, wherein the light source is a white light source. (Item 16) The system according to item 11, wherein the modulation array is a segmented modulation array comprising a first area configured to receive red light, a second area configured to receive green light, and a third area configured to receive blue light. (Item 17) The system according to claim 11, further comprising a diffractive optical assembly comprising one or more waveguide layers, wherein the one or more waveguide layers correspond to different image depth planes. (Item 18) The system according to item 11, further comprising a processor configured to instruct the plurality of light modulators to form a computer-generated hologram. (Item 19) The system according to item 18, wherein the processor is configured to instruct the plurality of light modulators to continuously form a plurality of computer-generated holograms. (Item 20) The system according to item 11, wherein the modulation array is an 8-bit LCOS. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings illustrate the design and utility of several embodiments of the present invention. Note that the figures are not drawn to exact scale, and elements of similar structure or function are represented by like reference numerals throughout the figures. To gain a deeper understanding of the foregoing and other advantages and objects of various embodiments of the present invention and the manner of obtaining them, a form for carrying out the invention briefly described above will be provided by reference to the specific embodiments illustrated in the accompanying drawings. It is understood that these drawings depict only typical embodiments of the invention and are not to be considered limiting of its scope, and that the invention will be described and explained with additional specificity and detail through the use of the accompanying drawings.

Figure 1A

Figure 1B

Figure 1C

Figure 2

Figure 3A

Figure 3B

Figure 4

Figure 5A

Figure 5B

Figure 5C

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14A

Figure 14B

Figure 15A

Figure 15B

Figure 16A

Figure 16B

[0024] Various embodiments are described hereinafter in this specification with reference to the figures. Note that the figures are not drawn to scale, and elements having similar structures or functions are denoted by like reference numerals throughout the figures. Note that the figures are intended only to facilitate the description of the embodiments. They are not intended as an exhaustive description of the invention or as a limitation on the scope of the invention. Additionally, the illustrated embodiments need not have all the aspects or advantages shown. Aspects or advantages described in connection with a particular embodiment are not necessarily limited to that embodiment and may be practiced in any other embodiment even if not so illustrated.

[0025] Furthermore, references to "some embodiments" or "other embodiments" throughout this specification mean that the particular features, structures, materials, or characteristics described in connection with the embodiments are included in at least one embodiment. Thus, the appearances of the phrases "in some embodiments" or "in other embodiments" in various places throughout this specification are not necessarily referring to the same embodiment or embodiments. In addition, for purposes of illustration and explanation, this disclosure is described in the context of enterprise applications in some embodiments. However, it should be noted that the present invention is not limited to enterprise applications in its scope and can actually be applied equally to other types of applications.

[0026] Before explaining examples illustratively depicted in some figures, a general introduction is provided to facilitate understanding. In some embodiments, a multi-depth image sequence can be generated using one or more light sources, the one or more light sources can be configured to direct light onto a plurality of light modulators, and the plurality of light modulators can be further configured to focus the light onto voxels. In some embodiments, a voxel is a volumetric imaging area or point in three-dimensional space and can be used to generate a three-dimensional graphical object. For example, the robot image 106 in FIG. 1A can consist of one or more voxels. The one or more voxels together enable a viewer to "see" the robot image in space. The plurality of light modulators can focus the light onto voxels on one or more image planes at different depths. Further, the plurality of light modulators can raster through the voxels at different image depths to create image objects that appear to the viewer to be at different depths.

[0027] FIG. 2 illustrates a system 200 for generating a multi-depth image sequence according to some embodiments. The system 200 includes a dynamic Fresnel module 202 and a DOE assembly 212. The dynamic Fresnel module 202 may receive an input signal 210 that includes information for generating a multi-depth image sequence. For example, in some embodiments, the input signal 210 may include information for generating a robot image 106 that appears to be at a distance of 100 meters and information for generating a comic-like avatar character 108 that appears to be at a distance of 1 meter. In some embodiments, the input signal 210 is input into an array controller 206. The array controller may include a modulation array 204 and a logic module 208. As will be described in more detail below, the modulation array 204 may include a plurality of light modulators (not shown in FIG. 2) that concentrate or direct light onto voxels or pixels to generate an image. In some embodiments, the array controller module 206 includes control codes for operating and managing the plurality of light modulators. In some embodiments, the logic module 208 receives the input signal 210 or image information, interprets and commands the image sequence information, and the array controller 206 may operate the modulation array 204 and follow the commands to rasterize the multi-depth image sequence.

[0028] In some embodiments, the dynamic Fresnel module 202 can be optically coupled to a diffractive optical element (DOE) assembly 212. For example, the dynamic Fresnel module 202 and the DOE assembly 212 can be physically coupled to each other or physically coupled to one or more common structures. According to some embodiments, the DOE assembly 212 includes one or more stacked planar waveguides or DOE layers 214a-214c with diffraction gratings 222a-222c (e.g., internal coupling gratings "ICG"), and the diffraction gratings deflect the image light along the span of the waveguides 214a-214c, enabling the image light 224 to exit the waveguides 214a-214c at an angle that mimics the natural real-world diffraction effect. Further, in some embodiments, the image light 224 exits the DOE layers 214a-214c toward the viewer using a second set of diffraction gratings (not shown). In some embodiments, each DOE layer 214a-214c is configured to mimic the light diffracted from an object at various distances. For example, the DOE layer 214a can be configured to simulate the diffracted light generated from an object 1 meter away (e.g., an avatar character 108 like a comic) (e.g., the depth plane 1 "DP1" is set to 1 meter). Similarly, the DOE layer 214b and the DOE layer 214c can be configured to simulate the diffracted light generated from an object at different distances (e.g., the depth plane 2 "DP2" can be set to 100 meters, and the depth plane 3 "DP3" can be set to optical infinity for a distant object).

[0029] Further details regarding the DOE assembly and lattice are described in U.S. Provisional Patent Application No. 61 / 909,174, filed November 27, 2013, under Attorney Docket No. ML30011.00, entitled "Virtual and augmented reality systems and methods", and U.S. Provisional Patent Application No. 62 / 005,807, filed May 30, 2014, under Attorney Docket No. ML30016.00, entitled "Methods and systems for virtual and augmented reality". The contents of the foregoing U.S. provisional patent applications are hereby expressly incorporated by reference herein for all purposes.

[0030] FIG. 3A shows an illustrative example. Therein, the flat image 300 shows a person 302, a tree 304 rooted in the ground 306, and a moon 308 in the sky. In the real world, light diffracts or spreads as it travels. Thus, light reflected from an object such as the moon 308 spreads more than light reflected from a closer object such as the person 302. As described above, the human visual system handles light from distant and nearby objects in at least two ways: (1) accommodation (e.g., vergence movement) and (2) focusing. For example, when viewing the moon in the real world, the eyes adjust by converging the lines of sight of each eye so that they intersect at the location where the moon is located. (Similarly, when gazing at the tip of one's own nose, the eyes adjust to converge the lines of sight of each eye so that they intersect at the location where the tip of the nose is located, and the object will appear to be "cross-eyed".) In addition to adjusting the line of sight, each eye must focus its crystalline system and account for the spread of light. For example, light reflected from the distant moon 308 may appear "blurry" compared to light reflected from the person 302 if it cannot be focused. Thus, to view the moon, each eye flattens it and adjusts the focus of its lens by refracting the moonlight less and less, which will ultimately focus on the moon. Similarly, to view the person, each eye rounds it and adjusts the focus of its lens by refracting the incident light more and more until it is focused on the person. As described above, adjusting the line of sight and focusing of each eye occur automatically together and are known as the "accommodation-vergence movement reflex."

[0031] Problems with conventional / legacy stereoscopic AR or VR configurations are that they violate the accommodation-vergence movement reflex. For example, referring to the flat image 300 in FIG. 3A, if a conventional / legacy stereoscopic AR or VR system displays the moon 308, the tree 304, and the person 302 all in focus at different perceived distances (e.g., the person appears closer and the moon appears farther away), the eyes do not need to refocus when looking at the moon or the person. This creates an inconsistency that violates the accommodation-vergence movement reflex. As stated, these types of legacy approaches are known to cause eye strain, headaches, or other forms of discomfort to the user.

[0032] In contrast, the multi-depth system 200 coordinates with the human accommodation-vergence reflex by displaying nearby and distant objects in different depth planes. For example, FIG. 3B shows the same flat image 300 (e.g., person 302, tree 304, ground 306, and moon 308) divided into three exemplary depth planes, DP1, DP2, and DP3, forming a depth composite image 310. The person 320, which is the object intended to be the closest, is displayed in depth plane 1 (DP1) adjusted to mimic light spreading out 1 meter away from the object. The central objects, the tree 304 and the ground 306, are displayed in depth plane 2 (DP2) adjusted to mimic light spreading out 5 meters away from the object. Finally, the moon 308, which is the farthest object, is displayed in depth plane 3 (DP3) adjusted to mimic light spreading out 384,400,000 meters away from the object. (384,400,000 meters is the approximate distance from the Earth to the moon. However, for objects beyond a certain distance, it is common to simply adjust the imaging system, such as a crystalline system, to optical infinity, whereby the incident light rays are approximated as substantially parallel rays.) Thus, when viewing the objects in the different depth planes of the depth composite image 310, the viewer need not adjust both their focusing and their line-of-sight convergence, and headaches or discomfort will not occur. Further, to generate an image sequence or video, one depth plane can be rastered or modified while the objects in the other depth planes are held constant. For example, while the tree 304, the ground 306, and the moon 308 are held constant in their respective depth layers, the image data corresponding to the person 302 within depth layer 1 can be updated multiple times per second, whereby the person will appear as if in a "video," e.g., the person can jump towards the moon 308 (from the viewer's perspective) or walk past the front of the tree 304.

[0033] Figure 4A illustrates a modulation array 400 according to some embodiments. The modulation array 400 can consist of a plurality of optical modulators 402. The optical modulators 402 are configured to modulate incident light (e.g., electromagnetic waves) that strikes the surface of the array (e.g., the surface of the plurality of optical modulators 402). The plurality of optical modulators 402 in Figure 4A are shown as a certain size (in relation to the overall height or width of the modulation array 400), but one of ordinary skill in the art will understand that the modulators can have their size and distance from each other (e.g., pitch, dot pitch) increased or decreased, and can accordingly increase or decrease the resolution. Figure 4B illustrates the modulation array 400 with a representative modulation shift pattern 404 assigned to the plurality of optical modulators. In some embodiments, the light bands within the modulated shift pattern 404 correspond to a 180-degree phase shift (e.g., shifting the incident light by 180 degrees), the dark bands correspond to a 0-degree phase shift (e.g., shifting the incident light by 0 degrees), and the intervening gray values correspond to shift values between 180 degrees and 0 degrees. The spacing between each of these bands can be adjusted such that constructive interference is created between the components of the light reflected at the desired focus.

[0034] As shown, the modulated shift pattern 404 focuses the light incident on the modulation array 400 onto a certain imaging area or voxel. On the other hand, for example, a modulated shift pattern 406 (showing a lower focus or central portion than the modulated shift pattern 404) focuses the light incident on the modulation array 400 onto a different voxel. Thus, by changing the modulated shift pattern, the modulation array can focus the incident light onto different voxels. In this way, the modulation array 400 can function as a dynamic Fresnel lens.

[0035] FIG. 5A illustrates a light source 502 that applies light onto a modulation array 500, according to some embodiments, where the modulation array focuses the light onto voxel {2, 2, 1}. In some embodiments, the light source 502 may be configured to project a beam of light onto at least a portion of the modulation array 500. The light source 502 may be configured to actively or passively shape the projected beam of light so as to reflect from the surface of the modulation array 500 or a particular portion thereof, and in some embodiments, may function as a point light source. In particular, the light source 502 may comprise a laser diode, a projector, an LED, an optical fiber, or other beam of light or source of inductive electromagnetic radiation, as is known in the art. The three-dimensional imaging area 504 is illustrated as comprising a plurality of voxels indicated by their relative locations with respect to the x-axis 508, y-axis 506, and z-axis 510. For example, as illustrated, the modulation array 500 focuses light onto voxel {2, 2, 1} having an x-value of 2 (e.g., “across 2”), a y-value of 2 (e.g., “above 2”), and a z-value of 1 (e.g., “behind 1”). In some embodiments, the size of the voxels is proportional to the distribution and size of the plurality of light modulators 402 within the modulation array 500. For example, if smaller voxels are required for higher resolution, the number and size of the light modulators 402 may be increased. Similarly, in some embodiments, the distance (e.g., pitch) between the light modulators may be decreased in some embodiments to increase the resolution.

[0036] FIG. 5B illustrates a light source 502 that projects light onto a modulation array 500, which focuses the light onto different voxels {3, 2, 2}. By varying the modulation shift pattern, the modulation array 500 can focus light not only onto different voxels in the X and Y dimensions, but also onto voxels located at different z distances. Thus, voxels within the first z-layer (in the case of the third voxel, all z values are "1", e.g., {__, __, 1}) may correspond to imaging objects in the DP1 depth plane. Similarly, voxels within the second z-layer (e.g., {__, __, 2}) may correspond to imaging objects in the DP2 depth plane, and voxels within the third z-layer (e.g., {__, __, 3}) may correspond to imaging objects in the DP3 depth plane. Thus, the person 302 from the depth composite image 310 in FIG. 3B may be rasterized within the first z-layer, the tree 304 and the ground 306 may be rasterized within the second z-layer, and the moon may be rasterized within the third z-layer. In some embodiments, the image is rasterized by illuminating each voxel. For example, to rasterize the person 302, the modulation array 500 may first use a first modulation shift pattern to generate voxels for {1, 1, 1} (including the person's right leg and foot in FIG. 5B), and then use a second modulation shift pattern to generate voxels for {2, 1, 1} (including the person's left leg and foot in FIG. 5B). In some embodiments, if an imaging object is not present within a voxel, the modulation array may skip the voxel. Thus, for example, a voxel {3, 1, 1} that does not contain any part of the person 302 to be imaged may be skipped, saving time and resources of the imaging system. In some embodiments, each voxel may be generated regardless of whether it contains an object to be imaged (e.g., an object). Thus, in some embodiments, the voxel {3, 1, 1} may still be focused, but a background value may be projected instead, or no light may be projected at all.

[0037] FIG. 5C illustrates a modulation array 500 with three depth planes (e.g., DP1, DP2, and DP3) according to some embodiments. In particular, each depth plane is configured such that they cannot be aligned in a row. In this configuration, the depth planes may correspond to the internal coupling lattices shown in FIG. 2. For example, the voxels within DP1 in FIG. 5C may correspond to the internal coupling lattice 222a, the voxels within DP2 in FIG. 5C may correspond to the internal coupling lattice 222b, and the voxels within DP3 in FIG. 5C may correspond to the internal coupling lattice 222c.

[0038] FIGS. 5A - 5C illustrate each depth plane as a 3×3 voxel matrix or tensor, but one of ordinary skill in the art will understand that the number and size of the voxels can be increased to generate a final image output of the desired resolution. Further, one light source (e.g., light source 502) is illustrated in the figures, but one of ordinary skill in the art will understand that one or more light sources may be implemented. In some embodiments, three light sources, namely one red light, one green light, and one blue light, may be implemented to generate RGB voxel values, which in turn may be rasterized to generate a color multi-depth image sequence. In some embodiments, three modulation arrays are implemented, one for each color, and they operate in conjunction to image one 3-channel voxel with RGB color space values. In some embodiments, one segmented modulation array is implemented, and different sections of the modulation array display different modulation shift patterns. In some embodiments, the segmented modulation array may have three segments, and each segment is configured to receive light of a different color.

[0039] FIG. 6 illustrates an exemplary AR system implementing a DOE assembly 212 and a dynamic Fresnel module 202, according to some embodiments. As shown, the system includes an image generation processor 602 with a memory 612, a CPU 616, a GPU 614, and other circuitry for image generation and processing. The image generation processor 602 can be programmed with desired virtual content for presentation to an AR system user. It should be understood that in some embodiments, the image generation processor 602 can be stored within a wearable AR system. In other embodiments, the image generation processor and other circuitry can be stored within a belt pack that is optically coupled to the wearable.

[0040] Virtual content or information generated by the image generation processor 602 can be transmitted to the display circuit 610. The display circuit 610 can include an interface circuit 632 that can communicate with the image generation processor 602 and can further interface with circuitry such as a Maxim chip 634, a temperature sensor 636, a piezoelectric drive / converter 638, a red light source 640, a blue light source 642, and a green light source 644. In some embodiments, a fiber optic combiner combines the generated light and uses a fiber (not shown) to route the light to the surface of a modulation array within the dynamic Fresnel module 202. However, as described, in some embodiments, the light sources can be integrated within the dynamic Fresnel module 202, which can have its own dedicated control circuitry, receive image information, and project light from one or more light sources onto the modulation array. The AR system can then use the dynamic Fresnel module 202 to generate voxels and direct the light towards one or more gratings within the DOE assembly 212.

[0041] (System Architecture Overview) FIG. 7 is a block diagram of an illustrative computing system 700 suitable for implementing an embodiment of the invention described herein. The computer system 700 includes a processor 707, a system memory 708 (e.g., RAM), a static storage device 709 (e.g., ROM), a disk drive 710 (e.g., magnetic or optical), a communication interface 714 (e.g., a modem or Ethernet (registered trademark) card), a display 711 (e.g., a CRT or LCD), an input device 712 (e.g., a keyboard), and a bus 706 or other communication mechanism for communicating information interconnecting the subsystems and devices.

[0042] According to one embodiment of the invention, the computer system 700 performs particular operations by the processor 707 executing one or more sequences of one or more instructions contained within the system memory 708. Such instructions may be read into the system memory 708 from another computer-readable / usable medium, such as the static storage device 709 or the disk drive 710. In alternative embodiments, hardwired circuitry may be used in place of or in combination with software instructions to implement the invention. Accordingly, embodiments of the invention are not limited to any specific combination of hardware circuitry and / or software. In one embodiment, the term "logic" shall mean any combination of software or hardware used to implement all or part of the invention.

[0043] The term "computer-readable medium" or "computer-usable medium" as used herein refers to any medium involved in providing instructions to the processor 707 for execution. Such a medium may take many forms, including but not limited to non-volatile media and volatile media. Non-volatile media includes, for example, optical or magnetic disks, such as the disk drive 710. Volatile media includes dynamic memory, such as the system memory 708.

[0044] A computer-readable medium in a general form includes, for example, a floppy (registered trademark) disk, a flexible disk, a hard disk, a magnetic tape, any other magnetic medium, a CD-ROM, any other optical medium, a punch card, a paper tape, any other physical medium with a pattern of holes, a RAM, a PROM, an EPROM, a FLASH-EPROM (e.g., NAND flash, NOR flash), any other memory chip or cartridge, or any other medium that can be read by a computer.

[0045] In certain embodiments of the present invention, the execution of a sequence of instructions for practicing the present invention is performed by a single computer system 700. According to other embodiments of the present invention, two or more computer systems 700 coupled by a communication link 715 (e.g., a LAN, a PTSN, or a wireless network) can cooperate with each other to perform a sequence of instructions required to practice the present invention.

[0046] The computer system 700 can transmit and receive programs, i.e., messages, data, and instructions including application code, through the communication link 715 and the communication interface 714. The received program code is executed when received by the processor 707 and / or can be stored in the disk drive 710 or other non-volatile storage device for later execution. The database 732 in the storage medium 731 can be used to store data accessible by the system 700 via the data interface 733.

[0047] Various portions of the modulation array of the dynamic Fresnel projector can act like one or more flat Fresnel lenses. Instead of a curved surface as in a conventional lens, an exemplary Fresnel lens has grooves or ridges that manipulate (e.g., pass and / or reflect) the characteristics of light (e.g., phase, amplitude, direction, etc.) that interact with the Fresnel lens, like a conventional lens. The Fresnel lens is a Fourier lens. The modulation array of the dynamic Fresnel projector has the ability to modulate the phase or amplitude of light by forming one or more Fresnel lenses. Examples of modulation arrays that can be used in one or more of the embodiments described herein include a liquid crystal on silicon (“LCOS”) module and one or more modules formed at least in part by an electro-optic material.

[0048] (Per-Voxel System) FIG. 8 illustrates a system 800 for generating a multi-depth image sequence comprising a dynamic Fresnel module 802 and a DOE assembly 812, according to some embodiments. The dynamic Fresnel module 802 reflects and focuses light 828 from a light source 826. In some embodiments, the dynamic Fresnel module 802 can be formed at least in part from an electro-optic material. For example, the dynamic Fresnel module 802 can include an indium phosphide material having properties that allow the Fresnel pattern to be refreshed at a rate in the GHz range. In other embodiments, the dynamic Fresnel module 802 can be an LCOS module.

[0049] At any given point in time, light 828 can correspond to a particular portion (e.g., a voxel) of the multi-depth image sequence. More specifically, the various characteristics of light 828, such as color and intensity, can correspond to the color and intensity of the voxels of the multi-depth image sequence to be displayed. Thus, the color and intensity of light 828 can be adjusted for each voxel over the course of the raster scan. In this way, light 828 can be a dynamic light stream in which the light characteristics (color, intensity, etc.) change over time. For example, in a scenario where the robot image 106 is to be displayed, light 828 can correspond to the voxels that form a part of the image of the robot image 106 that appears to be at a distance of 100 meters at any given point in time over the course of the raster scan. In this example, the color and intensity of light 828 can change such that each voxel of the image of the robot image 106 is displayed in an accurate manner. In another example where an avatar character like a comic is to be displayed, light 828 can correspond to the voxels that form a part of the avatar character 108 like a comic that appears to be at a distance of 1 meter at any given point in time over the course of the raster scan.

[0050] In some embodiments, the input signal corresponding to the light 828 is received at the array controller 806. The input signal synchronizes the characteristics of the light 828 from the light source 826 with the dynamic Fresnel module 802 that projects the light 828 to the appropriate location to form the desired voxels. The array controller may comprise a modulation array 804 and a logic module 808. As described above, the modulation array 804 may comprise a plurality of light modulators (not shown in FIG. 8) that concentrate or direct light onto voxels or pixels to generate an image or a portion thereof. The modulation array 804 may have 1-bit addressing (e.g., binary), as described below. In some embodiments, the array controller module 806 comprises control codes for operating and managing a plurality of light modulators. In some embodiments, the logic module 808 receives the input signal or image information, interprets and commands the image sequence information, and the array controller 806 may operate the modulation array 804 and rasterize the multi-depth image sequence according to the commands. In some embodiments, some or all of the functionality of the array controller 806 and the logic module 808 is provided using one or more modules that are logically and / or physically separate from the dynamic Fresnel module 802, such as circuitry stored within a belt pack optically coupled to the wearable optics, one or more cloud computing devices, etc.

[0051] In some embodiments, the dynamic Fresnel module 802 can be optically coupled to a diffractive optical element (DOE) assembly 812. For example, the dynamic Fresnel module 802 and the DOE assembly 812 can be physically coupled to each other or physically coupled to one or more common structures. According to some embodiments, the DOE assembly 812 includes one or more stacked planar waveguides or DOE layers 814a - 814c with diffraction gratings 822a - 822c (e.g., ICG) that deflect image light along the span of waveguides 814a - 814c and enable the image light 824 to exit the waveguides 814a - 814c at an angle that mimics natural real-world diffraction effects. Further, in some embodiments, the image light 824 exits from the DOE layers 814a - 814c toward the viewer using a second set of diffraction gratings (not shown). In some embodiments, each DOE layer 814a - 814c is configured to mimic light diffracting from an object at various distances. For example, the DOE layer 814a can be configured to simulate diffracted light originating from an object 1 meter away (e.g., an avatar character 108 like a comic) (e.g., depth plane 1 "DP1" is set to 1 meter). Similarly, the DOE layer 814b and the DOE layer 814c can be configured to simulate diffracted light originating from objects at different distances (e.g., depth plane 2 "DP2" can be set to 100 meters and depth plane 3 "DP3" can be set to optical infinity for a distant object).

[0052] In FIG. 8, the light 826 reflected from the modulation array 804 (which has a Fresnel pattern 830 thereon) converges to a specific target point (e.g., a voxel) in 3-D space. The Fresnel pattern 830 is configured to converge the light 826 from a known location of the light source 826 to the image light 824 at the target voxel. Changing the Fresnel pattern 830 moves the location of the target voxel. By rapidly changing the Fresnel pattern 830 on the modulation array 804, the dynamic Fresnel module 802 functions as a voxel raster projector that generates a 3-D image voxel by voxel.

[0053] (Per-frame system) FIG. 9 illustrates a system 900 for generating a multi-depth image sequence comprising a dynamic Fresnel module 902 and a DOE assembly 912, according to some embodiments. The system 900 depicted in FIG. 9 is similar to the system 800 depicted in FIG. 8. The basic difference between the two systems 800, 900 is that instead of the per-voxel raster of the system 800 depicted in FIG. 8 for forming a 3-D image, the system 900 depicted in FIG. 9 projects a complete 3-D image towards waveguides 914a - 914c. To project the entire image, the Fresnel pattern 930 is a computer-generated hologram ("CGH") 930 instead of the simple Fresnel pattern 830 as in FIG. 8.

[0054] The dynamic Fresnel module 902 reflects and focuses the light 928 from the light source 926. More specifically, the dynamic Fresnel module 902 may include an array controller 906 and a logic module 908 that performs one or more operations to determine the CGH 930 to be rendered by the modulation array 904 at any given point in time. The CGH 930 within the modulation array 904 may represent a pattern that modulates the incident light into the multi-depth image. For example, the CGH 930 may form an image that reflects the image light 924 and includes a robot image 106 that appears to be at a distance of 100 meters and a comic-like avatar character 108 that appears to be at a distance of 1 meter. In some embodiments, the modulation array 904 may be a phase-limited reflective LCOS module. As will be described below with reference to FIGS. 14 - 16, the quality of the multi-depth image generated by the system 900 may depend on the level of quantization of such an LCOS (i.e., the total number of different states in which each reflector within the LCOS can be placed). For example, an 8-bit LCOS modulation array such as the GAEA 4K Ultra HD 10MP phase-limited LCOS may be able to generate a multi-depth image with higher perceptual quality than that of a 1-bit LCOS modulation array.

[0055] The source light 928 can be a stream of light of a fixed color and intensity over the course of displaying multiple frames or images. In some embodiments, the source light 928 can be a stream of light that includes some or all of the color components represented within the multi-depth image to be displayed. For example, the source light 928 can be a static stream of white light. At any given point in time, the geometry of the CGH 930 can correspond to a particular frame of the multi-depth image. More specifically, the geometry and design of the CGH 930 that can correspond to a particular frame of the multi-depth image can correspond to the color and intensity of the frame to be displayed as well as the location within the three-dimensional space where each part of the frame is to be displayed. Thus, the CGH 930 can be adjusted on a frame-by-frame basis over the course of presenting the multi-depth image sequence. Thus, some of the characteristics (color, intensity, 3-D location, etc.) of the source light 928 reflected from the modulation array 400 will be changed over time by the changes within the CGH 930 on the modulation array 904. The CGH 930 can include a particular geometry that utilizes principles similar to those of their Fresnel lens patterns to create destructive interference among light of unwanted wavelengths at specific locations within the three-dimensional space. The CGH for a particular target 3-D image can be generated using various algorithms (e.g., the Gerchberg-Saxton algorithm). The static light source 926 within the system 900 is smaller and less power-intensive than the dynamic (e.g., RGB) light source 826 within the system 800. However, in some embodiments, the light source 826 within the system 800 can be substantially similar to that of the static light source 926 within the system 900. In these embodiments, the Fresnel pattern displayed by the modulation array 804 can be determined based on the desired focus of the voxels as well as the desired color of the voxels.

[0056] The Fresnel lens / projector can form a CGH for projecting a 3-D image with parts in multiple depth planes. For any 3-D projector, there is a path between the light source and the final image, which includes distance and image composition. Some or all of the voxels that make up the 3-D image can be generated simultaneously using the CGH. One explanation for this result is that sub-parts of the Fresnel lens are focused on different depth planes. An example would be four sub-parts, but more sub-parts can also exist. In the case of a composite Fresnel pattern, different parts of the modulation array focus different parts of the image corresponding to different depths.

[0057] An alternative explanation is that for different depths, all parts of the modulation array (e.g., LCOS) are used for different parts of the amplitude. Mathematically, the CGH is a composite. The CGH on the plane of the modulation array can represent the Fourier transform of the target 3-D image with respect to the light source. The CGH finally rendered on the plane of the modulation array can, in some embodiments, be a composite pattern generated based on the Fourier transform of the target 3-D image and the Fresnel patterning corresponding to specific locations in the three-dimensional space where each part of the target 3-D image should be presented. The spatial information is converted to an amplitude in the modulation array plane so that the light source can be modulated. As a result, the Fresnel modulation array can deliver the 3-D image towards the user's eye.

[0058] Each of the systems 800, 900 depicted in FIGS. 8 and 9 has different advantages. In FIG. 8, the locations of all voxels that can be created by the system 800 can be known in advance. The system generates all the Fresnel patterns 830 required to generate voxels at all possible locations (i.e., to represent it) and can store the Fresnel patterns in system memory (e.g., a field programmable gate array and / or RAM). Whenever a voxel is needed at a particular location, the system 800 can call the Fresnel pattern 830 representing that location from the system memory. When light 828 from the light source 826 reflects from the modulation array 804, the voxel appears at the target location.

[0059] In FIG. 9, an entire 3-D image is generated by the system 900 instead of a series of voxels. The system 900 uses a CGH 930 on the surface of the modulation array 904 to generate a 3-D image. When the system 900 grasps the image that will be generated (e.g., for a 3-D movie), the system 900 can generate all of the CGH in advance and store them before continuous display. When the system 900 does not grasp the image that will be generated, it cannot store the pre-generated CGH 930. The 3-D image generated using the CGH 930 in FIG. 9 may have improved resolution compared to the raster 3-D image generated using the Fresnel pattern 830 in FIG. 8.

[0060] The exemplary 3-D image has a 9MP image at each depth. In the system 800 depicted in FIG. 8, the Fresnel pattern for all those voxel locations can be stored, for example, on the system RAM. To generate the 3-D image, the system 800 circulates through all those voxels as fast as possible, assuming the speed limits of the processor (e.g., FPGA) and the modulation array 804. For example, a 9MP image rastered at 500 - 600 MHz can generate a relatively high-perception-quality 3-D image. Thus, a raster system such as that depicted in FIG. 8 is limited by the speed of the processor and the modulation array 804. For a single-pixel raster system such as that depicted in FIG. 8, the rate-limiting step / component can be the switch rate of the modulation array 804.

[0061] In some embodiments, the modulation array 904 in FIG. 9 operates at a lower speed than the modulation array 804 in FIG. 8 (e.g., LCOS at a KHz switch rate), because the rate at which frames are displayed in a frame-per-system can be much lower than the rate at which voxels are scanned within a voxel-per-system, yet can still be converted to a multi-depth image presented at an equal or higher rate. The system 900 pre-generates a series of CGHs corresponding to a 3-D series of frames. The system 900 can circulate through the series of CGHs / 3-D frames and display 3-D motion graphics. The frame-per-system with the pre-generated CGHs is limited by the frame rate of the high-density LCOS. The frame-per-system that generates the CGH on-the-fly (i.e., in real time), which is a computationally intensive step, is limited by the CGH generation step and the processor speed. The processor speed can be balanced with other metrics such as size, weight, heat, etc.

[0062] Figures 8 and 9 represent two methods of generating 3-D images of one voxel at a time (Figure 8) or one image at a time (Figure 9) along with the raster. The corresponding system speed requirements include high-speed raster (Figure 8) and rapid generation of the CGH representing each 3-D image (Figure 9).

[0063] Figures 10 and 11 schematically illustrate components of systems 1000, 1100 for generating multi-depth image sequences with dynamic Fresnel modules 1002, 1102, according to some embodiments. System 1000 depicted in FIG. 10 is similar to system 800 depicted in FIG. 8 in that both systems generate one voxel of a 3-D image at a time with a raster. An input signal 1032, which includes image information, is input to a light source controller 1034, which commands a dynamic light source 1026 to vary the characteristics of the source light in accordance with the generation of voxels of the 3-D image corresponding to the image information. A focus location determination unit 1036 determines the location of voxels within the 3-D image. The focus location determination unit 1036 transmits the determined location information to a modulation pattern determination unit 1038, which generates or calls a pre-generated Fresnel pattern for display on a modulation array 1004 that focuses the image light to a 3-D location corresponding to the location information. Thus, system 1000 projects one voxel of the 3-D image. Rastering through all of the voxels generates the 3-D image. With respect to embodiments where the modulation array 1004 reflects light from the light source 1026 into diffraction gratings (e.g., ICG) of one or more DOE layers, and the diffraction gratings deflect the image light and output it at an angle that ultimately mimics a real-world diffraction effect, the focus locations corresponding to the Fresnel patterns displayed on the modulation array 1004 can be the same for one or more voxels that should be displayed at the same depth. That is, since the angle at which light is provided to such a diffraction grating can determine the position in 3-D space at which such light appears to a viewer, the modulation array 1004, in these embodiments, can raster voxels located within the same depth plane by displaying Fresnel patterns that direct the light to the same focus on the DOE but from different locations on the surface of the modulation array 1004. Thus, the modulation array 1004 can operate to effectively modulate the angle at which light is provided to the DOE as input for a given depth.The system 1100 depicted in FIG. 11 is similar to the system 900 depicted in FIG. 9 in that both systems can use a CGH on a modulation array 1104 to generate a complete 3-D image. A light source controller 1134 commands a static light source 1126 to project or not project source light for generating a 3-D image. An input signal 1132 containing image information is input to a focus location determination unit 1136 that determines the voxel locations of the 3-D image. The focus location determination unit 1136 transfers the input signal 1132, along with the determined location information, to a modulation pattern determination unit 1138, which generates or calls a pre-generated Fresnel pattern / CGH that focuses a portion of the image light to various 3-D locations to form a 3-D image. Thus, the system 1100 generates and projects a 3-D image as a single frame.

[0064] The use of a CGH 930 such as that shown in the system 900 depicted in FIG. 9 can potentially minimize system components to a light source and a modulation array. Such systems can be implemented without lenses, reducing the volume and weight of these systems. These systems can be implemented without any waveguides (see FIG. 11). Even without waveguides, a Fresnel modulation array and a light source can focus light into the user's eye without other optical components (e.g., relay lenses, variable focus elements, etc.). These systems can include a meniscus lens that directs the output from the Fresnel modulation array into the user's eye. Thus, these systems can be small enough to be incorporated within a conventional eyeglass form factor.

[0065] Figure 12 depicts a system 1200 that is substantially the same as the system 1100 depicted in Figure 11. That is, elements 1204 - 1238 can provide similar utility as elements 1104 - 1138 that can be provided within system 1100, respectively, in system 1200. System 1200 is a per-frame system (e.g., Figures 9 and 11) instead of a per-voxel system (e.g., Figures 8 and 10). Figure 12 also depicts three patterns associated with various parts of system 1200. A target image 1240 (see also Figure 13) is associated with an input signal 1232. The target image 1240 represents an image that system 1200 is attempting to project. The target image 1240 is a 2-D image in this example, but can be a 3-D image similar to those described above. A CGH image 1242 is associated with a modulation array 1204. The CGH image 1242 simulates the appearance of a portion of the surface of the modulation array 1204 configured to generate / project the target image 1240. A projected image 1244 is associated with image light 1224 reflected from the modulation array 1204. The projected image 1244 simulates the appearance of the image projected by system 1200 (using the CGH image 1242) based on the target image 1240.

[0066] Figures 14A - 16B illustrate various CGH images 1442, 1542, 1642 and projected images 1444, 1544, 1644 based on the target image 1340 depicted in Figure 13. Figures 14A and 14B simulate patterns associated with a system using a 1-bit LCOS. Figure 14A depicts the simulated projected image 1444. Figure 14B depicts the CGH image 1442 simulated in the modulation array. With a 1-bit LCOS, all reflectors on the LCOS are in one of two states (i.e., binary or two-level quantization). This limited 1-bit LCOS results in a distorted (e.g., mirrored and inverted) projected image 1444. Thus, a 1-bit LCOS can be utilized to provide computational savings within an AR system that is robust to distortion, such as a per-voxel system.

[0067] Figures 15A and 15B simulate patterns associated with a system using a 2-bit LCOS. With a 2-bit LCOS, all reflectors on the LCOS are in one of four states (i.e., four-level quantization). As shown in FIG. 15A, the simulated projected image 1544 is not mirrored or inverted and thus represents an improvement over the simulated projected image 1444. FIG. 15B depicts the CGH image 1542 simulated in the modulation array.

[0068] Figures 16A and 16B simulate patterns associated with a system using an 8-bit LCOS. With an 8-bit LCOS, all reflectors on the LCOS are in one of 256 states (i.e., 256-level quantization). As shown in FIG. 16A, the simulated projected image 1644 is a representation of the target image 1340 of higher quality than the simulated projected image 1544. The per-frame system 900 depicted in FIG. 9 includes an 8-bit LCOS 904. FIG. 16B depicts the CGH image 1642 simulated in the modulation array.

[0069] The CGH 1642 of the 8-bit LCOS is more complex than the CGH 1542, 1442 of the 2-bit or 1-bit LCOS. This higher bit-level LCOS results in a higher quality representation of the target image, but the higher bit-level LCOS requires more processor cycles / calculations to generate a more complex CGH. Thus, the 8-bit LCOS can be utilized to provide a relatively high quality target image within an AR system with sufficient processing resources to handle the corresponding computational load. Other LCOSs can include 16-bit, 32-bit, and 64-bit depending on the speed of the processor and / or the architecture and capabilities of the LCOS drive circuit.

[0070] In the foregoing specification, the present invention has been described with reference to its specific embodiments. However, it will be apparent that various modifications and changes can be made therein without departing from the broader spirit and scope of the invention. For example, the foregoing process flow is described with reference to a particular order of process actions. However, many of the orders of the described process actions can be changed without affecting the scope or operation of the invention. The specification and drawings are, therefore, to be regarded in an illustrative rather than a limiting sense.

Claims

1. A system for generating a plurality of depth composite images, the system comprising: a plurality of light sources including a green light source, a red light source, and a blue light source; one or more modulation arrays comprising a plurality of optical modulators, the plurality of light sources being configured to project light onto the plurality of optical modulators, the one or more modulation arrays including segmented modulation arrays, the segmented modulation arrays including a first area configured to receive red light, a second area configured to receive green light, and a third area configured to receive blue light; one or more modulation arrays; a processor configured to synchronously direct the plurality of light sources and the plurality of optical modulators to generate a plurality of depth composite images in a time series, each of the plurality of depth composite images being generated by rasterizing a plurality of voxels in a plurality of different image depth planes; a processor comprising each of the plurality of depth composite images is an image formed by combining a plurality of voxels in the plurality of different image depth planes; To generate each voxel of each depth composite image, the processor receives an input signal including image information; determines a three-dimensional location of the voxel within an image depth plane of the depth composite image based at least in part on the received input signal; commanding the plurality of light sources to project light onto the plurality of optical modulators, wherein a Fresnel pattern is displayed on the plurality of optical modulators, and the plurality of optical modulators concentrate light corresponding to the Fresnel pattern onto the voxel at the determined three-dimensional location to form a part of the depth composite image; configured to perform The Fresnel pattern displayed in the first area, the Fresnel pattern displayed in the second area, and the Fresnel pattern displayed in the third area are different from each other. A system.

2. The system according to claim 1, wherein the plurality of optical modulators are configured to shift the phase of the light by an angle greater than 0 degrees and less than or equal to 180 degrees.

3. The system according to claim 1, wherein the plurality of optical modulators are configured to shift the phase of the light according to a modulation shift pattern.

4. The system according to claim 1, further comprising a diffractive optical assembly comprising two or more waveguide layers, each waveguide layer corresponding to a respective image depth plane comprising respective rasterized voxels.

5. The plurality of light sources, the one or more modulation arrays, and the processor are components of an augmented reality system wearable by a user of the system, the processor being stored within a belt pack of the wearable augmented reality system, the belt pack being optically coupled to a wearable that includes the plurality of light sources and the one or more modulation arrays, the system according to claim 1.

6. The system according to claim 1, wherein the processor is configured to instruct the plurality of light modulators to successively form respective Fresnel patterns for respective voxels.

7. The system according to claim 6, wherein at least two of the plurality of Fresnel patterns are configured to successively focus the light on respective different voxels in a plurality of different image depth planes.

8. The system according to claim 1, wherein the processor is configured to perform a raster scan on a per-voxel basis such that, at any point during the raster scan, light from the plurality of light sources is focused onto a particular voxel of the depth composite image.

9. The system according to claim 8, wherein the processor is configured to perform the raster scan on a per-voxel basis within a particular image depth plane of the depth composite image.

10. The system according to claim 1, wherein the processor is configured to adjust, on a per-voxel basis, the color and intensity of light emitted by the plurality of light sources.

11. The system according to claim 1, wherein the processor is configured to perform a raster scan and modify one or more voxels of a first image depth plane of the depth composite image while voxels in a different second image depth plane of the depth composite image are held such that they are not modified.

12. The system according to claim 1, wherein the processor is configured to perform a raster scan and modify one or more voxels in a first image depth plane of the depth composite image while voxels in a plurality of other image depth planes of the depth composite image are held unchanged.

13. The system according to claim 1, wherein the processor is configured to perform a raster scan by instructing the plurality of light modulators to successively form a plurality of Fresnel patterns successively focused on respective different voxels during the raster scan.

14. The system according to claim 13, wherein the processor is configured to instruct the plurality of light modulators to move the location of a target voxel during the raster scan by changing the Fresnel pattern of the light reflected from the one or more modulation arrays.

15. The system according to claim 14, wherein the one or more modulation arrays are configured to refresh the Fresnel pattern so as to function as a voxel raster projector that successively generates the plurality of voxels.

16. The system according to claim 1, comprising a dynamic Fresnel module configured to change respective modulated shift patterns, each of the respective modulated shift patterns being successively displayed to focus light on respective voxels.

Images