Holographic superposition of real world plenoptic opacity modulation via transparent conductive waveguide array for light-field, virtual and augmented reality

Abstract

To disclose a transparent energy relay waveguide system for superposing a holographic opacity modulation state for application of holographic, light-field, virtual, augmented, and mixed reality.SOLUTION: A light-field system may comprise one or more energy waveguide relay systems having one or more energy modulation elements, where each energy modulation element is configured to modulate energy passing therethrough, which allows the energy passing through the energy modulation element to be directed according to a 4D plenoptic function or its inverse function.SELECTED DRAWING: Figure 13

Description

[Technical field]

[0001] This disclosure relates to light field and 4D energy modulation systems, and more particularly to holographic propagation through an energy waveguide relay system for superimposing digital opacity onto a real-world coordinate system. [Background technology]

[0002] Popularized by Gene Roddenberry's Star Trek and originally envisioned by author Alexander Moszkowski in the early 1900s, the dream of an interactive virtual world inside a "holodeck" chamber has been an inspiration for science fiction and technological innovation for nearly a century. However, no groundbreaking realization of this experience exists outside the literature, media, and collective imaginations of children and adults alike. Summary of the Invention

[0003] A transparent energy relay waveguide system for superimposing holographic opacity modulation states for holographic, light field, virtual, augmented, and mixed reality applications is disclosed.

[0004] In one embodiment, a transparent display system comprises a first energy waveguide relay system configured to direct energy passing therethrough according to a first 4D plenoptic function; a second energy waveguide relay system following the first energy waveguide relay system, the second energy waveguide relay system configured to direct energy passing therethrough according to a second 4D plenoptic function, the second 4D plenoptic function being an inverse function of the first 4D plenoptic function; and a first energy modulating element disposed at a first position within the first energy waveguide relay system, at a second position within the second energy waveguide relay system, or at a third position between the first energy waveguide relay system and the second energy waveguide relay system, the first energy modulating element configured to modulate energy passing therethrough.

[0005] In some embodiments, a first energy waveguide relay system includes a first array of energy waveguides configured to direct energy therethrough along multiple energy propagation paths, the energy waveguides of the first array being positioned at different spatial coordinates, each energy waveguide directing energy from a respective spatial coordinate to the multiple energy propagation paths along a different direction following a first 4D plenoptic function.

[0006] In other embodiments, the second energy waveguide relay system includes a second array of energy waveguides configured to direct energy therethrough along multiple energy propagation paths, the energy waveguides of the second array being positioned at different spatial coordinates, each energy waveguide directing energy from a respective spatial coordinate to the multiple energy propagation paths along a different direction according to a second 4D plenoptic function.

[0007] In one embodiment, the system further comprises a second energy modulating element disposed at one of the first, second, or third locations, the second energy modulating element configured to modulate energy passing therethrough. In one embodiment, the first and second energy modulating elements are disposed at the same location. In another embodiment, the first and second energy modulating elements are disposed at different locations.

[0008] In one embodiment, the system further comprises a third energy modulating element disposed at one of the first, second, or third locations, the third energy modulating element configured to modulate energy passing therethrough, In some embodiments, the third energy modulating element and at least one of the first and second energy modulating elements are disposed at the same location.

[0009] In one embodiment, the first, second, and third energy modulating elements are located at the same location. In another embodiment, the first, second, and third energy modulating elements are located at different locations.

[0010] In some embodiments, each of the first, second, and third energy modulating elements comprises an LCD, LED, DLP, OLED, LCOS, quantum dot, or other suitable energy modulating element. In another embodiment, at least one of the first energy waveguide relay system and the second energy waveguide relay system is curved. In yet another embodiment, both the first energy waveguide relay system and the second energy waveguide relay system are curved.

[0011] In one embodiment, the transparent system includes a first energy waveguide relay system configured such that energy passing therethrough is directed according to a first 4D plenoptic function, a second energy waveguide relay system following the first energy waveguide relay system, the second energy waveguide relay system configured such that energy passing therethrough is directed according to a second 4D plenoptic function, the second 4D plenoptic function being an inverse function of the first 4D plenoptic function, and a second energy waveguide relay system following the first energy waveguide relay system, the second energy waveguide relay system configured such that energy passing therethrough is directed according to a second 4D plenoptic function, the second 4D plenoptic function being an inverse function of the first 4D plenoptic function. a first energy modulating element disposed at the first location, at a second location within the second energy waveguide relay system, or at a third location between the first energy waveguide relay system and the second energy waveguide relay system, the first energy modulating element configured to modulate energy passing therethrough, and a second energy modulating element disposed at one of the first, second, or third locations, the second energy modulating element configured to modulate energy passing therethrough.

[0012] In another embodiment, a first energy waveguide relay system includes a first array of energy waveguides configured to direct energy therethrough along multiple energy propagation paths, the energy waveguides of the first array being positioned at different spatial coordinates, each energy waveguide directing energy from a respective spatial coordinate to the multiple energy propagation paths along different directions according to a first 4D plenoptic function.

[0013] In another embodiment, the second energy waveguide relay system includes a second array of energy waveguides configured to direct energy therethrough along multiple energy propagation paths, the energy waveguides of the second array being positioned at different spatial coordinates, each energy waveguide directing energy from a respective spatial coordinate to the multiple energy propagation paths along a different direction according to a second 4D plenoptic function.

[0014] In one embodiment, the first and second energy modulating elements are located at the same location. In another embodiment, the first and second energy modulating elements are located at different locations.

[0015] In some embodiments, the optical fiber further comprises a third energy modulation element disposed at one of the first, second, or third location, the third energy modulation element configured to modulate energy passing therethrough. In one embodiment, the third energy modulation element and at least one of the first and second energy modulation elements are disposed at the same location.

[0016] In some embodiments, the first, second, and third energy modulating elements are located at the same location. In other embodiments, the first, second, and third energy modulating elements are located at different locations.

[0017] In some embodiments, each of the first, second, and third energy modulating elements comprises an LCD, LED, DLP, OLED, LCOS, quantum dot, or other suitable energy modulating element. In another embodiment, at least one of the first energy waveguide relay system and the second energy waveguide relay system is curved. In yet another embodiment, both the first energy waveguide relay system and the second energy waveguide relay system are curved.

[0018] In another embodiment, a first energy waveguide relay system configured such that energy passing therethrough is directed according to a first 4D plenoptic function, and a second energy waveguide relay system subsequent to the first energy waveguide relay system, the second energy waveguide relay system configured such that energy passing therethrough is directed according to a second 4D plenoptic function, the second 4D plenoptic function being an inverse function of the first 4D plenoptic function, and a first location within the first energy waveguide relay system, a second location within the second energy waveguide relay system, or a second location subsequent to the first energy waveguide relay system. a first energy modulating element disposed at a third location between the first energy modulating element and the second energy waveguide relay system, the first energy modulating element configured to modulate energy passing therethrough; a second energy modulating element disposed at one of the first, second, or third locations, the second energy modulating element configured to modulate energy passing therethrough; and a third energy modulating element disposed at one of the first, second, or third locations, the third energy modulating element configured to modulate energy passing therethrough.

[0019] In one embodiment, a first energy waveguide relay system includes a first array of energy waveguides configured to direct energy therethrough along multiple energy propagation paths, the energy waveguides of the first array being positioned at different spatial coordinates, each energy waveguide directing energy from a respective spatial coordinate to the multiple energy propagation paths along different directions according to a first 4D plenoptic function.

[0020] In another embodiment, the second energy waveguide relay system includes a second array of energy waveguides configured to direct energy therethrough along multiple energy propagation paths, the energy waveguides of the second array being positioned at different spatial coordinates, each energy waveguide directing energy from a respective spatial coordinate to the multiple energy propagation paths along a different direction according to a second 4D plenoptic function.

[0021] In one embodiment, the first and second energy modulation elements are located at the same location. In another embodiment, the first and second energy modulation elements are located at different locations. In some embodiments, the third energy modulation element and at least one of the first and second energy modulation elements are located at the same location. In other embodiments, the first, second, and third energy modulation elements are located at the same location. In yet some embodiments, the first, second, and third energy modulation elements are located at different locations.

[0022] In some embodiments, each of the first, second, and third energy modulating elements comprises an LCD, LED, DLP, OLED, LCOS, quantum dot, or other suitable energy modulating element. In another embodiment, at least one of the first energy waveguide relay system and the second energy waveguide relay system is curved. In yet another embodiment, both the first energy waveguide relay system and the second energy waveguide relay system are curved.

[0023] In one embodiment, a system includes a first energy waveguide relay system configured to direct energy passing therethrough according to a first 4D plenoptic function, a second energy waveguide relay system following the first energy waveguide relay system, the second energy waveguide relay system configured to direct energy passing therethrough according to a second 4D plenoptic function, the second 4D plenoptic function being an inverse function of the first 4D plenoptic function, a first energy modulating element disposed within the first energy waveguide relay system, a second energy modulating element disposed between the first energy waveguide relay system and the second energy waveguide relay system, and a third energy modulating element disposed within the second energy waveguide relay system, the first, second, and third energy modulating elements configured to modulate energy passing therethrough.

[0024] In one embodiment, a first energy waveguide relay system includes a first array of energy waveguides configured to direct energy therethrough along multiple energy propagation paths, the energy waveguides of the first array being positioned at different spatial coordinates, each energy waveguide directing energy from a respective spatial coordinate to the multiple energy propagation paths along different directions according to a first 4D plenoptic function.

[0025] In another embodiment, the second energy waveguide relay system includes a second array of energy waveguides configured to direct energy therethrough along multiple energy propagation paths, the energy waveguides of the second array being positioned at different spatial coordinates, each energy waveguide directing energy from a respective spatial coordinate to the multiple energy propagation paths along a different direction according to a second 4D plenoptic function.

[0026] In some embodiments, the first, second, and third energy modulating elements each comprise an LCD, LED, DLP, OLED, LCOS, quantum dot, or other suitable energy modulating element. In another embodiment, the one or more additional energy modulating elements disposed at one of the first, second, or third locations, the one or more additional energy modulating elements configured to modulate energy passing therethrough.

[0027] In one embodiment, the one or more additional energy modulating elements and at least one of the first, second, and third energy modulating elements are located at the same location. In another embodiment, the one or more additional energy modulating elements and at least one of the first, second, and third energy modulating elements are located at different locations. In one embodiment, at least one of the first energy waveguide relay system and the second energy waveguide relay system is curved. In another embodiment, both the first energy waveguide relay system and the second energy waveguide relay system are curved.

[0028] In some embodiments, a first energy modulating element comprises white opacity, a second energy modulating element comprises additional opacity or color, and a third energy modulating element comprises black opacity.

[0029] In one example, the first and third energy modulating elements are both OLEDs and the second energy modulating element is an LCD, so that to present an opaque black color, the first modulating element can be configured in an off state, the second modulating element can be configured in an on state, and the third modulating element can be configured in either the on state or the off state.

[0030] In another example, the first and third energy modulating elements are both OLED and the second energy modulating element is an LCD, so that to present a transparent black color, the first modulating element is configured in an off state, the second modulating element is configured in an off state, and the third modulating element is configured in an off state.

[0031] In one example, the first and third energy modulating elements are both OLEDs and the second energy modulating element is an LCD, so that to present an opaque red color, the first modulating element is configured in a red-only on state, the second modulating element can be configured in an on or off state, and the third modulating element is configured in an off state.

[0032] In another example, the first and third energy modulating elements are both OLED and the second energy modulating element is LCD, so that to present a transparent red color, the first modulating element is configured in a red-only state for a first percent, the second modulating element is configured in an off state, and the third modulating element is configured in a red-only state for a second percent, the second percentage being different from the first percentage.

[0033] In one example, the first and third energy modulating elements are both OLED and the second energy modulating element is an LCD, so that to present an opaque gray color, the first modulating element is configured in an on state for a first percent, the second modulating element is configured in an on state, and the third modulating element is configured in an off state.

[0034] In another example, the first and third energy modulating elements are both OLED and the second energy modulating element is LCD, so that to present a transparent gray color, the first modulating element is configured in an on state for a first percent, the second modulating element is configured in an off state, and the third modulating element is configured in an on state for a second percent, where the second percent is different from the first percent.

[0035] These and other advantages of the present disclosure will become apparent to those skilled in the art from the following detailed description and appended claims. [Brief description of the drawings]

[0036] [Figure 1] FIG. 1 is a schematic diagram illustrating design parameters for an energy directing system. [Diagram 2] FIG. 1 is a schematic diagram illustrating an energy waveguide relay system having an active device region with a mechanical envelope. [Diagram 3] FIG. 1 is a schematic diagram illustrating an energy relay system. [Figure 4] FIG. 13 is a schematic diagram illustrating one embodiment of an energy relay element bonded and fastened together to a base structure. [Figure 5A] 1 is a schematic diagram illustrating an example of an image relayed through a multi-core optical fiber; [Figure 5B] 1 is a schematic diagram illustrating an example of a relayed image through an optical relay exhibiting characteristics of the lateral Anderson localization principle; [Figure 6] FIG. 2 is a schematic diagram showing light rays propagating from an energy surface to a viewer. [Figure 7] 1 illustrates an orthogonal view of an optical relay system consisting of multiple energy waveguide pairs according to one embodiment of the present disclosure. [Figure 8] 1 illustrates orthogonal views of increasing size of each energy waveguide pair to account for the scaling of the field of view from the eye with respect to distance from the eye, according to one embodiment of the present disclosure. [Figure 9] 1 illustrates an orthogonal view of an energy waveguide that is both scaled and cylindrically radiating about the axis of rotation of the eye, according to one embodiment of the present disclosure. [Figure 10] 14 illustrates an orthogonal view of the planar relative position of each of three possible displays in relation to an energy waveguide pair according to one embodiment of the present disclosure. [Figure 11A]1 illustrates an orthogonal view of a configuration for coupling two displays together to increase efficiency of manufacturing and calibration, according to one embodiment of the present disclosure. [Figure 11B] 1 illustrates an orthogonal view of a configuration for coupling two displays together to increase efficiency of manufacturing and calibration, according to one embodiment of the present disclosure. [Figure 12] 1 illustrates an orthogonal view of an alternative to a volumetric opaque generating display that directly inverts the light rays so that one or more elements can be removed from the system, according to an embodiment of the present disclosure. [Figure 13] 1 illustrates an orthogonal view of light propagation paths along a set of individual elements in a waveguide relay system having multiple energy waveguide relay pairs according to one embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0037] One embodiment of the Holodeck (collectively referred to as the "Holodeck Design Parameters") provides sufficient energy stimuli to trick human sensory receptors into believing that energy impulses received within a virtual, social, and interactive environment are real, providing 1) binocular parallax without external accessories, head-mounted eyewear, or other peripherals; 2) precise motion parallax, occlusion, and opacity throughout the entire viewing volume for any number of viewers simultaneously; 3) visual focusing via synchronous convergence, ocular accommodation, and miosis for all perceived light rays; and 4) focused energy wave propagation of sufficient density and resolution to exceed human sensory "resolution" for vision, hearing, touch, taste, smell, and / or balance.

[0038] Based on current prior art, we are decades, if not centuries, away from technology that can provide all receptive fields in a groundbreaking way as suggested by the Holodeck Design Parameters, including visual, auditory, somatosensory, gustatory, olfactory, and vestibular systems.

[0039] In this disclosure, the terms light field and holographic may be used interchangeably to define energy propagation for the stimulation of any sensory receptor response. While the initial disclosure may refer to examples of electromagnetic and mechanical energy propagation through energy surfaces for holographic images and volumetric haptics, all forms of sensory receptors are contemplated within this disclosure. Furthermore, the principles disclosed herein for energy propagation along a propagation path may be applicable to both energy emission and energy capture.

[0040] Today, many technologies exist that are unfortunately often confused with holograms, including lenticular printing, Pepper's Ghost, autostereoscopic displays, horizontal parallax displays, head-mounted VR and AR displays (HMDs), and other such illusions that have been generalized as "folklography." While these technologies may exhibit some of the desired characteristics of true holographic displays, they lack the ability to stimulate the human visual sensory response in any way sufficient to address at least two of the four identified holodeck design parameters.

[0041] These challenges are that conventional techniques have not been successfully implemented to generate sufficiently seamless energy surfaces for holographic energy propagation. However, while there are various approaches to implementing volumetric and directionally multiplexed light field displays, including parallax barriers, hogels, voxels, diffractive optical elements, multi-view projection, holographic diffusers, rotating mirrors, multi-layer displays, time-sequential displays, head-mounted displays, etc., conventional approaches may require compromises in image quality, resolution, angular sampling density, size, cost, safety, frame rate, etc., which may ultimately render the technology unfeasible.

[0042] To arrive at holo-deck design parameters for the visual, auditory, and somatosensory systems, the human acuity of each of the systems is studied and understood to propagate energy waves sufficiently to fool the human sensory receptors. The visual system can resolve down to about one arc minute, the auditory system may distinguish placement differences of only 3 degrees, and the somatosensory system of the hand can discern points 2-12 mm apart. There are various and conflicting ways to measure these acuities, but these values ​​are sufficient to understand systems and methods for stimulating the perception of energy propagation.

[0043] Of the well-known sensory receptors, the human visual system is by far the most sensitive, with even a single photon capable of inducing a sensation. For this reason, much of this introduction focuses on visual energy wave propagation, with extremely low resolution energy systems coupled into the disclosed energy waveguide surfaces capable of focusing appropriate signals to induce holographic sensory perceptions. Unless otherwise noted, all disclosures apply to all energy and sensory domains.

[0044] When calculating the effective design parameters of energy propagation for a visual system given a viewing volume and viewing distance, the desired energy surface may be designed to include many gigapixels of effective energy location density. For large viewing volumes, or near-field viewing, the design parameters of the desired energy surface may include effective energy location densities of hundreds of gigapixels or more. In comparison, the desired energy source may be designed to have an energy location density of 1-250 effective megapixels in the case of ultrasonic propagation in volumetric haptics, or an array of 36-3,600 effective energy locations in the case of acoustic propagation in holographic acoustics, depending on the input environmental variables. What is important to note is that using the disclosed bidirectional energy surface architecture, all of the components may be configured to form the appropriate structure for any energy region and enable holographic propagation.

[0045] However, today, the main challenge to making the holodeck possible involves the limitations of available visual technology and electromagnetic devices. Acoustic and ultrasonic devices are less challenging given the difference in the desired density by a few orders of magnitude based on the sensory acuity in the respective receptive fields, but the complexity should not be underestimated. Holographic emulsions exist with resolutions that exceed the desired density and encode interference patterns in still images, whereas state-of-the-art display devices are constrained by resolution, data throughput, and manufacturing feasibility. To date, even extraordinary display devices have not been able to meaningfully generate light fields with holographic resolutions approaching visual acuity.

[0046] Fabricating a single silicon-based device capable of meeting the desired resolution for a groundbreaking light field display would be impractical and would involve extremely complex manufacturing processes beyond current manufacturing capabilities. The constraints on tiling multiple existing display devices together involve seams and gaps created by the physical size of packaging, electronics, housings, optics, and many other challenges that would inevitably result in an unviable technology from an imaging, cost, and / or size standpoint.

[0047] The embodiments disclosed herein may provide a real-world route to building a holodeck.

[0048] Exemplary embodiments will now be described hereinafter with reference to the accompanying drawings, which form a part hereof and which illustrate exemplary embodiments that may be practiced. As used in this disclosure and the appended claims, the terms "embodiment," "exemplary embodiment," and "exemplary embodiment" do not necessarily refer to a single embodiment, but they may be a single embodiment, and various exemplary embodiments may be readily combined and used interchangeably without departing from the scope or spirit of the exemplary embodiment. Furthermore, the terminology used herein is for the purpose of describing the exemplary embodiments only and is not intended to be limiting. In this regard, as used herein, the term "in" can include "in" and "on," and the terms "a," "an," and "the" can include singular and plural references. Furthermore, as used herein, the term "by" can also mean "from," depending on the context. Furthermore, as used herein, the term "if" can also mean "when" or "on," depending on the context. Furthermore, as used herein, the word "and / or" may refer to and include any and all possible combinations of one or more of the associated listed items.

[0049] Consideration of holographic systems Light Field Energy Propagation Resolution Overview Light field and holographic displays are the result of multiple projections where the energy surface location provides angle, color, and intensity information propagated within the viewing volume. The disclosed energy surfaces provide the opportunity for additional information to coexist and propagate through the same surface, eliciting other sensory system responses. Unlike volumetric displays, the viewed location of the converged energy propagation paths in space does not change as the viewer moves around the viewing volume, and multiple viewers may simultaneously observe the propagated objects in real-world space as if the objects were truly present there. In some embodiments, the energy propagation may be located within the same energy propagation path, but in opposite directions. For example, energy emission and energy capture along the energy propagation path are both possible in some embodiments of the present disclosure.

[0050] 1 is a schematic diagram illustrating variables related to the stimulation of a sensory receptor response. These variables include surface diagonal 101, surface width 102, surface height 103, defined target seat distance 118, target seat field of view relative to the field of view from the center of the display 104, number of intermediate samples demonstrated here as interocular samples 105, average adult interocular separation 106, average resolution per angular minute of the human eye 107, horizontal field of view formed between the target viewer position and the surface width 108, vertical field of view formed between the target viewer position and the surface height 109, resulting horizontal waveguide element resolution, or total number of elements across the surface 110, The resulting vertical waveguide element resolution, or total number of elements across the surface 111, may include a sample distance 112 based on the interocular spacing between the eyes and the intermediate sample number for the angular projection between the eyes, the angular sampling may be based on the sample distance and the target seat distance 113, the total resolution Horizontal per waveguide element may be derived from the desired angular sampling 114, the total resolution Vertical per waveguide element may be derived from the desired angular sampling 115, the Device Horizontal is a count of a determined number of desired discreet energy sources 116, and the Device Vertical is a count of a determined number of desired discreet energy sources 117.

[0051] A method for understanding the desired minimum resolution may be based on the following criteria to ensure sufficient stimulation of visual (or other) sensory receptor responses: surface size (e.g., 84 inch diagonal), surface aspect ratio (e.g., 16:9), seating distance (e.g., 128 inch distance from the display), seating field of view (e.g., 120 degrees or ±60 degrees relative to the center of the display), desired intermediate samples at a distance (e.g., one additional propagation path between the eyes), average separation between the lenses of an adult (approximately 65 mm), and average resolution of the human eye (approximately 1 arc minute). These example values ​​should be considered placeholders depending on the specific application design parameters.

[0052] Additionally, each of the values ​​attributed to the visual sensory receptors may be substituted for other systems to determine the desired propagation path parameters. For other energy propagation embodiments, the angular sensitivity of the auditory system may be considered to be as low as 3 degrees, and the spatial resolution of the somatosensory system of the hand may be considered to be as low as 2-12 mm.

[0053] Although there are various and conflicting ways to measure these perception acuity, these values ​​are sufficient to understand systems and methods that stimulate the perception of virtual energy propagation. While there are many ways to consider design resolution, the following proposed system of principles combines practical product considerations with the biological resolution limits of the sensory system. As will be appreciated by those skilled in the art, the following outline is a simplification of any such system design and should be considered for illustrative purposes only.

[0054] Once the resolution limit of the sensory system is understood, the total energy waveguide element density can be calculated such that the receiving sensory system is unable to distinguish a single energy waveguide element from adjacent elements, given by:

number

[0055] The above calculations result in a field of view of approximately 32×18°, resulting in an energy waveguide element of approximately 1920×1080 (rounded to nearest format) being desired. Also, variables may be constrained so that the field of view is consistent for both (u,v) and provides a more regular spatial sampling of the energy locations (e.g. pixel aspect ratio). Assuming the target viewing volume location defined between two points at the optimized distance, and additional propagating energy paths, the angular sampling of the system is given as follows:

number

[0056] In this case, the interocular distance is used to calculate the sample distance, although any scale may be used to account for the appropriate number of samples for a given distance. Taking into account the above variables, approximately 1 ray per 0.57° may be desired, and the overall system resolution for each individual sensory system may be calculated and given as follows:

number

[0057] Using the above scenario, given the energy surface size and angular resolution addressed for the vision system, the resulting energy surface may desirably include energy resolved locations of approximately 400k x 225k pixels, or a holographic propagation density of 90 gigapixels. These given variables are for illustrative purposes only, and many other sensory and energy metrology considerations should be considered for optimizing the holographic propagation of energy. In additional embodiments, energy resolved locations of 1 gigapixel may be desired based on the input variables. In additional embodiments, energy resolved locations of 1,000 gigapixels may be desired based on the input variables.

[0058] Limitations of current technology Active area, device electronics, packaging, and mechanical envelope 2 illustrates a device 200 having an active area 220 with a certain mechanical form factor. The device 200 may include a driver 230 and electronics 240 for power supply and connect to the active area 220, which has dimensions indicated by the x and y arrows. This device 200 does not take into account cabling and mechanical structures for driving power and cooling components, and furthermore, the mechanical footprint may be minimized by introducing flex cables into the device 200. The minimum footprint for such a device 200 may also be referred to as a mechanical envelope 210, which has dimensions indicated by the M:x and M:y arrows. This device 200 is for illustrative purposes only, and application specific electronics design may further reduce the overhead of the mechanical envelope, but in almost all cases may not be the exact size of the active area of ​​the device. In one embodiment, this device 200 illustrates the dependency of the electronics circuitry associated with the active image area 220 on a micro OLED, DLP chip, or LCD panel, or any other technology having the purpose of image illumination.

[0059] In some embodiments, it may also be possible to consider other projection technologies to aggregate multiple images across a larger display. However, this may result in costs due to greater complications with respect to throw distance, minimum focus, optical quality, uniform field resolution, chromatic aberration, thermal properties, calibration, alignment, additional size, or form factor. For most practical applications, hosting tens or hundreds of these projection sources 200 may result in less reliable and more extensive designs.

[0060] For illustrative purposes only, assuming an energy device having an energy location density of 3840 x 2160 sites, the number of individual energy devices (e.g., device 100) desired for the energy surface can be calculated and is given as follows:

number

[0061] Given the above resolution considerations, approximately 105 x 105 devices similar to the energy device shown in Figure 2 may be desired. Note that many devices are composed of various pixel structures that may or may not be mapped to a regular grid. If there are additional sub-pixels or positions within each complete pixel, these may be exploited to generate additional resolution or angular density. Additional signal processing may be used to determine how to transform the light field into the correct (u,v) coordinates according to the specified position of the pixel structure(s), which may be a known, calibrated, explicit characteristic of each device. Furthermore, other energy regions may require different handling of these ratios and device structures, and one skilled in the art will understand the direct inherent relationship between each of the desired frequency regions. This will be shown and discussed in more detail in the following disclosure.

[0062] The resulting calculations can be used to understand how many of these individual devices would be desired to generate a maximum resolution energy surface. In this case, approximately 105 x 105, or approximately 11,080 devices, would be desired to achieve visual acuity threshold. Challenges and novelty exist in creating a seamless energy surface from these available energy locations for sufficient sensory holographic propagation.

[0063] Seamless Energy Surface Overview: Energy relay array construction and design In some embodiments, approaches are disclosed to address the challenge of producing high energy location densities from an array of discrete devices without the seams constrained by the mechanical structure of each device. In one embodiment, an energy propagation relay system can make up an array of relays and form a single seamless energy surface, allowing for increasing the effective size of the active device area to meet or exceed the mechanical dimensions.

[0064] 3 illustrates one embodiment of such an energy relay system 300. As shown, the relay system 300 can include a device 310 mounted on a mechanical envelope 320 and have an energy relay element 330 that transmits energy from the device 310. The relay element 330 can be configured to provide the ability to reduce any gaps 340 that may occur when multiple mechanical envelopes 320 of a device are arranged in an array of multiple devices 310.

[0065] For example, if the active area 310 of the device is 20 mm x 10 mm and the mechanical envelope 320 is 40 mm x 20 mm, assuming that the arrays of these elements 330 can be seamlessly aligned together without changing or colliding the mechanical envelope 320 of each device 310, the energy relay elements 330 can be designed with a 2:1 scale factor to produce a tapered shape of about 20 mm x 10 mm on the reduced end (arrow A) and about 40 mm x 20 mm on the expanded end (arrow B). Mechanically, the relay elements 330 can be bonded or fused together to align and grind to ensure a minimum seam gap 340 between each device 310. In such an embodiment, it becomes possible to achieve a seam gap 340 smaller than the visual acuity limit of the eye.

[0066] FIG. 4 illustrates an example of a base structure 400 with energy relay elements 410 formed together and securely fastened to an additional mechanical structure 430. The mechanical structure of the seamless energy surface 420 provides the ability to serially couple multiple energy relay elements 410, 450 to the same base structure through bonding or other mechanical processes to mount the relay elements 410, 450. In some embodiments, each relay element 410 may be melted, bonded, glued, pressure fitted, aligned, or otherwise attached together to form the resulting seamless energy surface 420. In some embodiments, the device 480 may be mounted to the rear of the relay element 410 and passively or actively aligned to ensure proper alignment to the energy position within a defined tolerance.

[0067] In one embodiment, the seamless energy surface includes one or more energy locations and one or more energy relay element stacks include first and second sides, each energy relay element stack arranged to form a unique seamless display surface that directs energy along a propagation path extending between the one or more energy locations and the seamless display surface, where the distance between the ends of any two adjacent second sides of the terminating energy relay elements is less than the smallest perceptible contour as defined by human visual acuity better than 20 / 40 vision over a distance greater than a width of the unique seamless display surface.

[0068] In one embodiment, each of the seamless energy surfaces includes one or more energy relay elements each having one or more structures forming a first and second surface in a transverse and longitudinal orientation, the first relay surface having a different area from the second relay surface resulting in a positive or negative magnification, and configured with a pronounced surface contour for both the first and second relay surfaces that passes energy through the second relay surface to substantially fill an angle of ±10 degrees relative to the normal of the surface contour across the entire second relay surface.

[0069] In one embodiment, multiple energy fields may be configured within a single energy relay or among multiple energy relays to direct one or more sensory holographic energy propagation paths, including visual, auditory, tactile, or other energy fields.

[0070] In one embodiment, a seamless energy surface may be constructed with an energy relay that includes two or more first sides for each second side to both receive and emit one or more energy fields simultaneously to provide bidirectional energy propagation throughout the system.

[0071] In one embodiment, the energy relay is provided as a loosely coherent element.

[0072] Introducing component design constructs: Disclosed progress in transverse Anderson localized energy relay The properties of the energy relay can be significantly optimized according to the principles disclosed herein for energy relay elements that induce lateral Anderson localization, which is the propagation of light rays transported through a material that is laterally irregular but longitudinally coherent.

[0073] This may mean that the material effects that give rise to the Anderson localization phenomenon are less affected by total internal reflection than by randomization between multiple scattering paths, where wave interference may completely restrict the propagation of transverse orientations while continuing the propagation of longitudinal orientations.

[0074] An even more important advantage is the elimination of the cladding of conventional multicore optical fiber materials, which functionally eliminates scattering of energy between fibers, but at the same time acts as a barrier to the light energy, thereby reducing the transmission up to at least the core-to-cladding ratio (e.g., a 70:30 core-to-cladding ratio can transmit up to 70% of the received energy transmission), and also creates a strong pixelated patterning in the transmitted energy.

[0075] FIG. 5A illustrates an end view of one such non-Anderson localized energy relay 500, where an image may be relayed through a multi-core optical fiber that may exhibit pixelation and fiber noise due to the inherent characteristics of the optical fiber. With conventional multimode and multi-core optical fibers, the relayed image may be inherently pixelated due to the total internal reflection characteristics of the discrete array cores, where any inter-core crosstalk will degrade the modulation transfer function and increase edge blurring. The resulting images produced using conventional multi-core optical fibers tend to have a residual fixed noise fiber pattern similar to that shown in FIG. 3.

[0076] Figure 5B illustrates an example of the same relay image 550 through an energy relay including a material exhibiting the property of lateral Anderson localization, where the relay pattern has a greater density of grain structure compared to the fixed fiber pattern from Figure 5 A. In one embodiment, a relay including randomized micro-component design structures induces lateral Anderson localization and more effectively transports light with a higher resolvable resolution of propagation than commercially available multimode glass optical fiber.

[0077] There are significant advantages to lateral Anderson localized material properties with respect to both cost and weight, where similar optical grade glass material can cost and weigh 10-100 times more than the cost of the same material produced in one embodiment, where the disclosed systems and methods include randomized micro-component design structures that demonstrate significant opportunities to improve both cost and quality over other techniques known in the art.

[0078] In one embodiment, a relay element exhibiting lateral Anderson localization may include a plurality of at least two different component design structures in each of three orthogonal planes arranged in a one-dimensional lattice, the plurality of structures forming channels of randomized distribution of the wave propagation properties of the material in the lateral planes in the one-dimensional lattice, and similar values ​​of the wave propagation properties of the material in the longitudinal planes in the one-dimensional lattice, where energy waves propagating through the energy relay have higher transport efficiency in the longitudinal orientation relative to the lateral orientation and are spatially localized in the lateral orientation.

[0079] In one embodiment, multiple energy fields may be configured within a single, or between multiple lateral Anderson localized energy relays, to direct one or more sensory holographic energy propagation pathways, including visual, auditory, tactile, or other energy fields.

[0080] In one embodiment, the seamless energy surface is comprised of a lateral Anderson localized energy relay that includes two or more first sides for each second side to both receive and emit one or more energy fields simultaneously to provide bidirectional energy propagation throughout the system.

[0081] In one embodiment, the lateral Anderson localization energy relay is configured as a loosely coherent element or a flexible energy relay element.

[0082] Considerations regarding the 4D plenoptic function: Selective propagation of energy through holographic waveguide arrays. As discussed above and throughout this specification, a light field display system generally includes an energy source (e.g., an illumination source) and a seamless energy surface configured with sufficient energy position density, as clearly indicated in the above discussion. A plurality of relay elements may be used to relay energy from the energy device to the seamless energy surface. Once the energy is delivered to the seamless energy surface with the required energy position density, the energy may be propagated according to a 4D plenoptic function through the disclosed energy waveguide system. As will be understood by those skilled in the art, the 4D plenoptic function is well known in the art and will not be further detailed herein.

[0083] The energy waveguide system, together with a structure configured to change the angular direction of energy waves passing therethrough through representing the angular components of the 4D plenoptic function, selectively propagates energy through multiple energy locations along a seamless energy surface representing the spatial coordinates of the 4D plenoptic function, where the propagated energy waves may converge in space according to multiple propagation paths directed by the 4D plenoptic function.

[0084] Now referring to FIG. 6, an example of a light field energy surface in a 4D image space according to a 4D plenoptic function is illustrated. The figure shows a ray trace of the energy surface 600 to a viewer 620 in describing how rays of energy converge in space 630 from various locations in the viewing volume. As shown in the figure, each waveguide element 610 defines four-dimensional information describing the energy propagation 640 through the energy surface 600. The two spatial dimensions (referred to herein as x and y) are the physical energy locations that can be observed in the image space, and the angular components θ and φ (referred to herein as u and v) that are observed in the virtual space when projected through the energy waveguide array. Typically, and according to the 4D plenoptic function, the multiple waveguides (e.g., lenslets) can direct energy locations from the x, y dimensions to specific locations in the virtual space along directions defined by the u, v angular components in forming the holographic or light field systems described herein.

[0085] However, those skilled in the art will appreciate that a significant challenge to light field and holographic display technologies results in uncontrolled energy propagation due to designs that do not properly account for diffraction, scattering, diffusion, angular orientation, calibration, focus, collimation, curvature, uniformity, element crosstalk, and any of the many other parameters that contribute to reduced effective resolution and the inability to precisely focus energy with great fidelity.

[0086] In one embodiment, an approach to selective energy propagation to address challenges associated with holographic displays may include energy coding elements and substantially filling waveguide apertures with nearly collimated energy into an environment defined by a 4D plenoptic function.

[0087] In one embodiment, the array of energy waveguides may define multiple energy propagation paths through which each waveguide element is configured to extend, substantially filling the effective aperture of the waveguide element in a unique direction defined by a predetermined 4D function for the multiple energy locations along a seamless energy surface constrained by one or more elements positioned to restrict propagation of each energy location to only pass through a single waveguide element.

[0088] In one embodiment, multiple energy fields may be configured within a single energy field or between multiple energy waveguides to direct one or more sensory holographic energy propagation including visual, auditory, tactile, or other energy fields.

[0089] In one embodiment, the energy waveguides and seamless energy surfaces are configured to both receive and emit one or more energy fields, providing bidirectional energy propagation throughout the system.

[0090] In one embodiment, the energy waveguide is configured to propagate a nonlinear or non-regular energy distribution of energy that includes non-transparent void regions and leverages digitally encoded, diffractive, refractive, reflective, grin, holographic, Fresnel, or similar waveguide configurations for any seamless energy surface orientation including walls, tables, floors, ceilings, rooms, or other geometrically based environments. In additional embodiments, the energy waveguide elements can be configured to generate a variety of shapes that provide any surface profile and / or tabletop field of view that allows a user to view a holographic image from all around the energy surface in a 360 degree configuration.

[0091] In one embodiment, the energy waveguide array elements may be reflective surfaces and the arrangement of those elements may be hexagonal, square, irregular, semi-regular, curved, non-planar, spherical, cylindrical, tapered regular, tapered irregular, spatially varying, and / or multi-layered.

[0092] For any component within the seamless energy surface, the waveguide or relay component may include, but is not limited to, optical fibers, silicon, glass, polymers, optical relays, diffractive, holographic, refractive, or reflective elements, optical face plates, energy combiners, beam splitters, prisms, polarizing elements, spatial light modulators, active pixels, liquid crystal cells, transparent displays, or any similar material that exhibits Anderson localization or total internal reflection.

[0093] Holodeck Realization: Integration of a bidirectional seamless energy surface system for stimulating human sensory receptors within a holographic environment Large-scale environments of seamless energy surface systems can be constructed by tiling, fusing, bonding, attaching, and / or stitching multiple seamless energy surfaces together to form any size, shape, contour, or form factor, including entire rooms. Each energy surface system can include an assembly having a base structure, energy surfaces, relays, waveguides, devices, and electronics collectively configured for the propagation, emission, reflection, or detection of bidirectional holographic energy.

[0094] In one embodiment, tiled seamless energy system environments are aggregated to form large seamless flat or curved walls that contain the equipment that makes up all surfaces within a given environment, and are configured as any combination of seamless, discontinuous, faceted, curved, cylindrical, spherical, geometric, or irregular shapes.

[0095] In one embodiment, the planar surface aggregate tiles form a wall-sized system for theater or venue-based holographic entertainment. In one embodiment, the planar surface aggregate tiles cover a room with 4-6 walls, including both the ceiling and floor, for a cave-based holographic installation. In one embodiment, the curved surface aggregate tiles create a cylindrical seamless environment for an immersive holographic installation. In one embodiment, the seamless spherical surface aggregate tiles create a holographic dome for an immersive holodeck-based experience.

[0096] In one embodiment, aggregate tiles of seamless curved energy waveguides provide mechanical edges that follow a precise pattern along the boundaries of the energy suppression elements within the energy waveguide structure, bonding, aligning, or fusing adjacent tiled mechanical edges of adjacent waveguide surfaces, resulting in a modular, seamless energy waveguide system.

[0097] In further embodiments of the aggregated tiled environment, energy is propagated bidirectionally to multiple simultaneous energy regions. In additional embodiments, the energy surface provides the ability to both display and capture simultaneously from the same energy surface using a waveguide that is designed such that light field data can be projected by an illumination source through the waveguide and simultaneously received through the same energy surface. In additional embodiments, depth sensing and active scanning techniques may be further utilized to enable interaction between the energy propagation and the viewer in precise world coordinates. In additional embodiments, the energy surface and the waveguide are operable to emit, reflect, or focus frequencies that induce haptic excitation or volumetric haptic feedback. In some embodiments, any combination of bidirectional energy propagation and aggregated surfaces is possible.

[0098] In one embodiment, the system comprises an energy waveguide that allows bidirectional emission and detection of energy through an energy surface with one or more energy devices separately paired using two or more path energy couplers to pair at least two energy devices to the same portion of a seamless energy surface, or one or more energy devices are fixed behind the energy surface and in closest proximity to additional components fixed to the base structure or to a location in front and outside the FOV of the waveguide for off-axis direct or reflected projection or detection, the resulting energy surface providing bidirectional energy transmission that allows the waveguide to focus the energy, a first device to emit the energy, and a second device to detect the energy, where the information is processed to perform computer vision related tasks including, but not limited to, 4D plenoptic eyes, and retinal tracking or detection of interference within the propagating energy pattern, depth estimation, approximation, motion tracking, image, color, or sound information, or other energy frequency analysis. In additional embodiments, the tracked position actively calculates and corrects the position of the energy based on interference between bidirectionally captured data and projection information.

[0099] In some embodiments, multiple combinations of three energy devices including an ultrasonic sensor, a visible electromagnetic display, and an ultrasonic emitting device are configured together with each of the three first relay surfaces that propagate the combined energy into a single second energy relay surface together with each of the three first surfaces including design characteristics specific to the energy field of each device, as well as two design waveguide elements configured to provide the ultrasonic and electromagnetic energy with the ability to separately direct and focus the energy of each device and be substantially unaffected by other waveguide elements configured for the separate energy fields.

[0100] In some embodiments, a calibration procedure is disclosed that uses encoding / decoding techniques as well as a dedicated integrated system for converting data into calibration information suitable for energy propagation based on a calibrated configuration file, allowing efficient manufacturing, removal of system artifacts, and generation of a geometric mapping of the resulting energy surface.

[0101] In some embodiments, a series of additional energy waveguides and one or more energy devices may be integrated into one system to generate indistinct holographic pixels.

[0102] In some embodiments, additional waveguide elements may be integrated, including energy suppression elements, beam splitters, prisms, active parallax barriers, or polarization techniques, to provide spatial and / or angular resolution greater than the diameter of the waveguide, or for other super-resolution purposes.

[0103] In some embodiments, the disclosed energy system may also be configured as a wearable interactive device such as a virtual reality (VR) or augmented reality (AR) device. In other embodiments, the energy system may include conditioning optics(s) such that the displayed or received energy is focused to the nearest of a plane defined in space relative to the viewer. In some embodiments, the waveguide array may be incorporated into a holographic head mounted display. In other embodiments, the system may include multiple optical paths that allow the viewer to see both the energy system and the real world environment (e.g., a transparent holographic display). In these examples, the system may present as near field in addition to other methods.

[0104] In some embodiments, the transmission of data includes an encoding process with selectable or variable compression ratios that receives any dataset of information and metadata, analyzes the dataset, and receives or assigns material properties, vectors, surface IDs, new pixel data forming a sparser dataset, where the received data may include 2D, stereo, multi-view, metadata, light field, holographic, geometric shapes, vectors or vectorized metadata, and the encoder / decoder may provide the ability to transform real-time or offline data including image processing for 2D, 2D plus depth, metadata or other vectorized information, stereo, stereo plus depth, metadata or other vectorized information, multi-view, multi-view plus depth, metadata or other vectorized information, holographic, or light field content via depth estimation algorithms with or without depth metadata, and an inverse ray tracing method appropriately maps the resulting transformed data generated by inverse ray tracing from various 2D, stereo, multi-view, volumetric, light field, or holographic data to real world coordinates via a characterized 4D plenoptic function. In these embodiments, the total desired data transmission may be several orders of magnitude smaller than the transmitted information of the raw light field dataset.

[0105] Plenoptic opacity modulation via transparent waveguide arrays While much of this disclosure is directed to enabling sensory holographic experiences, the disclosed intermediate steps, including integration between virtual and augmented reality devices, significantly limit the data and processing requirements to simultaneously propagate and rasterize a complete sensory holographic data set. With other current VR and AR technologies, modulated opacity states, resolution, and lack of field of view dramatically limit the acuity of the sensory experience.

[0106] Alternatively, a novel transparent waveguide relay system is disclosed that allows holographic focusing of superimposed modulated opacity states across a real-world environment. A direct-view bidirectional energy waveguide relay design that allows transmission of real-world data with superimposed light field modulated attenuation of digital and real-world illumination includes HMD, holographic, sensory energy propagation, and traditional display applications. This energy waveguide relay design includes multiple energy waveguides and energy modulation elements in which there are a first pair and a second pair that may be separated by two focal lengths. The two pairs may be placed at equal distances apart. The transparent energy waveguide pair may be placed in front of the eye or at a visible location at a predetermined distance and may include additional waveguide elements to relay external energy that is not suppressed or attenuated by the opacity modulation. The light rays from the second pair, which is furthest from the eye, relay the reverse propagation path, and the introduction of the additional pair of energy waveguides at this distance once again rectifies the light rays directivity to the appropriate directionality. The 4D plenoptic function in combination with multiple energy modulating devices provides true 4D opacity and the propagation of holographic near-field or far-field digital information onto real-world coordinates with other modulated electromagnetic or sensory energies.

[0107] 7 illustrates an orthogonal view of a waveguide relay system 70 consisting of multiple energy waveguide pairs 72A, 72B, 72C, 72D according to one embodiment of the present disclosure. As shown in FIG. 7, each energy waveguide pair may be separated by a focal length and viewed from left to right 76, or vice versa, at the position of a viewer's eye 74. In some embodiments, additional waveguide elements, fewer waveguide elements, and various separations are contemplated within the scope of the present disclosure, and those specific embodiments should not be considered limiting in any way.

[0108] In one embodiment, the waveguide relay system 70 may include a first energy waveguide relay system 72A, 72B configured such that energy passing therethrough is directed according to a first 4D plenoptic function, and a second energy waveguide relay system 72C, 72D following the first energy waveguide relay system 72A, 72B, the second energy waveguide relay system 72C, 72D configured such that energy passing therethrough is directed according to a second 4D plenoptic function, the second 4D plenoptic function being an inverse function of the first 4D plenoptic function.

[0109] In one embodiment, system 70 includes a first energy modulating element (e.g., 102A, 102B, 102C, as best shown in FIGS. 10 and 11 ) disposed in a first location within the first energy waveguide relay system 72A, 72B, in a second location within the second energy waveguide relay system 72C, 72D, or in a third location between the first energy waveguide relay system 72A, 72B and the second energy waveguide relay system 72C, 72D, the first energy modulating element configured to modulate energy passing therethrough, as will be described in more detail in subsequent figures and discussion.

[0110] In some embodiments, the first energy waveguide relay system 72A, 72B includes a first array of energy waveguides configured to direct energy therethrough along multiple energy propagation paths, the energy waveguides of the first array being positioned at different spatial coordinates, each energy waveguide directing energy from a respective spatial coordinate to the multiple energy propagation paths along a different direction following a first 4D plenoptic function.

[0111] In other embodiments, the second energy waveguide relay system 72C, 72D includes a second array of energy waveguides configured to direct energy therethrough along multiple energy propagation paths, the energy waveguides of the second array being positioned at different spatial coordinates, each energy waveguide directing energy from its respective spatial coordinate to the multiple energy propagation paths along a different direction according to a second 4D plenoptic function.

[0112] In one embodiment, the system 70 further comprises a second energy modulating element (e.g., 102A, 102B, 102C, as best shown in FIGS. 10 and 11 ) disposed in one of a first, second, or third location, where the second energy modulating element is configured to modulate energy passing therethrough. In one embodiment, the first and second energy modulating elements are disposed in the same location. In another embodiment, the first and second energy modulating elements are disposed in different locations. This will also be explained in more detail in the following figures and discussion.

[0113] In one embodiment, the system further comprises a third energy modulating element (e.g., 102A, 102B, 102C, as best shown in FIGS. 10 and 11 ) disposed in one of a first, second, or third location, where the third energy modulating element is configured to modulate energy passing therethrough. In some embodiments, the third energy modulating element and at least one of the first and second energy modulating elements are disposed in the same location. This will also be explained in more detail in the following figures and discussion.

[0114] In one embodiment, the first, second, and third energy modulation elements (e.g., 102A, 102B, 102C, as best shown in FIGS. 10 and 11) are located at the same location. In another embodiment, the first, second, and third energy modulation elements (e.g., 102A, 102B, 102C, as best shown in FIGS. 10 and 11) are located at different locations.

[0115] In some embodiments, each of the first, second, and third energy modulating elements (e.g., 102A, 102B, 102C, as best shown in Figures 10 and 11) comprises an LCD, LED, DLP, OLED, LCOS, quantum dot, or other suitable energy modulating element. In another embodiment, at least one of the first energy waveguide relay system 72A, 72B and the second energy waveguide relay system 72C, 72D is curved. In yet another embodiment, both the first energy waveguide relay system 72A, 72B and the second energy waveguide relay system 72C, 72D are curved. This will also be described in more detail in the following figures and discussion.

[0116] Additional embodiments of this basic approach provide a graded waveguide that optimizes the energy propagation quality as seen by the eye. This may be implemented, but is not limited to, regionally varying functions, gradient-based gradients, additional energy suppression elements, diffractive properties, refraction, reflection, graded refractive index, holographic optics, etc., and / or incorporated into any of the potential designs above or below. A spatially varying energy waveguide design provides two or more regions with defined waveguide parameters to optimize a particular design for the eye. Gradient-based functions encompass the generation of a varying waveguide optimization for each element.

[0117] 8 illustrates an orthogonal view of a waveguide relay system 80 in which each energy waveguide pair 82A, 82B, 82C, 82D is increased in size to account for the scaling of the field of view from eye 84 with respect to distance from eye 84, according to one embodiment of the present disclosure. This embodiment provides higher waveguiding efficiency resulting in a propagation function more perpendicular to the aperture of the visual system, thereby more efficiently targeting the field of view of eye 84 through each element in system 86.

[0118] As described above, the waveguide relay system 80 may include a first energy waveguide relay system 82A, 82B configured such that energy passing therethrough is directed according to a first 4D plenoptic function, and a second energy waveguide relay system 82C, 82D following the first energy waveguide relay system 82A, 82B, the second energy waveguide relay system 82C, 82D configured such that energy passing therethrough is directed according to a second 4D plenoptic function, the second 4D plenoptic function being an inverse function of the first 4D plenoptic function.

[0119] In one embodiment, a first energy modulating element (e.g., 102A, 102B, 102C, as best shown in FIGS. 10 and 11 ) may be disposed at a first location within the first energy waveguide relay system 82A, 82B, at a second location within the second energy waveguide relay system 82C, 82D, or at a third location between the first energy waveguide relay system 82A, 82B and the second energy waveguide relay system 82C, 82D, where the first energy modulating element is configured to modulate energy passing therethrough, as will be explained in more detail in subsequent figures and discussion.

[0120] In another embodiment, a second energy modulating element (e.g., 102A, 102B, 102C, as best shown in Figures 10 and 11) may be positioned in one of the first, second, or third positions, where the second energy modulating element is configured to modulate energy passing therethrough.

[0121] In another embodiment, the first energy waveguide relay system 82A, 82B may include a first array of energy waveguides configured to direct energy therethrough along multiple energy propagation paths, where the energy waveguides of the first array are positioned at different spatial coordinates, with each energy waveguide directing energy from a respective spatial coordinate to the multiple energy propagation paths along a different direction following a first 4D plenoptic function.

[0122] In yet another embodiment, the second energy waveguide relay system 82C, 82D may include a second array of energy waveguides configured to direct energy therethrough along multiple energy propagation paths, where the energy waveguides of the second array are positioned at different spatial coordinates, with each energy waveguide directing energy from its respective spatial coordinate to the multiple energy propagation paths along a different direction that follows a second 4D plenoptic function.

[0123] In one embodiment, the first and second energy modulation elements (e.g., 102A, 102B, 102C, as best shown in Figures 10 and 11) are located at the same location. In another embodiment, the first and second energy modulation elements are located at different locations.

[0124] In some embodiments, system 80 may further comprise a third energy modulating element (e.g., 102A, 102B, 102C, as best shown in FIGS. 10 and 11 ) disposed at one of a first, second, or third location, the third energy modulating element configured to modulate energy passing therethrough. In one embodiment, the third energy modulating element and at least one of the first and second energy modulating elements are disposed at the same location. In some embodiments, the first, second, and third energy modulating elements are disposed at the same location. In other embodiments, the first, second, and third energy modulating elements are disposed at different locations. In some embodiments, each of the first, second, and third energy modulating elements comprises an LCD, LED, DLP, OLED, LCOS, quantum dot, or other suitable energy modulating element. In another embodiment, at least one of the first energy waveguide relay system and the second energy waveguide relay system is curved. In yet another embodiment, both the first energy waveguide relay system and the second energy waveguide relay system are curved. These embodiments will be described in more detail in the following figures and discussion.

[0125] FIG. 9 illustrates an orthogonal view of a waveguide relay system 90 of energy guides 92A, 92B, 92C, 92D that are both scaled and cylindrically radiated around the eye's axis of rotation, according to one embodiment of the present disclosure. In this embodiment, the energy guides 92A, 92B, 92C, 92D are curved, and the viewer's eye 94 is the axis of rotation. This can be implemented as a non-scaled design, or a scaled and curved design. The curved design can be implemented as a horizontal or vertical cylindrical shape, or formed as a concave or spherical shape. Using this approach, the perceived transparency of the light beam shown to the eye 94 through the relay system 90 can be increased. It should also be noted that the cylindrical or spherical approach allows for a much more efficient number of angular samples per waveguide since the normal to each waveguide is more accurately oriented toward the entrance pupil of the eye 94, thereby more efficiently reproducing the original energy propagation rather than a planar function that propagates rays off-axis toward the periphery of the field of view of the eye 94.

[0126] As depicted in FIG. 9, at least one of the first energy waveguide relay system 92A, 92B and the second energy waveguide relay system 92C, 92D may be curved. Although both energy waveguide relay systems 92 are shown as curved, it will be understood that in one embodiment, the first energy waveguide relay system 92A, 92B may be curved and the second energy waveguide relay system 92C, 92D may be planar. In an alternative embodiment, the first energy waveguide relay system 92A, 92B may be planar, whereas the second energy waveguide relay system 92C, 92D may be curved. The curvature of the energy waveguide relay system 92 may be similarly applied to all systems disclosed herein (e.g., FIGS. 7, 8, and 10-12).

[0127] As described above, in one embodiment, the system 90 may include a first energy waveguide relay system 92A, 92B configured such that energy passing therethrough is directed according to a first 4D plenoptic function, and a second energy waveguide relay system 92C, 92D following the first energy waveguide relay system 92A, 92B, the second energy waveguide relay system 92C, 92D configured such that energy passing therethrough is directed according to a second 4D plenoptic function, the second 4D plenoptic function being an inverse function of the first 4D plenoptic function.

[0128] In one embodiment, a first energy modulating element (e.g., 102A, 102B, 102C, as best shown in FIGS. 10 and 11 ) may be disposed at a first location within the first energy waveguide relay system 92A, 92B, at a second location within the second energy waveguide relay system 92C, 92D, or at a third location between the first energy waveguide relay system 92A, 92B and the second energy waveguide relay system 92C, 92D, where the first energy modulating element is configured to modulate energy passing therethrough, as will be explained in more detail in subsequent figures and discussion.

[0129] In yet another embodiment, a second energy modulating element (e.g., 102A, 102B, 102C, as best shown in FIGS. 10 and 11) may be disposed in one of a first, second, or third location, where the second energy modulating element is configured to modulate energy passing therethrough, and a third energy modulating element (e.g., 102A, 102B, 102C, as best shown in FIGS. 10 and 11) may be disposed in one of a first, second, or third location, where the third energy modulating element is configured to modulate energy passing therethrough. This will be explained in more detail in subsequent figures and discussion.

[0130] In one embodiment, the first energy waveguide relay system 92A, 92B includes a first array of energy waveguides configured to direct energy therethrough along multiple energy propagation paths, where the energy waveguides of the first array are positioned at different spatial coordinates, and each energy waveguide directs energy from a respective spatial coordinate to the multiple energy propagation paths along a different direction that follows a first 4D plenoptic function.

[0131] In another embodiment, the second energy waveguide relay system 92C, 92D includes a second array of energy waveguides configured to direct energy therethrough along multiple energy propagation paths, where the energy waveguides of the second array are positioned at different spatial coordinates, and each energy waveguide directs energy from its respective spatial coordinate to the multiple energy propagation paths along a different direction that follows a second 4D plenoptic function.

[0132] In one embodiment, the first and second energy modulation elements (e.g., 102A, 102B, 102C, as best shown in Figs. 10 and 11) may be located at the same location. In another embodiment, the first and second energy modulation elements may be located at different locations. In some embodiments, the third energy modulation element and at least one of the first and second energy modulation elements may be located at the same location. In other embodiments, the first, second, and third energy modulation elements may be located at the same location. Furthermore, in some embodiments, the first, second, and third energy modulation elements may be located at different locations. In some embodiments, each of the first, second, and third energy modulation elements comprises an LCD, LED, DLP, OLED, LCOS, quantum dot, or other suitable energy modulation element. These embodiments will be described in more detail in the following figures and discussion.

[0133] 10 illustrates an orthogonal view of a waveguide relay system 100 showing the relative positions of each of three possible energy modulation devices 102A, 102B, 102C in relation to energy waveguides 104A, 104B, 104C, 104D, in accordance with one embodiment of the present disclosure. In one embodiment, each pair of energy waveguides (e.g., 104A and 104B, or 104C and 104D) may be configured to form an energy waveguide relay system.

[0134] In order to superimpose a virtual light field onto the real world, as viewed through the energy waveguide relay configuration described above, integration of multiple energy propagation paths is necessary. Energy waveguide relay with three energy devices (transparent OLED 102A, transparent LCD 102B, and transparent OLED 102C, provided for illustrative purposes only) enables true focusing of holographic opacity through transmitted light field energy directing relay. In some embodiments, explicit reference to a particular type of display technology is for illustrative purposes only and is in no way intended to limit the disclosure. The pairing of three specifically mentioned (but not limited to) energy modulation devices 102A, 102B, 102C positioned at or around the focal distance between each of the energy waveguide pairs 104A, 104B, 104C, 104D adds the ability to digitally superimpose variable light field opacity and transmission states for any range of energies within a particular energy region at the focal distance from the viewer's eye 106 between the pairs 104A, 104B, 104C, 104D.

[0135] In one embodiment, the first energy modulation device may include a structure for volumetric spectral modulation of the opaque energy propagation in an active state. The spectral modulation may be further propagated through a combination of other energy modulation device pairs 102A, 102C. A particular modulation device 102A, 102C may provide precise control over the propagation of saturation and transparency. For the purposes of this application, the addition of all three energy modulation devices 102A, 102B, 102C may or may not be used as the only energy modulation devices required for energy propagation. The design may be implemented in any combination, with specific reference to the previous embodiments including planar waveguiding systems, curved, scaled, variable techniques, as disclosed depending on the specific application.

[0136] As depicted in FIG. 10, the system 100 may include a first energy waveguide relay system 104A, 104B configured such that energy passing therethrough is directed according to a first 4D plenoptic function, and a second energy waveguide relay system 104C, 104D following the first energy waveguide relay system 104A, 102A, 104B, the second energy waveguide relay system 104C, 102C, 104D configured such that energy passing therethrough is directed according to a second 4D plenoptic function, which is an inverse function of the first 4D plenoptic function.

[0137] 10, a first modulation element 102A may be an LCD, LED, DLP, OLED, LCOS, quantum dot, or other suitable energy modulation element. The system may include a second modulation element 102B and a third modulation element 102C similar to that of the first modulation element 102A.

[0138] In one embodiment, the first energy modulating element 102A may be disposed within the first energy waveguide relay system 104A, 104B, while the second energy modulating element 102B may be disposed between the first energy waveguide relay system 104A, 104B and the second energy waveguide relay system 104C, 104D. The third energy modulating element 102C may be disposed within the second energy waveguide relay system 104C, 104D. In this embodiment, each of the first, second, and third energy modulating elements 102A, 102B, 102C may be configured to modulate energy passing therethrough.

[0139] Although the first modulation element 102A is shown to be in the first energy waveguide relay system 104A, 104B, the second modulation element 102B is shown to be between the first energy waveguide relay system 104A, 104B and the second energy waveguide relay system 104C, 104D, and the third modulation element 102C is shown to be in the second energy waveguide relay system 104C, 104D, it will be understood that the modulation elements 102A, 102B, 102C may be located anywhere throughout the system 100. For example, all three modulation elements 102 may be located in the first energy waveguide relay system 104A, 104B. In another embodiment, all three modulation elements 102 may be located in the second energy waveguide relay system 104C, 104D. In yet another embodiment, all three modulation elements 102 may be disposed between the first energy waveguide relay system 104A, 104B and the second energy waveguide relay system 104C, 104D. It will be understood by those skilled in the art that various combinations and permutations may be utilized. In some embodiments, the system 100 may further include four or five or more modulation elements 102 depending on the application.

[0140] Similar to that of FIG. 9, in one embodiment, the first energy waveguide relay system 104A, 104B may be curved. In another embodiment, the second energy waveguide relay system 104C, 104D may be curved. In yet another embodiment, both the first energy waveguide relay system 104A, 104B and the second energy waveguide relay system 104C, 104D may be curved. Although two energy waveguide relay systems 104 are shown, it will be understood by those skilled in the art that there may be three or four or more energy waveguide relay systems 104 as needed depending on the application. Furthermore, one or two or any number of these energy waveguide relay systems 104 may be curved or not.

[0141] In one embodiment, the transparent display system 100 comprises a first energy waveguide relay system 104A, 104B configured such that energy passing therethrough is directed according to a first 4D plenoptic function, and a second energy waveguide relay system 104C, 104D following the first energy waveguide relay system 104A, 104B, the second energy waveguide relay system 104C, 104D configured such that energy passing therethrough is directed according to a second 4D plenoptic function, the second 4D plenoptic function being an inverse function of the first 4D plenoptic function. In this embodiment, a first energy modulating element 102A is disposed within a first energy waveguide relay system 104A, 104B, a second energy modulating element 102B is disposed between the first energy waveguide relay system 104A, 104B and the second energy waveguide relay system 104C, 104D, and a third energy modulating element 102C is disposed within the second energy waveguide relay system 104C, 104D, where the first, second, and third energy modulating elements 102A, 102B, 102C are configured to modulate energy passing therethrough.

[0142] In one embodiment, the first energy waveguide relay system 104A, 104B may include a first array of energy waveguides configured to direct energy therethrough along multiple energy propagation paths, where the energy waveguides of the first array are positioned at different spatial coordinates, and each energy waveguide directs energy from a respective spatial coordinate to the multiple energy propagation paths along a different direction following a first 4D plenoptic function.

[0143] In another embodiment, the second energy waveguide relay system 104C, 104D may include a second array of energy waveguides configured to direct energy therethrough along multiple energy propagation paths, the energy waveguides of the second array being positioned at different spatial coordinates, each energy waveguide directing energy from a respective spatial coordinate to the multiple energy propagation paths along a different direction according to a second 4D plenoptic function.

[0144] In some embodiments, the system 100 may further comprise one or more additional energy modulating elements (not shown) disposed at one of a first, second, or third location, the one or more additional energy modulating elements configured to modulate energy passing therethrough. The first location may be within the first energy waveguide relay system 104A, 104B, the second location may be between the first energy waveguide relay system 104A, 104B and the second energy waveguide relay system 104C, 104D, and the third location may be within the second energy waveguide relay system 104C, 104D. In some embodiments, the one or more additional energy modulating elements (not shown) and at least one of the first, second, and third energy modulating elements 102 may be disposed at the same location. In other embodiments, the one or more additional energy modulating elements (not shown) and at least one of the first, second, and third energy modulating elements 102 may be disposed at different locations.

[0145] In one embodiment, three energy modulating elements 102 cumulatively propagate electromagnetic energy that includes a modulated blend of color, transparency, intensity, opacity, and other holographic conditions. Table 1 is provided for reference of several variable opacity and transmission states for a number of spectral values ​​and corresponding exemplary values ​​that each of the three energy modulating elements may exhibit. [Table 1]

[0146] Table 1 illustrates a matrix of potential color values ​​for each of the respective modulation elements in the relay system to generate a volumetric light ray that includes all colors, intensities, transparencies, and opacities in the full light field projection.

[0147] The values ​​indicating on or off refer to the possibility of either configuration, depending on the efficiency of the modulation system to absorb electromagnetic radiation in the configuration. In the case of values ​​indicating X or Y percent, this is because it is necessary to determine the efficiency of the energy modulation device 102A aggregated through the energy modulation device 102C to generate the effective and desired holographic opacity and spectral energy transmission. Y can be off or some percentage of the total desired transmission.

[0148] In some embodiments, the modulation of the energy waveguide relay system includes all sensory energy domains, where the energy modulation devices may include other forms of sensory energy devices, including visual, auditory, gustatory, somatosensory, or other sensory systems disclosed. Additionally, the scope of this system includes bidirectional potential to and through the energy propagation waveguide relay system, which is utilized for additional embodiments including any of such disclosures regarding holographic or sensory energy directionality. In one embodiment, the energy relay system is designed as a bidirectional large format energy directing device with both modulation devices and other energy devices to enable transparent and holographic superposition of sensory energy through the waveguide relay system, where the systems may appear invisible through the transparent system and may include energy sensors or any combination of those utilized for sensory energy propagation, holographic parameters, interactivity, or other applications not otherwise expressly disclosed within this application.

[0149] Note that this approach provides the ability to truly "paint" black or opaque values ​​in volumetric space, where multiple modulation devices and waveguiding systems cause the convergence of energy propagation paths such that projected coordinates in space contain the properties of real-world objects. In an embodiment, interactivity, volumetric masking, or other novel applications may be utilized outside of the disclosed embodiment. This novel approach to recreating real-world reflected points from near-field, augmented, virtual, or other head-mounted displays is a remarkable leap in immersive quality, providing a greater sense of presence than was possible with other approaches.

[0150] As shown in Figure 10 and disclosed in Table 1, the first energy modulating element may include a white opacity, the second energy modulating element may include an additional opacity or color, and the third energy modulating element may include a black opacity. In one embodiment, both the first and third energy modulating elements may be OLEDs 102A, 102C, and the second energy modulating element may be an LCD 102B, so that to present an opaque black color, the first modulating element may be configured in an off state, the second modulating element may be configured in an on state, and the third modulating element may be configured in an on or off state.

[0151] 13 illustrates a single energy waveguide relay element system 100, where a single effective aggregation of four waveguide element functions, including waveguide relay element pairs 104A, 104B, and 104C, 104D, provides efficient energy propagation. Within a subset of propagated energy waves 130A, 130B, 130C, the function defined by the energy waveguide when guided by the 4D function allows precise focusing for a given bundle of rays propagating along the system, where knowledge of the energy field, frequency, or other environmental parameters provides both projection and sensing of energy according to the 4D function.

[0152] With reference to Table 1 and considering Figure 13, one skilled in the art will understand how energy that converges modulation states via 102A, 102B, 102C, thereby traversing propagation paths 130A, 130B, 130C and entering the system adjacent waveguide element 104D at location 131, can propagate bidirectionally through the system. As energy propagates through the system from location 131 to viewed location 106, the energy waves propagating throughout the waveguide functions 104A, 104B, 104C, 104D and energy modulation devices 102A, 102B, 102C exit in directions that are substantially similar between the entrance path 131 and the exit path 132. For the viewed light beam and superimposed information generated by energy modulation elements 102A, 102B, and 102C from viewed location 106 and propagating along 132, the bidirectional propagation of energy remains substantially similar throughout the entire energy waveguide relay system except for location 131.

[0153] In this manner, information can be propagated through a transparent waveguide relay system, and a viewer 106 may receive energy 132 from 131 substantially unmodified from the original energy wave if the modulating elements 102A, 102B, and 102C were devoid of modulation. However, activation of the 4D function induced energy modulating device provides the ability to propagate converging 4D plenoptic opaque states, such that the apparent difference between a truly opaque object and a superimposed virtual object cannot be distinguished even with sufficient calibration and technology maturity.

[0154] In one embodiment, the first and third energy modulating elements may both be OLED 102A, 102C, and the second energy modulating element may be an LCD 102B, so that to present a transparent black color, the first modulating element is configured in an off state, the second modulating element is configured in an off state, and the third modulating element is configured in an off state.

[0155] In one embodiment, the first and third energy modulating elements are both OLED 102A, 102C and the second energy modulating element is LCD 102B, so that to present an opaque red color, the first modulating element is configured in a red-only on state, the second modulating element can be configured in an on or off state, and the third modulating element is configured in an off state.

[0156] In one embodiment, the first and third energy modulating elements may both be OLED 102A, 102C and the second energy modulating element may be LCD 102B, so that to present a transparent red color, the first modulating element is configured in a red-only state for a first percent, the second modulating element is configured in an off state, and the third modulating element is configured in a red-only state for a second percent, the second percentage being different from the first percentage.

[0157] In one embodiment, the first and third energy modulating elements may both be OLED 102A, 102C and the second energy modulating element may be LCD 102B, so that to present an opaque gray color, the first modulating element is configured in an on state at a first percent, the second modulating element is configured in an on state, and the third modulating element is configured in an off state.

[0158] In one embodiment, the first and third energy modulating elements may both be OLED 102A, 102C, and the second energy modulating element may be LCD 102B, so that to present a transparent gray color, the first modulating element is configured in an on state with a first percentage value, the second modulating element is configured in an off state, and the third modulating element is configured in an on state with a second percentage value, the second percentage value being different from the first percentage value.

[0159] In some embodiments, the present disclosure may be explicitly implemented for any size variety, including displays that are not head-mounted in any way. In this manner, the methods and systems presented herein can be utilized to "paint" transparency values ​​with any form of light field, including visual or other sensory display systems.

[0160] In one embodiment for manufacturing efficiency, an additional embodiment proposes coupling either the energy modulation elements 102A and 102B, or 102C and 102B, together, which helps increase the efficiency of calibration and solves many mechanical alignment challenges.

[0161] 11A illustrates an orthogonal view of a configuration 110 for coupling two energy modulation elements 102A, 102B together to increase manufacturing and calibration efficiency, according to one embodiment of the present disclosure. FIG. 11B illustrates an orthogonal view of a configuration 116 for coupling two energy modulation elements 102A, 102B together to increase manufacturing and calibration efficiency, according to one embodiment of the present disclosure.

[0162] As depicted in Figures 11A and 11B, in one embodiment, system 110 (Figure 11A) or 116 (Figure 11B) may include a first energy waveguide relay system 112A, 112B configured such that energy passing therethrough is directed according to a first 4D plenoptic function, and a second energy waveguide relay system 112C, 112D following the first energy waveguide relay system 112A, 112B, where the second energy waveguide relay system 112C, 112D is configured such that energy passing therethrough is directed according to a second 4D plenoptic function, which is an inverse function of the first 4D plenoptic function.

[0163] In one embodiment, the system 110 may include an OLED modulation element 102A, an LCD modulation element 102B, and another OLED modulation element 102C, as well as any combination thereof. As shown in FIG. 11A, the OLED modulation element 102A and the LCD modulation element 102B may be disposed in a first energy guide relay system 112A, 112B, whereas the OLED modulation element 102C may be disposed in a second energy guide relay system 112C, 112D. Conversely, as shown in FIG. 11B, the OLED modulation element 102A may be disposed in a first energy guide relay system 112A, 112B, whereas the LCD modulation element 102B and the OLED modulation element 102C may be disposed in a second energy guide relay system 112C, 112D. It will be understood by those skilled in the art that the modulation elements 102 may be randomly distributed throughout the energy guide relay system 112.

[0164] In one embodiment, the first energy modulation element may be disposed at a first location in the first energy waveguide relay system, at a second location in the second energy waveguide relay system, or at a third location between the first and second energy waveguide relay systems, the first energy modulation element configured to modulate energy passing therethrough. In another embodiment, the second energy modulation element may be disposed at one of the first, second, or third locations, the second energy modulation element configured to modulate energy passing therethrough. In yet another embodiment, the third energy modulation element may be disposed at one of the first, second, or third locations, the second energy modulation element configured to modulate energy passing therethrough. In one example, the third energy modulation element may be disposed at the same location of at least one of the first and second energy modulation elements. In another example, the third energy modulation element may be disposed at a different location than at least one of the first and second energy modulation elements.

[0165] In one embodiment, the first energy waveguide relay system 112A, 112B may include a first array of energy waveguides configured to direct energy therethrough along multiple energy propagation paths, the energy waveguides of the first array being arranged at different spatial coordinates, each energy waveguide directing energy from a respective spatial coordinate to the multiple energy propagation paths along different directions according to a first 4D plenoptic function. The second energy waveguide relay system may include a second array of energy waveguides configured to direct energy therethrough along multiple energy propagation paths, the energy waveguides of the second array being arranged at different spatial coordinates, each energy waveguide directing energy from a respective spatial coordinate to the multiple energy propagation paths along different directions according to a second 4D plenoptic function.

[0166] The second energy modulating element may be located at one of a first, second, or third location, where the second energy modulating element is configured to modulate energy passing therethrough. In one embodiment, the first and second energy modulating elements are located at the same location, however, the first and second energy modulating elements may also be located at different locations.

[0167] In some embodiments, the third energy modulating element may be located at one of the first, second, or third locations, where the third energy modulating element is configured to modulate energy passing therethrough. In one embodiment, the third energy modulating element and at least one of the first and second energy modulating elements are located at the same location, whereas the first, second, and third energy modulating elements are located at the same location or different locations.

[0168] Additional embodiments provide the capability to manufacturers of energy waveguide relay optics via a multi-stage wafer-level bonding process. Due to the resolution requirements of near-field displays and the high pixel density required, approaches similar to other disclosures discussed within this application for coding energy waveguides for holographic super-resolution may further be utilized with sophisticated modifications to the energy waveguide relay design. When using high refresh rate synchronous energy modulation elements, two higher density additional energy waveguides are implemented in the center of each of the two original energy waveguide pairs. The increased density ratio may be a result of the following: D 2 =Nn / Nc

[0169] where D is the increased density ratio required (squared for X and Y, respectively), Nn is the new amount of angular samples per waveguide element desired, and Nc is the current system angular sample value without super-resolution applied. For example, if the current sampling provides 9 samples along X in the current system and 27 are desired, then the density increase is 3× 2 It will be double.

[0170] To determine the number of time series samples required, FPSn=FPSs*D 2

[0171] where FPSn is the resulting frame rate required, FPSs is the frame rate of the original content, and D 2 is the ratio calculated from above. For example, if the source content frame rate is 24 fps and D 2 If is equal to 9, the new sampling frequency may include 216 fps.

[0172] 12 illustrates an orthogonal view of an alternative system 120 to a waveguide relay system that directly reverses light rays so that one or more elements from the system can be eliminated, according to one embodiment of the present disclosure. This system 120 may implement hardware modifications that provide a direct reversal of any propagation path presented to the viewer's eye 124 by utilizing a reflective waveguide relay system 122A, 122B. This may be further advantageous for either an HMD system or an opacity generating device, as the waveguide relay may be eliminated from the system entirely.

[0173] Although various embodiments according to the principles disclosed herein have been described above, it should be understood that these embodiments are presented for illustrative purposes only and are not limiting. Thus, the breadth and scope of the invention(s) should not be limited by any of the exemplary embodiments described above, but should be defined only in accordance with the claims that derive from this disclosure, and their equivalents. Moreover, the above advantages and features are provided in the described embodiments, but do not limit the application of such claims to processes and structures that achieve any or all of the above advantages.

[0174] It should be understood that the principal features of this disclosure can be employed in various embodiments without departing from the scope of the disclosure. Those skilled in the art will recognize, or be able to discover using no more than routine experimentation, many equivalents to the specific procedures described herein. Such equivalents are considered to be within the scope of this disclosure and are covered by the claims.

[0175] Furthermore, the section headings herein are provided for consistency with the teachings under 37 CFR 1.77 or to otherwise provide organizational guidance. These headings are not intended to limit or characterize the invention(s) described in any claim that may originate from this disclosure. Specifically, as an example, even if a heading is entitled "Field of the Invention," such claims should not be limited by the language of this heading to describe the so-called technical field. Furthermore, the description of a technology in the "Background of the Invention" section should not be construed as an admission that the technology is prior art to any invention(s) in this disclosure. The "Summary of the Invention" is in no way intended to characterize the invention(s) described in the claims at issue. Furthermore, references to the singular "invention" in this disclosure should not be used to assert that there is only a single novelty in this disclosure. Multiple inventions may be described according to the limitations of multiple claims originating from this disclosure, and such claims thus define the invention(s) and their equivalents protected thereby. In all instances, the scope of such claims will be considered on their own merits in light of this disclosure, but should not be constrained by the headings set forth herein.

[0176] The words "a" or "an" when used in conjunction with the term "comprising" in the claims and / or specification may mean "one," but it is also consistent with the meanings of "one or more," "at least one," and "one or more than one." The term "or" used in the claims is used to mean "and / or" unless expressly referring to alternatives only or the alternatives are mutually exclusive, but the present disclosure supports the definition referring to alternatives only and "and / or." Throughout this application, the term "about" is used to indicate that a value includes the inherent error variation of the device and the method is used to determine the value, or the variation that exists among the research subjects. Generally, but subject to the foregoing considerations, numerical values ​​herein modified by approximation terms such as "about" can vary from the stated value by at least ±1, 2, 3, 4, 5, 6, 7, 10, 12, or 15%.

[0177] As used in this specification and the claims, the words "comprising" (and any form of comprising, such as "comprise" and "comprises"), "having" (and any form of having, such as "have" and "has"), "including" (and any form of including, such as "includes" and "include"), or "containing" (and any form of including, such as "contains" and "contain") are inclusive or open-ended and do not exclude additional, unrecited elements or method steps.

[0178] Words relating to comparisons, measurements, and timings such as "at the time," "equivalent," "during," "complete," and the like, should be understood to mean "substantially at the time," "substantially equivalent," "substantially during," "substantially complete," and the like, where "substantially" means that such comparisons, measurements, and timings are practical to achieve the desired results implicitly or explicitly stated. Words relating to the relative positions of elements, such as "near," "proximate to," "adjacent to," and the like, are intended to mean close enough to substantially affect the interaction of the respective system elements. Other words of approximation similarly refer to conditions that, when so modified, are not necessarily understood to be absolute or complete, but are understood to be sufficiently close that one of ordinary skill in the art would consider them to warrant specifying the conditions as existing. The extent to which the description may vary will depend on how significant a change can be made and would allow one of ordinary skill in the art to recognize the modified feature as still having the required properties and capabilities of the unmodified feature.

[0179] The term "or combinations thereof" as used herein refers to all permutations and combinations of the listed items preceding the term. For example, "A, B, C, or combinations thereof" is intended to include at least one of A, B, C, AB, AC, BC, or ABC, as well as BA, CA, CB, CBA, BCA, ACB, BAC, or CAB, if the order is important in a particular context. Continuing with this example, combinations including repeats of one or more items or terms, such as BB, AAA, AB, BBC, AAABCCCC, CBBAAA, CABABB, etc., are expressly included. Those skilled in the art will understand that there is typically no limit to the number of items or terms in any combination, unless otherwise clear from the context.

[0180] All of the compositions and / or methods disclosed and claimed herein can be made and executed without undue experimentation in light of the present disclosure.The compositions and methods of the present disclosure are described in terms of preferred embodiments, but it will be apparent to those skilled in the art that various variations can be applied to the compositions and / or methods, and in the steps or sequence of steps of the methods described herein, without departing from the concept, spirit and scope of the present disclosure.All such similar substitutes and modifications that are apparent to those skilled in the art are deemed to be within the spirit, scope and concept of the disclosure as defined by the appended claims.

Claims

[Claim 1] 1. A system comprising: a first energy waveguide relay system configured such that energy passing therethrough is directed according to a first 4D plenoptic function; a second energy waveguide relay system subsequent to the first energy waveguide relay system, the second energy waveguide relay system configured such that energy passing therethrough is directed according to a second 4D plenoptic function, the second 4D plenoptic function being an inverse function of the first 4D plenoptic function; a first energy modulating element disposed at a first location within the first energy waveguide relay system, at a second location within the second energy waveguide relay system, or at a third location between the first energy waveguide relay system and the second energy waveguide relay system, the first energy modulating element configured to modulate energy passing therethrough.

Images