System and method for displaying 3d multimedia

Abstract

A method for a display device begins by receiving data representing a set of two-dimensional (2D) scene layers, where each 2D scene layer has a corresponding predetermined display depth in a focus space, and generating a first hologram pattern for each 2D scene layer of the set of 2D scene layers to create a set of first hologram patterns, wherein each first hologram pattern of the set of first hologram patterns is adapted to place an associated 2D scene layer at an infinite depth. The method continues by converting each first hologram pattern in the set of first hologram patterns into a second hologram pattern using a mathematical lens function to create a set of second hologram patterns, wherein each second hologram pattern is adapted to place an associated 2D scene layer at a corresponding predetermined display depth in the focal space. Finally, the method continues by aggregating the set of second hologram patterns to provide an aggregated hologram pattern.

Description

[0001] Inventor:

[0002] E. Barkley, A. Kacciorowski, T.M. Maresco, R. Starr, G.Y. Belle, and J.S. Colin Technical Field

[0003] This subject matter discloses optical systems and related applications for displaying three-dimensional (3D) media for virtual reality (VR) and augmented / extended reality (AR). Attached Figure Description

[0004] Now refer to the accompanying drawings, which may not be drawn to scale, and in which:

[0005] Figure 1 An exemplary three-dimensional (3D) focal space for an augmented reality device is shown, according to various aspects described herein;

[0006] Figure 2 This is a logic diagram of exemplary methods for generating holographic patterns according to the various aspects described herein;

[0007] Figure 3 Exemplary use of hologram replication in a holographic display system according to various aspects described herein is demonstrated;

[0008] Figure 4 A three-dimensional optical lens assembly is shown, based on the various aspects described herein;

[0009] Figure 5A This is a logic diagram of an exemplary method for displaying multiple two-dimensional image layers in a corresponding selectable depth plane in an augmented reality device, based on various aspects described herein;

[0010] Figure 5B This is a logic diagram of an exemplary method for displaying an image in a selectable depth plane in an augmented reality device using a replicated holographic pattern, according to various aspects described herein.

[0011] Figure 5C This is a logic diagram of an exemplary method for displaying multiple two-dimensional image layers in a corresponding selectable depth plane in an augmented reality device using replicated holographic patterns, according to the various aspects described herein.

[0012] Figure 6A and Figure 6B This is an exemplary logic diagram of an exemplary method for performing visual search in an augmented reality system according to the various aspects described herein;

[0013] Figure 7This is an exemplary schematic block diagram of another embodiment of a system for implementing augmented reality according to the various aspects described herein;

[0014] Figure 8 This is an exemplary schematic block diagram of another embodiment of a system for implementing augmented reality according to the various aspects described herein;

[0015] Figure 9A Exemplary holographic dot pattern thresholding based on dither mask coding is shown according to various aspects described herein;

[0016] Figure 9B An example is shown using a jitter mask designed for blue noise on an input image, based on the various aspects described in this paper;

[0017] Figure 9C Examples of using a jitter mask with a signal window designed in the frequency domain on an input image are shown, according to the various aspects described in this paper;

[0018] Figure 9D Examples of using a jitter mask with a signal window designed in Fourier space on an input hologram are presented, according to the various aspects described in this paper.

[0019] Figure 9E Examples of using jitter masks and anti-jitter masks according to the various aspects described in this article are presented;

[0020] Figure 9F Exemplary uses of mask-based dithering to quantize holograms are demonstrated according to various aspects described herein;

[0021] Figure 10 Exemplary schematic block diagrams illustrating embodiments of an ecosystem for implementing augmented reality according to various aspects described herein;

[0022] Figure 11 Exemplary optical modules for generating and projecting augmented reality content are shown, according to various aspects described herein;

[0023] Figure 12 Exemplary specific implementations of augmented reality glasses according to the various aspects described herein are shown;

[0024] Figure 13 These are schematic block diagrams of exemplary embodiments of systems for implementing augmented reality according to the various aspects described herein;

[0025] Figure 14A Exemplary specific implementations of augmented reality glasses according to the various aspects described herein are shown;

[0026] Figure 14BAnother exemplary embodiment of augmented reality glasses according to the various aspects described herein is shown; and

[0027] Figure 14C Another exemplary implementation of augmented reality glasses based on the various aspects described herein is shown. Detailed Implementation

[0028] Figure 1 An exemplary three-dimensional (3D) focal space 188 of the augmented reality device 182 is shown, wherein a depth plane 184 is configured to be selectable from a near-eye distance 180 to an infinite distance 186. In one example, a second spatial light modulator is adapted to display one or more holographic objects that can be viewed by a user, and an optical system (such as the augmented reality device 182) can be configured to display virtual media content at the selectable depth plane 184, wherein the depth of the depth plane can be selected based on the desired virtual media content to be displayed. In one specific implementation, a holographic lensing function can be applied to the hologram pattern to program the depth of the image displayed in space. In one specific example, the selectable depth plane 184 can be programmed to be represented within the focal space from near to near-eye distance 180 to far to substantially infinite distance 186. In one example, the holographic lensing function can be a mathematical model adapted to simulate a physical lensing function to change the focal plane of the image to be displayed. In one related embodiment, the holographic processor can be configured to apply a holographic lensing function to a holographic pattern for an image to be displayed by the augmented reality device 182 at a desired depth in space. In one specific embodiment, the augmented reality device 182 can be configured to display two-dimensional (2D) content on a single depth plane in space, wherein the depth of the depth plane can be determined using a holographic lensing function. In yet another exemplary embodiment, the augmented reality device 182 can be configured to display two-dimensional (2D) content simultaneously on multiple depth planes, wherein the depth of each depth plane can be determined individually.

[0029] In various instances, stereoscopic display technology can be used to provide content at a fixed, single depth plane in space. In one instance, complex and / or bulky optical components can be implemented to provide stereoscopic display technology capable of providing a single, non-fixed depth plane, despite other undesirable weight and / or volume disadvantages. Therefore, augmented reality devices such as augmented reality glasses, head-up displays, etc., can benefit from reducing and / or eliminating these additional complex and / or bulky components, while allowing images, virtual objects, and virtual scenes to be displayed and / or overlaid at virtually any desired depth in space. In an alternative instance, holographic-based techniques can be used to display images, virtual objects, and virtual scenes at desired depths in space with little or no additional optical components.

[0030] In an alternative implementation and operation, a holographic lensing function can be used to change the depth of virtual media content from infinity to a desired depth in the focal space. In a related example, virtual media content can be computed for display at an infinite relative depth, and then a holographic lensing function is applied to change the display depth of the virtual media content for display at a desired perceived depth. In one example, a holographic lensing function can be used to provide virtual images to cover at virtually any depth in the focal space. In an additional example, a holographic lensing function can be adapted to provide virtual media content to cover multiple depth planes, including providing a full 3D scene. In one example, two-dimensional content can be displayed simultaneously on multiple depth planes, where the relative distance between the depth planes can be small enough that it is assumed the viewer can aggregate individual two-dimensional content at each depth of the multiple depth planes to perceive three-dimensional (3D) virtual objects and / or three-dimensional (3D) virtual scenes. In one specific implementation instance, a holographic processor can be adapted for use in an augmented reality device (such as any augmented reality device disclosed herein) to execute one or more holographic lens functions as part of a computational process for rendering holographic patterns on one or more spatial light modulators of the augmented reality device, for displaying images, virtual objects, and virtual scenes at a desired depth in space using the augmented reality device.

[0031] In one specific implementation and operation applicable to one or more of the examples shown herein, an augmented reality device (such as augmented reality device 182) may be configured to provide a free-space beam path together with an optical combiner element. In one example, the optical combiner element may be any of a semi-transparent angle-selective combiner, an angle-selective semi-transparent mirror, or a beam splitter. In one example, the optical combiner element may be a holographic optical element or a metasurface. In one example, a semi-transparent angle-selective combiner may be an optical component configured to allow light to pass through at a specific angle while reflecting or blocking light at other angles. In another specific example, a semi-transparent angle-selective combiner may be used to selectively transmit or reflect light based on the angle of incidence of the incident ray. In yet another specific implementation, a semi-transparent angle-selective combiner may be integrated as part of the optical system of an augmented reality device (such as augmented reality device 182). In a related example, a semi-transparent angle-selective combiner may be configured together with other optical elements as a holographic optical element, wherein the holographic optical element is configured as a diffractive optical element.

[0032] In another specific embodiment relating to an augmented reality device (such as augmented reality glasses), one or more optical combiner elements may be configured as elements on the surface and / or in the lenses of the augmented reality glasses. In yet another specific embodiment, the optical combiner elements may include one or more translucent reflective coatings adapted for implementation on the surface and / or in the lenses of the augmented reality glasses.

[0033] In one instance, one or more spatial light modulators adapted for use in an augmented reality device (such as any augmented reality device disclosed herein) may be configured to modulate the phase, amplitude, and / or polarization of an incident light beam. In one instance, one or more spatial light modulators may be adapted to modulate the phase of the incident light beam. In another instance, one or more spatial light modulators may be adapted to modulate the amplitude of the incident light beam. In yet another instance, one or more spatial light modulators may be adapted to provide both phase modulation and amplitude modulation of the incident light beam. In yet another related instance, one or more spatial light modulators may be adapted for use in an augmented reality device (such as the augmented reality device disclosed herein) to act as a holographic display. Exemplary specific implementations of a holographic display system include an input plane corresponding to a plane displaying a holographic pattern and an output plane corresponding to a plane in which the viewport of a hypothetical viewer for viewing an image, virtual object, and / or virtual scene is located. In one specific implementation example, a first lens group with a focal length f1 can be positioned between the input plane and the output plane at a distance f1 from the input plane. This first lens group can be adapted to convert the desired holographic pattern from the spatial domain to the spatial frequency domain by converting spatial information at the input plane into frequency components at an intermediate plane (called the Fourier plane). In a related example, the holographic display system can be configured to include a second lens group with a focal length f2, positioned between the input plane and the output plane at a distance f2 from the output plane. In one example, the distance between the first and second lens groups is equal to f1 + f2, where the intermediate plane (also called the Fourier plane) is located between the first and second lens groups at a distance f1 from the first lens group and a distance f2 from the second lens group. In a related example, the second lens group can be adapted to convert the frequency components of the holographic pattern at the intermediate plane back to spatial information at the output plane. In yet another related example, the first and second lens groups can form a 4f optical system. In one specific implementation and operational example, optical modules (such as...) Figure 11The optical module 100 includes one or more optical elements forming a first lens group of a 4F optical system and one or more optical elements forming a second lens group of the 4F optical system together with one or more optical elements external to the optical module. The one or more optical elements external to the optical module and part of the second lens group of the 4F optical system may include one or more optical combiner elements, such as those disclosed herein. In exemplary augmented reality glasses, one or more optical combiner elements part of the second lens group of the 4F optical system may be configured as either holographic optical elements or metasurfaces and adapted as lenses for the augmented reality glasses. In one specific embodiment, one or more filter elements are adapted to the optical module to filter out unwanted signal components, such as noise and / or signal components at conjugate images. In a related embodiment, one or more filter elements are placed at the mid-plane of the 4F optical system formed by the first and second lens groups. In one embodiment, both the first and second lens groups may include a single optical element or a combination of optical elements, wherein the optical element is not limited to a lens but may be any optical element including lenses and / or (freeform) mirrors. In one embodiment, the optical module (such as...) Figure 2 The optical module 100 may have any of the following functions: 1) relaying an image at an input plane corresponding to the plane in which the hologram pattern is displayed to another plane in space; 2) applying magnification or reduction to the image at the input plane so that the image at the output plane is magnified or reduced respectively; 3) filtering out unwanted optical signals, including noise and / or conjugate images.

[0034] In a specific implementation and operational instance, an augmented reality device (such as any augmented reality device disclosed herein) may be adapted to provide techniques for reducing granular interference patterns (speckle) associated with the augmented reality device. In one instance, coherence associated with one or more illumination sources in an optical module may introduce granular interference patterns (speckle) that can degrade the relative quality of virtual media content used for display. In one implementation instance, the optical module of a given augmented reality device may be adapted to include one or more techniques among a variety of techniques for mitigating the effects of speckle. Exemplary speckle reduction techniques include:

[0035] - Use depolarization techniques (such as polarization reversal) to modify the statistical properties of the laser illumination source to introduce the spatial / temporal coherence of the controlled quantity, or use multiple laser sources with different properties;

[0036] - Modulate the frequency or wavelength of the laser illumination source (laser chirp) to decouple speckle sources of different wavelengths;

[0037] - Provide random phase modulation to the optical system to disrupt the coherent properties of the laser illumination source by introducing random phase changes, for example, by using a vibrating or rotating diffuser or by employing a spatial light modulator;

[0038] - Rapidly scan one or more laser illumination sources or holographic imagers to sample different speckle patterns over a time span;

[0039] - Adding a diffuser to the optical path causes the laser illumination source to scatter in different directions, reducing the coherence of the light and thus reducing the visibility of the speckle pattern; and

[0040] - Provide deformable mirrors to introduce time-varying (time) random phase shifts in the illumination source output.

[0041] In another specific implementation and operation applicable to one or more of the examples provided herein, an augmented reality device (such as augmented reality device 182) may be configured with a spatially varying pattern of color filters to provide multicolor virtual media content. In one example, an array of color filters may be formed above the top surface of one or more spatial light modulators in the augmented reality device (such as augmented reality device 182), for example, the top surface of one or more spatial light modulators includes light modulation elements. In a particular example, the color filter array includes a set of sub-regions, each of which is adapted to be: 1) transparent to red light and blocking / absorbing green and blue light; 2) transparent to green light and blocking / absorbing red and blue light; or 3) transparent to blue light and blocking / absorbing red and green light. In one example, the color filter array may be adapted for displaying multicolor virtual media content using a single spatial light modulator integrated circuit (IC).

[0042] In another relevant embodiment, the optical system associated with an augmented reality device (such as augmented reality device 182) can be configured to include multiple spatial light modulators, possibly implemented as integrated circuits, wherein each spatial light modulator can be associated with n separate color channels (such as those used in a red, green, and blue (RGB) color model). In an example related to the RGB color model, a first group of one or more spatial light modulators can be adapted to interact only with red light, a second group of one or more spatial light modulators can be adapted to interact only with green light, and finally a third group of one or more spatial light modulators can be adapted to interact only with blue light, wherein the combined output from the first, second, and third groups of one or more spatial light modulators is configured to generate multicolor virtual media content for display. In another relevant example, time-division multiplexing is used to generate multicolor virtual media content for display, thereby sequentially illuminating one or more spatial light modulators with, for example, red, green, and blue light.

[0043] An exemplary method for displaying virtual media content begins by receiving data representing an image, a three-dimensional (3D) object, and / or a three-dimensional (3D) scene, wherein one or more holographic processors (such as the holographic processors disclosed herein) are configured to compute a holographic pattern based on the received data and provide the computed holographic pattern to one or more spatial light modulators for rendering on the one or more spatial light modulators. Exemplary optical systems (such as optical systems for augmented reality devices) can be configured to apply techniques for minimizing and / or attenuating granular interference patterns (speckles) by modulating the wavelength of a laser illumination source, depolarizing the laser illumination source, randomly modulating the phase of the laser illumination source, or diffusing the laser illumination source.

[0044] Figure 2 This is a logic diagram of an exemplary method for generating a holographic pattern. The method begins at step 200, where the augmented reality device receives data representing 2D media content to be displayed. In a related instance, the 2D media content may be designed for display at a predetermined depth plane in space. In a specific instance, the data representing the 2D media content to be displayed may include information indicating the desired display depth of the 2D media content to be displayed, such as metadata. In an alternative instance, the display depth of the 2D media content to be displayed can be determined by elements associated with the augmented reality device. In yet another specific instance, a user may determine the desired display depth of the 2D media content to be displayed. The method continues at step 201, where a random phase is applied to the data representing the 2D media content. At step 202, the method continues by generating a holographic pattern for display at substantially infinite depth. The method then continues at step 204, where a mathematical lensing function (holographic lensing function) may be applied to the previously determined holographic pattern to change the relative display depth of the image to be displayed to the previously determined desired display depth. The method then continues in step 206, where an aberration correction function is applied to the hologram pattern (as changed in step 204) to correct distortions introduced by optical elements, such as those associated with and / or located outside the optical module. In a related instance, the aberration correction function or another function may be used to correct distortions associated with a hypothetical user viewing 2D media content, including but not limited to correcting distortions associated with a hypothetical viewer's refractive errors (such as astigmatism) to accommodate vision correction. In a related instance, the user's prescription correction may be... Figure 2The input parameters of the method can be those of a holographic processor, to adjust the holographic pattern based on prescription correction. In one specific implementation and operational example, the mathematical lens function of step 204 and the aberration correction function of step 206 can be combined into a single mathematical function and applied as a single step to the holographic pattern provided as in step 202. The method then continues in step 208 by quantizing the holographic pattern according to different optical states of the light modulation elements of one or more spatial light modulator devices using a quantization method (e.g., error diffusion, mask-based dithering, or others), and finally in step 210, a complete holographic pattern can be rendered on one or more spatial light modulator devices, the complete holographic pattern being configured to display 2D media content at a desired display depth in the focal space. In one specific implementation and operational example, a holographic processor (such as any holographic processor disclosed herein) is configured to perform actions for generating... Figure 2 An exemplary method for presenting holographic patterns, wherein the holographic processor is adapted to an augmented reality device, such as augmented reality glasses.

[0045] An exemplary method for displaying an image using a viewing device, such as an augmented reality device, begins by receiving data representing virtual media content and a predetermined depth of focus for the virtual media content to be displayed. The method continues by generating an initial hologram pattern from the data for display at infinity and applying a mathematical lensing function (holographic lensing function) to the initial hologram pattern to move the depth of the image to be displayed from infinity to the predetermined depth of focus. In one instance, the mathematical lensing function may also be adapted to correct distortions introduced by optical elements and / or also correct distortions associated with a hypothetical viewer's refractive error (such as, for example, astigmatism) to accommodate vision correction. In one instance, the hologram pattern may be quantized using quantization techniques, such as, for example, error diffusion or mask-based dithering. Finally, the method continues by rendering the quantized hologram pattern on one or more spatial light modulators configured to display the virtual media content.

[0046] Figure 3The use of hologram replication in a holographic display system is demonstrated. In one example, one or more spatial light modulator devices can be used as a holographic display within an optical module. In one example, a spatial light modulator configured as a holographic display may require a relatively large array of light modulation elements to provide acceptable and / or optimal viewing resolution. In this example, a spatial light modulator device used as a holographic display may require significantly more pixels (light modulation elements) compared to a conventional two-dimensional (2D) display, because multiple pixels in the spatial light modulator (which may be configured as part of an integrated circuit) may be needed to create a single voxel, where a voxel refers to a volume element similar to a pixel (picture element) in a 2D image. In one example, a holographic display system may require a pixel (light modulation element) to content to be displayed with a relative resolution ratio greater than 1:1 compared to a conventional two-dimensional (2D) display system. In one specific implementation, the pixel pitch of the spatial light modulator configured as a holographic display can be adapted to be equal to or less than the wavelength of visible light, or equal to or less than half the wavelength of visible light. In a specific related example, a 16k x 16k optical modulator array can be configured as a spatial light modulator in a 4x4mm space. 2 Implemented on silicon. In yet another specific instance, adding a large number of pixels (light modulation elements) to a spatial light modulator configured as a holographic display can significantly increase the computational requirements for processing holographic patterns (interference patterns).

[0047] In one embodiment and operational example, a holographic display system can be adapted to include an operational mode in which an array of light modulation elements of one or more spatial light modulator devices used as a holographic display can be divided to provide multiple light modulation element subarrays, enabling the rendering of the same or similar holographic pattern on each of the subarrays. In one example, the use of subarrays can facilitate a reduction in the computational requirements of the holographic display system by calculating a holographic pattern for a portion of the total number of pixels (light modulation elements), wherein the holographic pattern calculated for that portion (subarray) is copied to each of the multiple subarrays. In one example, using a subarray of one or more spatial light modulators to calculate a holographic pattern and copying it to other subarrays can reduce the computational requirements of larger arrays while also increasing the contrast of the displayed virtual media content. In one embodiment and operational example, a viewing device (such as the augmented reality device disclosed herein) can be adapted to implement an operational mode in which a holographic pattern for a single subarray of multiple subarrays of one or more spatial light modulators can be calculated by a holographic processor, and the holographic pattern calculated for a single subarray can be copied to the remaining subarrays of the multiple subarrays.

[0048] In a specific relevant example, the array of optical modulation elements of one or more spatial light modulators configured as a holographic display can be divided into four quarters or four subarrays. In this example, a holographic pattern can be calculated for only one quarter of the optical modulation element array (one of the four subarrays), and then the holographic pattern calculated for only one quarter is used to display the holographic pattern in each of the four quarters. In one example, replicating the holographic pattern calculated for a subarray of the optical modulation element array of one or more spatial light modulators configured as a holographic display across the entire array of optical modulation elements of the one or more spatial light modulators configured as a holographic display can enable virtual media content to have a relatively lower resolution and a relatively higher contrast than calculating a holographic pattern for the entire array of optical modulation elements, while facilitating lower relative computational requirements.

[0049] Figure 4A three-dimensional optical lens assembly, including a first lens group (lens group 310) and a second lens group (lens group 302), is shown in an optical relay system 300. In one example, the optical relay system can use a set of optical components (such as lenses, freeform mirrors, etc.) to relay an optical signal from one point to another with minimal distortion or quality loss. In one example, a 4f optical relay system includes two lens groups, a first lens group having a focal length f1 (lens group 310) and a second lens group having a focal length f2 (lens group 302), wherein the first and second lens groups are separated from each other by a distance of (approximately) f1 + f2, where f1 is the focal length of lens group 310 and f2 is the focal length of lens group 302. The first lens group (lens group 310) and the second lens group (lens group 302) of the 4f optical relay system can include different optical components, including but not limited to lenses and freeform mirrors. In one example, the optical relay system 300 includes an input plane containing a holographic pattern 308. In one example, lens group 310 is positioned at a distance of (approximately) f1 from the plane of holographic pattern 308. In one example, lens group 310 “performs” a first Fourier transform, converting holographic pattern 308 from the spatial domain to the spatial frequency domain. In one example, the space between lens group 310 and lens group 302 includes an intermediate plane, such as a Fourier plane 304, which represents the spatial frequency of the converted holographic pattern 308. In one example, the intermediate plane (such as the Fourier plane 304) may be located between the first lens group (lens group 310) and the second lens group (lens group 302), wherein the distance from the first lens group (lens group 310) is f1 and the distance from the second lens group (lens group 302) is f2. In one example, lens group 302 is placed at a distance of (approximately) f2 from the intermediate plane (such as the Fourier plane 304) and “performs” a second Fourier transform, converting the signal from the spatial frequency domain back to the spatial domain. Finally, the processed or transformed signal can be observed at the output plane (e.g., the plane in which the eye box (such as eye box 306) may be located).

[0050] In one specific implementation and operational example, an optical lens assembly (such as optical lens assembly 300) includes a spatial filter used at an intermediate plane (such as Fourier plane 304) formed by a 4f optical relay system to effectively perform noise filtering (such as quantization noise introduced by the discrete number of optical modulation elements of the spatial light modulator) and / or "hide" at least one of the conjugate image. In another related example, the spatial filter may be placed in a plane different from the intermediate plane (Fourier plane 304) within the 4f optical relay system (such as optical lens assembly 300). In one example, the optical module of a viewing device may be configured to provide a 4nf optical relay system, where n = 1, 2, 3, configured with optical components selected from a group of optical components including, but not limited to, lenses and / or (freeform) mirrors. In a clear example, a 4nf optical repeater system includes the formation of an intermediate plane (such as the Fourier plane 304) in which noise (such as quantization noise) can be placed in a region outside the desired signal window and can be filtered out and / or its conjugate image can be filtered out. In the example of Figure 9, the 4nf system is shown with n equal to 1.

[0051] Figure 5AThis is a logical diagram of an exemplary method for displaying a three-dimensional (3D) object and / or a three-dimensional scene with multiple selectable depth planes in a focal space. The method begins at step 320 by receiving 3D object and / or 3D scene data (further referred to as 3D data) for display in the focal space, and continues at step 322 by forming a set of two-dimensional (2D) layers from the 3D data. In one instance, a 3D object or 3D scene can be decomposed into a set of 2D layers by slicing the 3D object or 3D scene at regular or irregular intervals, wherein the 2D layers are parallel to each other, and each slice represents a cross-section of the 3D object or 3D scene; this slicing forms a set of 2D layers. The method then represents the 3D data at step 324 by applying a random phase to each of the 2D layers in this set, and continues at step 326 by generating a holographic pattern for each of the 2D layers in this set for display at infinite depth. In one instance, a holographic pattern can be generated for each of the 2D layers in this set using a Fourier transform function of the data representing the 2D layers. The method proceeds in step 328 by applying a mathematical lensing function (holographic lensing function) to the holographic pattern of each 2D layer (as calculated in step 326) to transform each 2D layer of the set of 2D layers from infinity to a desired depth in the focal space, and then in step 330 by aggregating the transformed holographic patterns of the set of 2D layers into a single holographic pattern. In step 332, the method applies an optical aberration correction function to the single holographic pattern to correct distortions introduced by optical elements configured as part of or located outside the optical module. In one instance, the optical aberration correction function may be adapted to correct distortions associated with a hypothetical viewer's refractive error (such as, for example, astigmatism) to accommodate vision correction. In a related instance, the user's prescription correction may be... Figure 5A The input parameters of the method can be those of a holographic processor to adjust the holographic pattern based on prescription correction. In an alternative example, the mathematical lensing function applied to each of the 2D layers in the set of 2D layers can also be adapted to correct optical distortions introduced by optical components and adapt to vision correction, thereby allowing elimination of step 332. The method continues in step 334 by applying quantization methods, such as, for example, error diffusion or mask-based quantization. Finally, in step 336, the method renders the complete holographic pattern on a spatial light modulator, wherein the spatial light modulator can be adapted to display 3D objects and / or 3D scenes associated with the 3D data in a visually perceptible form based on the complete holographic pattern. In one example, when the data to be displayed includes a 2D image at a predetermined depth plane in the focal space, it can be used... Figure 5A The method, in which step 322 can be eliminated. In another instance, the holographic processor (such as the holographic processor disclosed herein) can be configured to perform... Figure 5A The method.

[0052] Figure 5B This is a logic diagram of an exemplary method for displaying a two-dimensional (2D) image at a selectable depth plane in focal space. The method begins at step 340 by receiving data (such as data including color information for each pixel in the image and the desired display depth of the image (RGBD data), representing a two-dimensional (2D) image to be displayed at a predetermined depth in focal space), and continues at step 341 by applying a random phase to the data. At step 342, the method can generate a holographic pattern from the data as if the 2D image were displayed at infinite depth, wherein the size of the holographic pattern corresponds to a sub-region of the spatial light modulator used to render the holographic pattern. The method continues at step 344 by copying the holographic pattern generated for the sub-region of the spatial light modulator to the entire region of the spatial light modulator, and continues at step 346 by applying a mathematical lensing function (holographic lensing function) to transform the image to be displayed from infinite depth to the desired predetermined depth in focal space. In step 348, the method can be used to correct the holographic pattern for distortions introduced by optical elements configured as part of or outside an associated optical module using an aberration correction function. In one related instance, the aberration correction function can be adapted to accommodate vision correction associated with a specific hypothetical user viewing a two-dimensional (2D) image. In one related instance, the user's prescription correction could be... Figure 5B The input parameters of the method can be those of a holographic processor to adjust the holographic pattern based on prescription correction. In another instance, the mathematical lensing function and aberration correction function are implemented as a single function executed in a single step. In step 350, the method applies a quantization method, such as error diffusion or mask-based quantization, to the holographic pattern of step 348. Finally, in step 352, the method renders the completed holographic pattern on a spatial light modulator, wherein the spatial light modulator can be adapted to display a two-dimensional (2D) image in a visually perceptible form at a predetermined depth in the focal space based on the completed holographic pattern. In one instance, a holographic processor (such as the holographic processor disclosed herein) can be configured to perform... Figure 5B The method.

[0053] Figure 5CThis is a logical diagram of an exemplary method for displaying a three-dimensional (3D) object and / or a three-dimensional (3D) scene with multiple selectable depth planes in a focal space. The method begins at step 360 by receiving 3D object and / or 3D scene data (further referred to as 3D data) for display in the focal space, and continues at step 362 by forming a set of two-dimensional (2D) layers from the 3D data. In one instance, the 3D object or 3D scene can be decomposed into a set of parallel 2D layers by slicing the 3D object or 3D scene at regular or irregular intervals, each slice representing a cross-section of the 3D object or 3D scene, which forms the set of 2D layers. The method then represents the 3D data at step 364 by applying a random phase to each of the 2D layers in the set, and continues at step 366 by generating a holographic pattern for each of the 2D layers in the set for display at infinite depth, wherein the size of each holographic pattern corresponds to a sub-region of a spatial light modulator. The method proceeds in step 368 by copying the holographic pattern of each 2D layer in the set of 2D layers to the entire region of the spatial light modulator using a size equal to that of a sub-region of the spatial light modulator to provide a set of second holographic patterns. Then, in step 370, it continues by applying a mathematical lensing function (holographic lensing function) to each of the second holographic patterns in the set of second holographic patterns to generate a set of third holographic patterns, where each third holographic pattern is associated with a different desired depth. The method continues in step 372 by aggregating the set of third holographic patterns into a single aggregated holographic pattern and in step 374, whereby the method uses an optical aberration correction function to correct distortions introduced by optical elements configured as part of the optical module or optical elements located outside the optical module. In one instance, the optical aberration correction function can be adapted to correct refractive errors (such as, for example, astigmatism) to accommodate the viewer's visual correction. In a related instance, the user's prescription correction could be... Figure 5C The input parameters of the method can be the input parameters of a holographic processor to adjust the holographic pattern based on prescription correction. In an alternative example, the mathematical lens function applied to each of the 2D layers in the set of 2D layers can be adapted to correct optical distortions introduced by optical components and adapt to vision correction, thereby eliminating step 374. Finally, in step 376, the method completes the holographic pattern using a quantization method (such as, for example, error diffusion or mask-based quantization), and in step 378, the method renders the completed holographic pattern on a spatial light modulator, wherein the spatial light modulator can be adapted to display the virtual content media in a visually perceptible form based on the completed holographic pattern. In one example, when the data to be displayed includes a 2D image for display at a predetermined depth plane in the focal space, it can be used... Figure 5CThe method eliminates step 362. In one instance, a holographic processor (such as the holographic processor disclosed herein) can be configured to perform... Figure 5C The method.

[0054] Figure 6A This is a logic diagram of an exemplary method for performing a visual search in an augmented reality system, such as the augmented reality device disclosed herein. The method begins at step 400, where the augmented reality system receives a visual search request. In one instance, the visual search request may be received from a user. In an alternative instance, the visual search request may be received from one or more third parties, from metadata associated with the augmented reality system, or from another source. At step 402, the method continues, where the augmented reality system captures a scene / environment. In one instance, the augmented reality system may include one or more front-facing cameras for capturing the scene (i.e., the cameras capture the user's view of the scene).

[0055] In step 404, the augmented reality system segments the scene into distinct elements and / or objects. In one specific implementation, the augmented reality system includes one or more computer vision algorithms adapted to segment captured scene images into elements and / or objects. In another specific implementation, one or more scene images captured by one or more front-facing cameras can be transmitted wirelessly (e.g., via Bluetooth or Wi-Fi) to another (mobile) electronic device (such as a smartphone, smartwatch, or laptop), enabling the use of external computer vision algorithms to segment the scene images into distinct elements and / or objects. In one instance, the segmented scene images can be transmitted back for use in the augmented reality system. The method continues in step 406, where the augmented reality system determines the depth of one or more objects of interest in the scene. In one instance, the augmented reality system can be implemented using one or more depth sensors adapted to capture the depth or distance of the objects of interest relative to the user of the augmented reality system. In a related specific implementation, the augmented reality system includes one or more gaze or eye-tracking sensors adapted to determine the assumed direction of the user's gaze in order to define the objects of interest. In an alternative implementation, the augmented reality system can be implemented without gaze or eye tracking. In one instance, the object at the center of the user's field of vision will be identified as the object of interest.

[0056] In step 408, the augmented reality system uses an augmented reality overlay to outline one or more objects of interest. In one instance, the augmented reality overlay may be adapted to display at the same depth as the one or more objects of interest. In step 410, the user or the augmented reality system determines whether the object of interest outlined in step 408 is the desired object of interest, and when the desired object of interest has been identified, the method continues in step 412, where object-related sensor data is transmitted to the search engine. In one instance, the user may indicate that the desired object of interest has been identified through one or more of the following: gestures, audible sounds, toggle buttons, touch or swipes on a touchscreen, and / or keystrokes on a keyboard. In one instance, the search engine may be located in another location, where sensor data is transmitted using a wireless link such as, for example, Bluetooth or Wi-Fi. In an alternative instance, the search engine may be located in a mobile device relatively close to the augmented reality system. In yet another alternative instance, the search engine may include an artificial intelligence engine implemented as part of the augmented reality system, implemented in a mobile device, or implemented at a third party. In another instance, an augmented reality system can be connected to another (mobile) electronic device, such as a smartphone, smartwatch, mobile computer, or desktop computer, using a cable, and data can be transferred between the augmented reality system and the (mobile) electronic device via the cable.

[0057] In step 414, the reverse image search result is received. In one instance, the reverse image search result is received at the augmented reality system. In an alternative instance, the reverse image search result is received at a mobile device located near the augmented reality system. In one specific implementation, the reverse image search result may include one or more of the following: contextual information for user use, metadata related to the desired object of interest, or a Uniform Resource Locator (URL). Finally, in step 416, the augmented reality system displays the result using an augmented reality overlay at the sensing depth of the desired object of interest. In one instance, the contextual information can be provided using an augmented reality overlay at the sensing depth of the desired object of interest, such that both the desired object of interest and the contextual information are in focus.

[0058] Another exemplary method begins by receiving a search request, capturing a scene to provide the captured scene, and segmenting the captured scene into multiple elements and / or objects. The method continues by determining the depth of objects in the scene and outlining the object's contour using an overlay. The method then continues by determining whether the object is an object of interest, and in response to the determination that the object is an object of interest, transmitting sensor data associated with the scene to a third party for reverse image search. Finally, the method continues by receiving search results from the third party and displaying information representing the results using an overlay at the depth of the object in the focal space.

[0059] Figure 6B This is a logic diagram of an exemplary method for performing a visual search in an augmented reality system, such as the augmented reality device disclosed herein. The method begins at step 500, where the augmented reality system receives a visual search request. In one instance, the visual search request may be received from a user. In an alternative instance, the visual search request may be received from one or more third parties, from metadata associated with the augmented reality system, or from another source. At step 502, the method continues, where the augmented reality system captures a scene / environment. In one instance, the augmented reality system may include one or more front-facing cameras for capturing the scene (i.e., the cameras capture the user's view of the scene).

[0060] In step 504, the augmented reality system segments the scene into distinct elements and / or objects. In one specific implementation, the augmented reality system includes one or more computer vision algorithms adapted to segment captured scene images into elements and / or objects. In another specific implementation, one or more scene images captured by one or more front-facing cameras can be transmitted wirelessly (e.g., via Bluetooth or Wi-Fi) to another (mobile) device (such as a smartphone, smartwatch, or laptop), enabling the use of external computer vision algorithms to segment the scene images into distinct elements and / or objects. In one instance, the segmented scene images can be transmitted back for use in the augmented reality system.

[0061] In step 506, the augmented reality system determines what the user is looking at. In one specific implementation example, the augmented reality system includes one or more eye or gaze tracking sensors, and an eye or gaze tracking algorithm adapted to determine where the user is looking and / or what the user is looking at. The method continues in step 508, where the augmented reality system determines the depth of one or more objects of interest. In one instance, the augmented reality system may be implemented using one or more depth sensors adapted to capture the depth or distance of the objects of interest relative to the user.

[0062] In step 518, the augmented reality system uses an augmented reality overlay to outline one or more objects of interest. In one instance, the augmented reality overlay may be adapted to display at the same depth as the one or more objects of interest. In a related implementation instance, the augmented reality system may be configured to include one or more gaze or eye-tracking sensors adapted to determine the direction of the user's gaze in order to define the object of interest. In an alternative implementation instance, the augmented reality system may be implemented without any gaze or eye tracking. In one instance, an object at the center of the user's field of vision will be identified as the object of interest. In step 510, the user or the augmented reality system determines whether the object of interest outlined in step 518 is the desired object of interest, and when the desired object of interest has been identified, the method continues in step 512, where object-related sensor data is transmitted to a search engine. In one instance, the user may indicate that the desired object of interest has been identified through one or more of the following: gestures, audible sounds, toggle buttons, touch or swipes on a touchscreen, and / or keystrokes on a keyboard. In one instance, the search engine could be located in another location, where sensor data is transmitted using a wireless link such as Bluetooth or Wi-Fi. In an alternative instance, the search engine could be located in a mobile device relatively close to the augmented reality system. In yet another alternative instance, the search engine could include an artificial intelligence engine, implemented as part of the augmented reality system, either in the mobile device or at a third party. In yet another instance, the augmented reality system could be connected to another (mobile) electronic device, such as a smartphone, smartwatch, mobile computer, or desktop computer, via a cable, and data could be transmitted between the augmented reality system and the (mobile) electronic device via the cable.

[0063] In step 514, the reverse image search result is received. In one instance, the reverse image search result is received at the augmented reality system. In an alternative instance, the reverse image search result is received at a mobile device located near the augmented reality system. In one specific implementation, the reverse image search result may include one or more of the following: contextual information for user use, metadata related to the desired object of interest, or a Uniform Resource Locator (URL). Finally, in step 516, the augmented reality system displays the result using an augmented reality overlay at the sensing depth of the desired object of interest. In one instance, the contextual information can be provided using an augmented reality overlay at the sensing depth of the desired object of interest, such that both the desired object of interest and the contextual information are in focus.

[0064] Another exemplary method begins by receiving a search request, capturing a scene to provide the captured scene, and segmenting the captured scene into multiple elements and / or objects. The method continues by identifying objects, where this identification may be based on information representing the gaze and / or eyes of the user being tracked. The method then continues by determining whether the object is an object of interest, and in response to the determination that the object is an object of interest, transmitting sensor data associated with the scene to a third party for reverse image search. Finally, the method continues by receiving search results from the third party and displaying information representing the results using overlays at the depth of the objects in the focal space.

[0065] Figure 7 This is a schematic block diagram of an embodiment of an augmented reality system 440, including an optical module 424, a holographic processor 438 with associated memory, an application processor 436, a radio transceiver 428, a camera input 430, and a depth sensor 432, as well as a power management unit 422 and one or more batteries 426 constituting the power unit 420. In one example, the holographic processor 438 and one or more spatial light modulator devices 442 may be adapted to compute and / or generate holographic patterns for projection and / or display on an associated augmented reality device, such as the augmented reality system 440. The optical module 442 can be described in more detail above. (See, for example, Reference) Figure 11 (Optical module 100.)

[0066] In one specific implementation example, in addition to the sensors provided above, additional sensors may be adapted for and / or integrated into an augmented reality device, such as augmented reality system 440. In a related example, the augmented reality device (such as augmented reality system 440) may be adapted to provide input to the additional sensors. In various examples, the additional sensors include, but are not limited to, one or more of the following:

[0067] Cameras used to capture the user's surroundings;

[0068] Eye or gaze tracking sensors;

[0069] One or more microphones;

[0070] One or more speakers;

[0071] One or more tactile sensors;

[0072] Global Positioning System (GPS) device;

[0073] One or more accelerometers;

[0074] One or more gyroscopes;

[0075] One or more magnetometers (compasses) for providing navigation compass functionality; one or more pedometers;

[0076] One or more inertial measurement units;

[0077] One or more ambient light sensors;

[0078] One or more thermometers and / or temperature sensors;

[0079] One or more humidity sensors;

[0080] One or more barometer sensors and / or altimeter sensors; and

[0081] One or more touchscreens (or touchpads) (e.g., located on the periphery of AR glasses, such as on the temples).

[0082] In various related instances, sensor data collected by associated augmented reality devices (such as augmented reality system 146) can be used for a variety of functions, including but not limited to image sensing, depth sensing, audio capture, user attention (such as eye tracking), determining geographic location, and local environmental factors (such as those that can be determined based on any of the sensors included above). In another related instance, simultaneous localization and mapping (SLAM) can be used to build and update a map of the user's environment while tracking the user's position within the environment, thus providing yet another source of sensor information.

[0083] In one specific implementation, the optical module 424 may be implemented using a light source 444, which may include one or more individual light sources adapted to provide illumination for the spatial light modulator 442. In another specific implementation, the light source 444 may be adapted to provide light wavelengths in individual color channels, such as those used in the red, green, and blue (RGB) color model. In yet another specific implementation, the light source 444 may be implemented as a single white light emitter, covering wavelengths that essentially cover the entire spectrum.

[0084] In one relevant instance of operation and implementation, the holographic processor 438 may be configured as one or more computing elements adapted to perform processor functions for computing a diffraction (hologram) pattern for display on one or more spatial light modulators embedded in an associated augmented reality device, such as augmented reality system 440. In one instance, the diffraction pattern may include digital media content adapted for observation by a user of augmented reality system 440. In one specific instance, the hologram pattern for display / projection may be generated based on commonly understood functions, such as Fresnel transform-based models, angular spectrum models, point source methods, Gerchberg-Saxton algorithms, Iterative Fourier Transform (IFTA) algorithms, or wavefront propagation algorithms (such as, for example, wavelet transform, wavefront recording and reconstruction, and phase holograms). Additional methods may include using random phase encoding / decoding techniques to represent complex scenes and / or deep learning techniques, such as convolutional neural networks.

[0085] Radio transceiver 428 may include one or more receiver units and / or transmitter units configured to enable the exchange of information between the augmented reality system (such as augmented reality system 440) and wide area networks, local area networks, and / or mobile electronic devices (such as smartphones, smartwatches, mobile computers, or desktop computers). In one example, power supply unit 420 may include a power management unit 422 comprising one or more power management integrated circuits (PMICs) to manage and control the power usage of individual components of augmented reality system 440, thereby providing efficient power consumption and performance.

[0086] In one embodiment, the augmented reality system 440 may include one or more memory devices, each including, for example, read-only memory, random access memory, volatile memory, non-volatile memory, static memory, dynamic memory, flash memory, cache memory, quantum registers or other quantum memories, and / or any other device that stores data in a non-transient manner. In an additional embodiment, the augmented reality system 440 may include one or more control modules or control units configured to drive and / or synchronize various functions associated with the augmented reality system 440. In various embodiments, the one or more control units may be implemented using one or more application processing devices. In yet another embodiment, the optical module 424 may include one or more optical front paths and / or one or more optical back paths. In one embodiment, the optical front path and / or optical back path, together with the light source 444 and the spatial light modulator 442, constitute the optical module 424.

[0087] In one example, the various components included in the augmented reality system 440 (such as the power management unit 442, control unit, holographic processor 438, and radio transceiver 428) can be implemented as separate components on one or more printed circuit boards. In an alternative example, the power management unit 422, control unit, holographic processor 438, and radio transceiver 428 can be implemented on a single integrated circuit or integrated on multiple integrated circuits. Furthermore, in yet another example, the various components of the augmented reality system 440 can be adapted for implementation within the frame of an augmented reality device, such as in the temples (or multiple temples) of the augmented reality device. In one operational example, the integration of some or all of the components of the augmented reality system 440 can be used to reduce the overall footprint of the augmented reality system 440, providing a potentially more compact implementation while potentially improving performance and / or power efficiency. In another specific implementation example, integrating some or all of the components of the augmented reality system 440 can reduce the manufacturing cost of the augmented reality system 440 while providing a lower overall cost for the associated augmented reality viewing device.

[0088] An exemplary optical module for displaying images includes: a first interface for network connectivity, a depth sensor, a second interface for camera connectivity, a light source, a spatial light modulator, a holographic processing module, a combiner mirror, and a memory; and a processing module operatively coupled to the interface and the memory. In one example, the processing module is operable to receive an image of a scene via the second interface, determine the depth of focus of an object using the depth sensor, and transmit information representing the scene and depth of focus to a third party via the first interface. In a further example, the processing module may be further operable to receive information representing an object from a third party and provide the object-representing information to the holographic processing module, wherein the holographic processing module is operable to generate a holographic pattern based on the object-representing information and display the holographic pattern on the spatial light modulator, wherein the spatial light modulator may be configured to display digital media content based on the holographic pattern using the combiner mirror. In various examples, the combiner mirror may be replaced by another semi-transparent angle-selective combiner element, which is implemented as an optical component and configured to allow light to pass through at a specific angle while reflecting or blocking light at other angles. In one specific related example, a translucent angle-selective combiner can be used to selectively transmit or reflect light based on the angle of incidence of the incident ray. In another specific implementation example, the final optical element may include a translucent reflective coating that is adapted as part of a lens of augmented reality glasses.

[0089] Figure 8This is a schematic block diagram of an embodiment of a system for implementing augmented reality. In this example, the augmented reality system 520 includes an optical module 526, a holographic processor 548 with associated memory, an application processor 530, an artificial intelligence engine 534, a radio transceiver 532, a camera input 550, and a depth sensor 552, as well as a power management unit 524 and one or more batteries 528 that together constitute a power supply unit 522. In one example, the optical module 526 may include one or more spatial light modulator devices 542, which are configured together to project and / or display digital media content in an associated augmented reality device (such as augmented reality device 520) to form a holographic pattern displayed and / or rendered using one or more spatial light modulator elements. The optical module 526 can be described in more detail herein. (See, for example, References) Figure 11 In various specific instances, the associated augmented reality device (such as augmented reality system 520) may be configured to include additional sensors and / or inputs for those additional sensors. In various instances, the additional sensors include, but are not limited to, one or more of the following:

[0090] Cameras used to capture the user's surroundings;

[0091] Eye or gaze tracking sensors;

[0092] One or more microphones;

[0093] One or more speakers;

[0094] One or more tactile sensors;

[0095] Global Positioning System (GPS) device;

[0096] One or more accelerometers;

[0097] One or more gyroscopes;

[0098] One or more magnetometers (compasses) for providing navigation compass functionality; one or more pedometers;

[0099] One or more inertial measurement units;

[0100] One or more ambient light sensors;

[0101] One or more thermometers and / or temperature sensors;

[0102] One or more humidity sensors;

[0103] One or more barometer sensors and / or altimeter sensors; and

[0104] One or more touchscreens (or touchpads) (e.g., located on the periphery of AR glasses, such as on the temples).

[0105] In a relevant implementation and operational example, sensors associated with an associated augmented reality device (such as Augmented Reality System 520) may be adapted to capture data for subsequent processing by an artificial intelligence engine (such as a neural network processor and / or inference engine). In one specific example, the artificial intelligence engine may be trained to process sensor data collected by the augmented reality device (such as Augmented Reality System 520) using one or more of the sensors listed above. In one example, the artificial intelligence engine may be embedded in the associated augmented reality device, such as Augmented Reality System 520. In an alternative example, all or part of the sensor data collected by the augmented reality device may be transmitted via a wireless link over the World Wide Web for processing by a remote artificial intelligence engine. In one example, classification results from remote artificial intelligence processing may be transmitted back to the augmented reality device and used, for example, to display contextual information to a user. In one example, the contextual information may include augmented reality overlays of one or more objects relevant to the user. In a relevant implementation and operational example, processed sensor data associated with the augmented reality device (such as Augmented Reality System 520) may be used to provide additional contextual information for image or object search.

[0106] In a specific implementation and operational instance, sensor data from multiple augmented reality devices can be used to train an artificial intelligence engine to generate a trained (neural network) model. In various instances, the trained model can be packaged and sent back to the augmented reality device, such as Augmented Reality System 520 using, for example, standardized formats (such as TensorFlowSavedModel or ONNX (Open Neural Network Exchange)). In another relevant instance, the trained model parameters can be serialized into a file or a set of files using formats such as HDF5, JSON, or a custom binary format, where the artificial intelligence engine in the augmented reality device (such as Augmented Reality System 146) deserializes and loads the model for use by the augmented reality device for inference.

[0107] In an additional instance of specific implementation and operation, the trained (neural network) model may include training from multiple sources (besides augmented reality devices). Examples of other sources include datasets from virtually all relevant resources. In another instance, multiple augmented reality systems may be used together as a crowdsourced training model. In yet another instance, Figure 8The system can be used to provide additional functionality, such as segmenting elements associated with a specific environment and encoding these elements into descriptors for parameterization, with the goal of reducing the relative size of sensor data storage.

[0108] In one specific implementation, the optical module 526 may be implemented using a light source (or multiple light sources) 544, wherein the light source 544 may include one or more individual light sources adapted to provide illumination for the spatial light modulator 542. In another specific implementation, the light source 544 may be adapted to provide light wavelengths in individual color channels, such as those used in the red, green, and blue (RGB) color model. In yet another specific implementation, the light source 544 may be implemented to cover wavelengths across substantially the entire spectrum as, for example, a single white light emitter.

[0109] In one relevant instance of operation and implementation, the holographic processor 438 may be configured as one or more computing elements adapted to perform processor functions for computing a diffraction pattern for display and / or projection onto one or more spatial light modulators 541 embedded in an augmented reality device, such as augmented reality system 440. In one instance, the diffraction pattern may include a holographic pattern adapted to display and / or project digital media content for observation by a user of augmented reality system 440. In one specific instance, the holographic pattern for display / projection may be generated based on commonly understood functions, such as Fresnel transform-based models, angular spectrum models, point source methods, Gerchberg-Saxton algorithms, Iterative Fourier Transform (IFTA) algorithms, or wavefront propagation algorithms (such as, for example, wavelet transform, wavefront recording and reconstruction, and phase holograms). Additional methods may include using random phase encoding / decoding techniques to represent complex scenes and / or deep learning techniques, such as convolutional neural networks.

[0110] Radio transceiver 532 may include one or more receiver units and / or transmitter units configured to enable the exchange of information between the augmented reality system (such as augmented reality system 520) and wide area networks, local area networks, and / or mobile electronic devices (such as smartphones, smartwatches, mobile computers, or desktop computers). In one example, power supply unit 522 may include a power management unit 524 comprising one or more power management integrated circuits (PMICs) to manage and control the power usage of individual components of augmented reality system 520, thereby providing efficient power consumption and performance.

[0111] In one embodiment, the augmented reality system 520 may include one or more memory devices, each including, for example, read-only memory, random access memory, volatile memory, non-volatile memory, static memory, dynamic memory, flash memory, cache memory, quantum registers or other quantum memories, and / or any other device that stores data in a non-transient manner. In an additional embodiment, the augmented reality system 520 may include one or more control modules or control units configured to drive and / or synchronize various functions associated with the augmented reality system 520. In various embodiments, the one or more control units may be implemented using one or more application processing devices. In yet another embodiment, the optical module 526 may include one or more optical front paths and / or one or more optical back paths. In one example, the optical front path and / or optical back path, together with the light source 544 and the spatial light modulator 542, constitute the optical module 526.

[0112] In one instance, the various components included in the augmented reality system 520 (such as power supply unit 522, control unit, holographic processor 548, memory, and radio transceiver 532) can be implemented as separate components on one or more printed circuit boards. In an alternative instance, the power supply unit 522, control unit, holographic processor 548, memory, and radio transceiver 532 can be implemented on a single integrated circuit or integrated onto multiple integrated circuits. Furthermore, in yet another instance, the various components of the augmented reality system 520 can be adapted for implementation within the frame of an augmented reality device, such as in the temples (or multiple temples) of the augmented reality device. In one operational instance, integrating some or all of the integrated circuits / components of the augmented reality system 520 onto a single integrated circuit (e.g., a SoC) can reduce the footprint of the augmented reality system 520, thereby achieving a compact implementation while improving performance and / or power efficiency. In another specific implementation example, integrating some or all of the components of the augmented reality system 520 onto a single integrated circuit (e.g., a SoC) can reduce the manufacturing cost of the augmented reality system 520, while also providing lower costs for the associated augmented reality viewing device.

[0113] An exemplary optical module for displaying images includes: a first interface for network connectivity, one or more sensors, a second interface for camera connectivity, a light source, a spatial light modulator, a holographic processing module, an artificial intelligence engine, a combiner mirror, and a memory; and a processing module operatively coupled to the interface and the memory. In one example, the processing module is operable to receive an image of a scene via the second interface, receive sensor information from one or more sensors, and classify the scene based on the image and sensor information by the artificial intelligence engine to provide a classification result. In one example, the artificial intelligence engine is operable to provide the classification result to the holographic processing module, wherein the holographic processing module is operable to generate a holographic pattern based on information representing an object and display the holographic pattern on the spatial light modulator. In a related example, the spatial light modulator may be configured to display digital media content based on the holographic pattern using the combiner mirror. In various examples, the combiner mirror may be replaced by another semi-transparent angle-selective combiner element, which is implemented as an optical component and configured to allow light to pass through at a specific angle while reflecting or blocking light at other angles. In one specific related example, a translucent angle-selective combiner can be used to selectively transmit or reflect light based on the angle of incidence of the incident ray. In another specific implementation example, the final optical element may include a translucent reflective coating that is adapted as part of a lens of augmented reality glasses.

[0114] In various instances, the holographic pattern can be represented in binary format using a binarization process. In one operational instance, the holographic pattern can be computed using a computer algorithm, such as a computer-generated holography (CGH) algorithm. In a related instance, the holographic pattern can be adapted to align with multiple available optical states provided by the optical modulation elements of a spatial light modulator using a quantization method, thereby enabling the holographic pattern to be mapped to a given spatial light modulator. In one instance, the number of available optical states provided by the optical modulation elements of the spatial light modulator can be finite. In a related instance, the optical modulation elements of the spatial light modulator can employ two or more optical states, each interacting differently with the incident light. In another instance, error diffusion can be used as a quantization method, allowing the quantization error of each pixel of the holographic display to be distributed among neighboring pixels to minimize the impact of the quantization error on the visual quality of the display. In a related instance, the holographic display can be implemented using one or more spatial light modulators. Error propagation algorithms include, but are not limited to, the Floyd-Steinberg algorithm, the Jarvis-Judice-Ninke algorithm, or the Stucki algorithm, each adapted to define a specific pattern for distributing errors to neighboring pixels of the holographic display. In a related instance, a holographic processor (such as the holographic processor disclosed herein) can be adapted to perform the error propagation algorithm on the holographic pattern.

[0115] Figure 9AAn exemplary dithering mask is shown for processing derived from a set of dot patterns, each dot pattern representing a different gray level. Dithering masks can be used in image processing to create depth illusions in images with a finite number of pixel states. In various related instances, dithering masks (such as dithering mask 602) can be used in dithering mask processing (such as dithering mask processing 602) to introduce controlled forms of noise into the image, making the quantization error appear relatively random rather than structured. In one specific implementation and operational example, the exemplary dithering mask can be used to quantize pixel values ​​in a conventional 2D image. In one instance, a small, static dithering mask (e.g., 128 x 128 pixels) can be used to provide a set of thresholds for dithering (quantizing) a conventional 2D image having a size greater than 128 x 128 pixels. In one example, a dithering mask for quantizing a conventional 2D image can be derived from a set of dot patterns, each representing a different gray level. The dot pattern for each desired gray level is designed using simulated annealing to incorporate blue noise characteristics (blue noise contains more energy at higher frequencies and less energy at lower frequencies, making it less perceptible to human vision). In a specific implementation, the dithering process can be configured for use in hardware by comparing, for example, pixel values ​​of a conventional 2D image with a threshold in the dithering mask, thus achieving relatively low computational requirements. In one example, dithering can be enabled for two available states (upper and lower states) when a pixel value of, for example, the conventional 2D image is greater than the corresponding threshold in the dithering mask, causing the pixel to be quantized to the upper state. In this example, when a pixel value of, for example, the conventional 2D image is less than the corresponding threshold in the dithering mask, the pixel is quantized to the lower state. In one relevant example, a smaller dithering mask (e.g., 128 x 128 pixels) can be tiled across an image where the image is larger than the size of the dithering mask. This allows for dithering of relatively large images using a smaller dithering mask, with the advantage of smaller memory requirements for hardware implementation. In another relevant example, because dithering can be achieved by comparing pixel values ​​to a threshold in the dithering mask, each pixel can be acquired in any order, enabling a more flexible hardware design.

[0116] Figure 9BThis demonstrates the use of a dithering mask on a conventional 2D input image, where the dithering mask can be designed to incorporate blue noise characteristics. In one instance, dithering using a dithering mask with blue noise characteristics is suitable for conventional 2D images because the quantization noise can be shifted to higher spatial frequencies where the human visual system can more easily integrate it. In another instance, a given dithering mask can be configured such that the quantization noise of the 2D image can be shifted to relatively high spatial frequencies, where the human visual system can more easily integrate it for visual interpretation. In a specific instance, the dithering mask can be adapted for quantizing holograms. In a specific instance, while the dithering mask used for quantizing conventional 2D images incorporates blue noise characteristics, the dithering mask used for quantizing hologram patterns can be designed to incorporate appropriate characteristics in the frequency domain, where the quantization noise can be shifted to a region outside the desired signal window in the frequency domain. Quantizing hologram patterns using a dithering mask results in relatively low computational requirements in hardware because the process can be a relatively simple comparison operation in hardware.

[0117] Figure 9C The use of a jitter mask designed using a desired signal window in the frequency domain is demonstrated. Figure 9C A dithering mask can be designed to quantize holographic patterns, where quantization noise can be shifted outside the desired signal window in the frequency domain. In one instance, a dithering mask optimized for quantizing holographic patterns may not be optimal for quantizing conventional 2D images because it does not incorporate blue noise characteristics.

[0118] Figure 9D This demonstrates the use of dither masks on holographic patterns, where the dither mask can be designed using a window of the desired signal in the frequency domain. Figure 9DIn one example, the Fast Fourier Transform of the dithered holographic pattern includes a relatively well-defined signal window in the frequency domain, where quantization noise can be largely shifted outside the well-defined signal window. In one example, the dithering mask designed to quantize the holographic pattern may have a size requirement that is substantially the same as the holographic pattern to be quantized. In one example, the real-valued holographic pattern can be normalized such that each pixel of the real-valued holographic pattern maintains a value between -1 and 1 (inclusive). In a related example, a dithering mask can be provided to quantize the pixel values ​​of a normalized real-valued holographic pattern, where each pixel of the normalized real-valued holographic pattern maintains a value between -1 and 1 (inclusive), to -1, or to 1, where the dithering mask can be designed to provide a well-defined signal window in the frequency domain, where quantization noise is largely shifted outside the well-defined signal window. In one example, the dithering mask optimized for quantizing the holographic pattern has the same size as the holographic pattern to be quantized. In one instance, an augmented reality device (such as the augmented reality device disclosed herein) is adapted to use one or more mask-based dithering methods to dither a holographic pattern to be rendered on one or more spatial light modulators integrated as part of the augmented reality device. In a related instance, the dithering mask may be pre-computed and stored on one or more memory devices of the augmented reality device, wherein the dithering mask may be designed to provide a well-defined signal window, wherein quantization noise is largely shifted outside the well-defined signal window.

[0119] Figure 9EAn exemplary use of a dither mask optimized for quantizing a holographic pattern is shown, wherein the size of the dither mask can be smaller than the size of the holographic pattern to be quantized. In one example, the pixels of the holographic pattern are quantized to -1 or 1. In another example, the dither mask can be adapted to quantize the pixels of the holographic pattern to -1 or 1, wherein the dither mask can be smaller than the holographic pattern to be quantized, and wherein the dither mask can be designed to provide a relatively well-defined signal window in the frequency domain, wherein quantization noise is largely shifted outside the well-defined signal window in the frequency domain. The threshold in the dither mask is maintained at a value between -1 and 1 (inclusive). In a related example, the second dither mask can be provided by taking the threshold at the corresponding position in the first dither mask for each position in the second dither mask and inverting the sign. The second dither mask can be obtained from the first dither mask by applying a sign flip to each value in the first dither mask. In a related example, both the first and second dither masks have the same size, smaller than the size of the holographic pattern to be quantized. In one example, the hologram pattern can be divided into four quadrants. The upper-left and lower-right quadrants are quantized by tiling a first dither mask across each of these quadrants. For the upper-right and lower-left quadrants, sign flipping is applied to each value in the upper-right and lower-left quadrants, followed by tiling a second dither mask over the sign-flipped quadrants for quantization, resulting in an incomplete quantized hologram pattern, where sign flipping is applied to the incomplete hologram pattern to achieve a complete quantized hologram pattern in the upper-right and lower-left quadrants of the hologram pattern. In a specific implementation example, dithering is performed in the quadrants such that the continuous real hologram H... r The pixel values ​​can be quantized into a binary hologram H. b Pixel values, such as Figure 9E As demonstrated, acceptable quantization results can be produced on holographic patterns.

[0120] Figure 9F An exemplary use of mask-based dithering to quantize holographic patterns is demonstrated. In this example, a 4096 x 4096 holographic pattern can be quantized using a 256 x 256 dithering mask, thereby providing a well-defined signal window in the frequency domain, where quantization noise is largely shifted outside the well-defined signal window. In one specific implementation and operational example, the dithering mask may require a memory device (256 x 256 x 8 bits in this example), but the dithering operation requires almost no additional computation. In yet another related example, the values ​​of the holographic pattern can be compared with a threshold in the dithering mask in any order.

[0121] An exemplary method includes receiving a holographic pattern representing one of an image, a three-dimensional (3D) object, or a three-dimensional (3D) scene for display. In one instance, the holographic pattern may be spatially divided into four sub-space quadrants. In a related instance, a first dithering mask and a second dithering mask may be used to quantize the holographic pattern, wherein the value of the second dithering mask is the value of the first dithering mask, but with the associated sign reversed. In one instance, the first dithering mask may be applied to two of the four sub-space quadrants, such as, for example, the upper-left and lower-right sub-space quadrants, while the second dithering mask may be adapted for use with the remaining two sub-space quadrants, such as, for example, the upper-right and lower-left sub-space quadrants, wherein the values ​​in these two sub-space quadrants are sign-flipped before tiling across the second dithering mask, resulting in an incompletely quantized holographic pattern in each of the two sub-space quadrants. In one instance, the sign flip can then be applied to each value of the incompletely quantized hologram in each of the two subspace quadrants to obtain a completely quantized hologram for all four quadrants. In a related instance, each of the four subspace quadrants can be aligned with a common first axis and a common second axis, wherein the first and second axes intersect at the four corner points of the four subspace quadrants.

[0122] Figure 10 An exemplary schematic block diagram illustrating an embodiment of an ecosystem for implementing augmented reality is shown. In one example, augmented reality module 104 includes: an optical module 104-3 configured to display images, virtual objects, and / or virtual scenes in conjunction with an associated augmented reality device; a processor 104-2, such as a holographic processor, for processing images, three-dimensional (3D) objects, and three-dimensional (3D) scenes for display using the optical module 104-3; and a wireless transceiver 104-1 for enabling communication with a wireless network, such as a wide area network (WAN) 108.

[0123] In one specific implementation example, augmented reality module 104 may be adapted for use with an augmented reality device, such as augmented reality glasses. In a related example, augmented reality module 104 may be adapted to receive media, such as, but not limited to, one or more of images, partial images, or audiovisual content used with augmented reality glasses. In a specific related but non-limiting example, augmented reality module 104 may be adapted to receive one or more point clouds and / or red, green, blue, and depth (RGBZ) datasets and / or one or more 2D image forms of three-dimensional (3D) data, where each image has an associated depth value (RGBD). In a further related example, media (such as, for example, two-dimensional (2D) images, three-dimensional (3D) objects, and / or three-dimensional (3D) scenes) may be provided to augmented reality module 104 via a wireless network from one or more third-party resources (such as third-party media resource 106), where the wireless network may be one or more of a wide area network (such as WAN 108, wireless local area network (LAN), global network, or cellular network).

[0124] In yet another related example, augmented reality module 104 may be wirelessly coupled to a mobile device, such as mobile device 102. In this example, mobile device 102 may be adapted to provide media such as, for example, two-dimensional (2D) images, three-dimensional (3D) objects and / or three-dimensional (3D) scenes and / or provide processing functions for use with augmented reality module 104.

[0125] Figure 11 An optical module for generating and displaying augmented reality content in viewing devices such as augmented reality glasses is illustrated. The terms "optical module" and "optical engine" are used interchangeably to refer to the same meaning. In one specific embodiment, optical module 100 provides structural components for positioning various optical elements. In one example, the optical module may include one or more illumination sources, such as illuminator 110, configured to provide illumination by a spatial light modulator device (such as spatial light modulator 112). In one specific example, illuminator 110 may be a single light source of a predetermined wavelength. In an alternative example, illuminator 110 may be a light source configured to provide a limited range of wavelengths, wherein the limited range may include multiple wavelengths within the predetermined wavelength range. In another example, illuminator 110 may be adapted to provide a white light source, wherein the white light source may be a combination of visible wavelengths.

[0126] In yet another example, illuminator 110 may be adapted to provide light wavelengths in individual color channels, such as those used in the red, green, and blue (RGB) color model. Other exemplary color models (besides RGB) include, but are not limited to, 1) cyan, magenta, yellow, (primary / black) (CMY(K)); 2) hue, saturation, and lightness (or brightness) (HSV); 3) hue, saturation, and lightness (HSL); 4) YCbCr, which separates lightness (brightness) information (Y) from chromaticity (color) information (Cb and Cr); 5) LAB (and its variants), which can be described as a three-component model (L* (brightness), a* (green to red), and b* (blue to yellow)); and 6) the XYZ color model. In an alternative example, illuminator 110 may be adapted to provide light wavelengths according to a color model that combines four or more individual color channels.

[0127] In one specific implementation and operational example, the spatial light modulator 112 may be one or more spatial modulator devices implemented using one or more integrated circuits. In one example, one or more holographic processors may be configured to compute and / or generate holographic patterns (such as interference patterns) based on the execution of an algorithmic model. In a related example, the holographic pattern may be rendered on one or more spatial light modulators of the spatial light modulator 112 for use with a viewing device (such as augmented reality glasses) to display images, virtual objects, and / or virtual scenes, etc. In one example, the spatial light modulator 112 may be configured to implement a pixel pitch close to or smaller than the wavelength of the light used with the spatial light modulator 112. In one example, the light used with the spatial light modulator 112 may be visible light. In another specific example, one or more spatial light modulator chips may be configured to implement a pixel pitch in the range of half the wavelength or less than half the wavelength of the light used with the spatial light modulator, which, in the example of augmented reality glasses, may be visible light. In one operational example, a spatial light modulator with a pixel pitch close to or smaller than the visible light wavelength can achieve a relatively large field of view (FoV), while a pixel pitch larger than the visible light wavelength can result in a smaller field of view (FoV). In this example, a smaller field of view (FoV) may lead to a poorer immersive experience for the user of the viewing device.

[0128] In one embodiment and operational example, optical module 100 may be configured to guide light emitted from illuminator 110 toward spatial light modulator 112. Exemplary embodiments may include one or more optical elements, such as collimating lenses, in the optical path between illuminator 110 and spatial light modulator 112 to provide collimated light to spatial light modulator 112. In an alternative embodiment, optical module 100 may be configured to provide alignment of illumination provided by illuminator 110 relative to spatial light modulator 112 based on the physical position of illuminator 110 relative to physical position of spatial light modulator 112. In another embodiment and operational example, optical module 100 may be configured to guide a wavefront generated by spatial light modulator 112 toward one or more optical elements located outside optical module 100 using one or more optical elements (such as mirror 114) configured as part of optical module 100, wherein the one or more optical elements located outside optical module 112 may be used to guide the wavefront generated by spatial light modulator 112 and transmitted perceptibly to the user's eye via the path of optical module 100. Examples of one or more optical elements located outside the optical module 100 may include, but are not limited to, reflective and partially reflective optical elements, projection lenses, and polarizing elements. In one example, one or more optical elements located outside the optical module 100 may be a single optical combiner, enabling images, virtual objects, and virtual scenes generated by the optical module 100 to be overlaid onto the real-world environment. In another example concerning augmented reality glasses, one or more optical elements are located outside the optical module 100, wherein the optical elements are configured to guide a wavefront delivered by the optical module 100 to the eyes of a hypothetical user, wherein the optical elements are implemented as part of the lenses of the augmented reality glasses.

[0129] Figure 12 Illustrations are provided for smart / augmented reality (AR) glasses adapted to overlay holographic content onto a real-world environment. In one instance, the smart glasses can be configured to render dynamic and / or full-color holographic content overlaid on a real-world environment in front of the user's eyes.

[0130] In one specific implementation example, the smart glasses can be configured with various exemplary components, such as one or more illumination sources, one or more spatial light modulators, one or more optical subsystems, and one or more computing elements / chips (such as holographic processor devices). In one exemplary embodiment, the smart glasses can be configured with any one of an illumination source, a spatial light modulator, an optical subsystem, and a computing element / chip (such as a holographic processor device) for each eye. Thus, the smart glasses can be configured to include two illumination sources, two spatial light modulators, two optical subsystems, and two computing elements (holographic processor devices). Configurations where the number of illumination sources, spatial light modulators, optical subsystems, and / or computing chips (holographic processor chips) is more or less than two are also possible. In one specific example, the optical module 122 can be configured to combine many of the aforementioned components into a single unit.

[0131] Exemplary smart glasses can be adapted to include additional components such as a control management subsystem, memory, a power management subsystem, one or more embedded batteries, and / or one or more connectors adapted for connection to an external battery. In another example, Figure 12 The various components of the smart glasses can be integrated into the frame of the smart glasses.

[0132] In one specific implementation, the smart glasses configuration may include an illumination source, a spatial light modulator, an optical subsystem, and computing elements (such as a holographic processor device) for each eye in a user's eyes. In one specific implementation, the illumination source (e.g., a miniaturized laser source with coherent light) may be adapted to provide illumination via the spatial light modulator. In a related example, the spatial light modulator includes an array of individually programmable optical pixels adapted to generate a dynamic wavefront for displaying / projecting holographic content. In one example, the pixel size may be designed to be half or less the wavelength of visible light. In one example, the array of optical pixels may be adapted to interact with a portion of the incident beam provided by the illumination source, wherein the amplitude and phase of the resulting light wave generated by the optical pixels may depend on the programming state of the optical pixels.

[0133] In one specific implementation, the optical pixel can be adapted to modulate the amplitude of its generated light wave as a function of the amplitude of the resulting incident beam. In one example, the optical pixel can be adapted to modulate the phase of its generated light wave as a function of the phase of the resulting incident beam. In one example, the optical pixel can be adapted to modulate both the amplitude and phase of its generated light wave as functions of the amplitude and phase of the incident beam.

[0134] In a further example, individual light waves generated by each optical pixel can form a wavefront to display or project holographic content. In one example, a wavefront generated by a spatial light modulator is provided to an optical subsystem. Exemplary optical systems can be configured with a variety of elements, including combinations of lenses (e.g., pancake lenses, metallic lenses, etc.), mirrors (e.g., freeform mirrors), and diffractive optical elements, for (re)guiding, filtering, and / or amplifying / reducing the static and / or dynamic wavefront provided by the spatial light modulator. In a specific example, the wavefront provided by the optical subsystem can be guided to a partial reflector (or partial reflective coating), which can be configured to be located in front of the user's eye and configured for use with lenses (such as optical correction lenses) of a pair of smart glasses. In an operational example, the partial reflector / coating can be adapted to reflect the static and / or dynamic wavefront from the optical subsystem toward the user's eye, allowing the user's eye to capture the wavefront as holographic content.

[0135] In a specific implementation example, suppose the reflector / coating (lens processing 124) in front of the user's eyes can be adapted to be partially reflective, so that incident light from the real-world environment is transmitted through the optical lens to be perceived at the user's eyes. Thus, the user is able to see holographic content overlaid on the real-world environment. In an alternative example, the reflector can be adapted to be substantially reflective, so that virtually no light incident from the real-world environment is received at the user's eyes. In this example, the user will only receive holographic content without it being overlaid on the real-world environment. In yet another alternative example, the mirror can be adapted to switch between a partially reflective state and a fully reflective state. In a more generalized example, the optical lens of the smart glasses can be configured as a corrective lens, for example, in cases where the user would typically wear prescription glasses.

[0136] In various instances, computing chips and / or holographic processor chips can be adapted to execute computer-generated holography (CGH) algorithms. In one instance, a holographic interference pattern computed by one or more computing chips (such as a holographic processor chip) executing a CGH algorithm can be used to determine the programming of optical pixels associated with a spatial light modulator. In one instance, the CGH algorithm can be used to compute a digital hologram of the desired holographic content. In a related instance, a control and management subsystem can be adapted to receive and transmit digital data and / or control subsystems and components to achieve optimal interaction with each other.

[0137] Exemplary smart glasses can be configured to include a control and management subsystem, memory, a power management subsystem, one or more embedded batteries, and / or one or more connectors for connecting to an external battery. In one instance, the control and management subsystem receives and transmits digital data and / or controls various smart glasses systems and components, causing them to interact with each other in a desired manner. In various instances, the memory included in the exemplary smart glasses can be implemented using virtually any electrical storage technology. In a related instance, predetermined / pre-computed holographic information can be adapted for storage in memory, allowing the information to be directly loaded onto the spatial light modulator without additional / excessive computation.

[0138] Exemplary uses of smart glasses (such as those described above) include viewing photos and videos and / or reading text messages presented as if received at the user's eyes. Other exemplary uses include enabling navigation using smart glasses, where information can be presented in front of the user's eyes to aid navigation (e.g., presenting street names or arrows indicating the direction the user should follow to reach a final destination). Another exemplary use includes visual search, which can be used if a user needs information related to physical objects in a real-world environment. In one related instance, smart glasses can be adapted to instruct a given user to receive information about a physical object by, for example, viewing it. In this instance, image recognition can be implemented to identify the physical object the user can view and then input for use in a search engine. In a further related instance, information received from a search engine can be displayed for use at the user's eyes. In one instance, smart glasses can be configured with one or more cameras to enable the capture of the real-world environment the user is viewing and eye tracking to determine the direction the user is looking in.

[0139] In one specific implementation example, the augmented reality glasses 120 are configured with an optical module 122 (such as a reference) coupled to one of the temples of the augmented reality glasses 120. Figure 11 (Optical module 100). In Figure 3In one example, the optical module 122 may be configured to guide the wavefront to one or more lenses of the augmented reality glasses 120. In a specific example, the lenses (or lenses) of the augmented reality glasses 120 may be adapted with “perspective” combiner elements having reflective and / or partial reflective properties, such as lens processing 124, to combine “virtual media content” with a “real-world” scene, thereby allowing the user to see both the virtual media content and the real-world scene simultaneously. In various instances throughout this document, virtual media content may be defined as media delivered for visual perception by one or more spatial light modulators, including but not limited to two-dimensional (2D) images, three-dimensional (3D) objects, and three-dimensional (3D) scenes. In some instances, the virtual media content may need to be displayed on a surface for perceived by the hypothetical user of the augmented reality device; however, “virtual media content” may be intended to encompass both the output of the spatial light modulator (such as the spatial light modulators shown in the various figures described herein) and the output of the spatial light modulator on a display surface.

[0140] In one specific example, lens processing 124 can be adapted to provide a partially reflective surface that balances the transmission and reflection of light, allowing a user to view virtual media content overlaid on a real-world scene transmitted through lens processing 124 via reflections from lens processing 124. Exemplary lens processing includes a holographic optical element coating or a metasurface coating; this coating is adapted to guide the wavefront delivered by the optical module toward one or both eyes of a hypothetical viewer for the viewer to perceive the virtual media content while still being able to see through to perceive the real-world environment. Additional exemplary lens processing includes: 1) a dielectric coating designed to enhance reflectivity at a specific wavelength; 2) a dichroic coating that selectively reflects or transmits light based on color; 3) a beam splitter coating designed to split incident light into two parts (reflecting one part while transmitting the other); 4) an anti-reflective coating used to maximize unwanted glare or ghosting effects in digital media content; 5) a polarizing coating used in combination with a polarizing light source to reproduce digital media content; and 6) a hybrid coating that combines multiple coating types.

[0141] Figure 13 This is an exemplary schematic block diagram of an embodiment of a system for implementing augmented reality, including an optical module 136, a holographic processor 140, a memory, a control unit, a data transceiver 138, and a power unit 132 including a management unit 134 and one or more batteries 130. In one example, the holographic processor 140 may be configured to compute and / or generate holographic patterns to be rendered on one or more spatial light modulators 142 to display virtual media content on an associated augmented reality device, such as augmented reality system 146. The optical module 136 has been described in more detail above. (See, for example, Reference) Figure 2(Optical module 100) and other parts of this document.

[0142] In one specific implementation, the optical module 136 may be implemented using an illumination source 144, which may include one or more individual light sources adapted to provide illumination to the spatial light modulator 142. In another specific implementation, the illumination source 144 may be adapted to provide light wavelengths in individual color channels, such as those used in the red, green, and blue (RGB) color model. In yet another specific implementation, the illumination source 144 may be implemented to cover wavelengths across substantially the entire spectrum as, for example, a single white light emitter.

[0143] In one relevant instance of operation and implementation, the holographic processor 140 may be configured as one or more computing elements, wherein the computing elements may be configured as integrated circuits adapted to perform processing functions for computing holographic patterns rendered on one or more spatial light modulators 142, so that the spatial light modulators deliver virtual media content for display together with an associated augmented reality device, such as augmented reality system 146. In one specific instance, the holographic patterns for rendering on one or more spatial light modulators 142 may be generated based on commonly understood functions, such as Fresnel transform-based models, angular spectrum models, point source methods, Gerchberg-Saxton algorithms, Iterative Fourier Transform (IFTA) algorithms, and / or wavefront propagation algorithms (such as, for example, wavelet transform, wavefront recording and reconstruction, and phase holograms). Additional methods may include using random phase encoding / decoding techniques to represent complex scenes and / or deep learning techniques, such as convolutional neural networks.

[0144] In one relevant example, virtual media content can be configured to be displayed at a single depth plane selectable in space between a near-eye position (relative to the user) and infinity. In one example, a holographic processor can be configured to compute a holographic pattern from an input medium, wherein the input medium comprises two-dimensional (2D) and / or three-dimensional (3D) data, wherein the holographic processor is adapted to perform one or more methods to correct optical aberrations, such as optical aberrations introduced by one or more optical elements configured as part of and / or located outside the optical module, one or more methods for vision correction, and / or one or more methods for minimizing quantization noise in the displayed virtual media content. Data transceiver 138 may include one or more receiver units and / or transmitter units, which may be implemented as one or more integrated circuits configured to enable the exchange of information between an augmented reality system (such as augmented reality system 146) and a wireless network (such as a wide area network (WAN) and / or a local area network (LAN)) and / or between an augmented reality system (such as augmented reality system 146) and a (mobile) electronic device (such as a smartphone, smartwatch, mobile computer, or desktop computer). In one example, power supply unit 132 may include a power management unit 134 comprising one or more power management integrated circuits (PMICs) to manage and control the power usage of augmented reality system 146 in order to provide relatively efficient power consumption and performance of the system.

[0145] In one embodiment, the augmented reality system 146 may include one or more storage devices, including, for example, read-only memory, random access memory, volatile memory, non-volatile memory, static memory, dynamic memory, flash memory, cache memory, quantum registers or other quantum memories, and / or any other device that stores data in a non-transient manner. In one embodiment and operational example, one or more pre-computed holographic patterns are stored on one or more memory devices. In another example, one or more holographic patterns are computed outside the augmented reality system 146 and transmitted to the augmented reality system 146 via a data transceiver 138 for storage on one or more storage devices of the augmented reality system 146. In an additional embodiment, the augmented reality system 146 may include one or more control units or control modules, which may be implemented as one or more integrated circuits configured to drive and / or synchronize various functions of the augmented reality system 146. In various examples, the one or more control units may be implemented using one or more application processing devices. In yet another specific embodiment, optical module 136 may include one or more optical front paths and / or one or more optical rear paths. In one example, the optical front path and / or optical rear path together with illumination source 144 and spatial light modulator 146 constitute optical module 136.

[0146] In one example, the various components included in the augmented reality system 146 (such as the power management unit 134, control unit, holographic processor 140, memory, and data transceiver 138) may be implemented as separate components on one or more printed circuit boards. In another example, the various components of the augmented reality system 146 may be adapted for implementation within the frame of an augmented reality device, such as in the temples (or multiple temples) of augmented reality glasses. In an alternative example, some or all of the power management unit 134, control unit, holographic processor 140, memory, and data transceiver 138 may be implemented on a single electronic chip (such as a system-on-a-chip or SoC) or integrated onto multiple electronic chips. In one operational example, integrating some or all of the components of the augmented reality system 146 (including the power management unit 134, control unit, holographic processor 140, memory, and radio transceiver 138) onto a single electronic chip (such as a system-on-a-chip or SoC) can, for example, reduce the overall footprint of the augmented reality system 146 and / or provide a potentially more compact implementation, while potentially improving performance and / or power efficiency. In another specific implementation example, integrating some or all of the components of the augmented reality system 146 can reduce the manufacturing cost of the augmented reality system 146 on the SoC, resulting in a lower overall cost for the associated augmented reality viewing device.

[0147] Exemplary augmented reality devices include an interface for network connectivity, a lighting source, a spatial light modulator, a holographic processor, an optical combiner element, a memory, and a processing module operatively coupled to the interface and the memory. In one example, the processing module may be operable to receive data via the interface, which may represent a two-dimensional (2D) image, a three-dimensional (3D) object, and / or a three-dimensional (3D) scene, provide the data to the holographic processor, receive a hologram pattern from the holographic processor, and provide the hologram pattern to the spatial light modulator to render the hologram pattern on the spatial light modulator, wherein the data received via the interface can be perceived by a hypothetical user using the optical combiner element. In various examples, the optical combiner element may be a semi-transparent angle-selective combiner element, which is implemented as an optical component and configured to allow light to pass through at a specific angle while simultaneously reflecting or blocking other angles. In a specific related example, the semi-transparent angle-selective combiner may be used to selectively transmit or reflect light based on the angle of incidence of the incident light. In another specific implementation example, the optical combiner element may include a translucent reflective coating that is adapted to be part of a lens of augmented reality glasses.

[0148] In one specific implementation, the illumination source, spatial light modulator, and holographic processor can be implemented within the optical module. In another implementation, the augmented reality device can be implemented as augmented reality glasses, with the optical module coupled to the temples of the glasses.

[0149] In another specific embodiment, the spatial light modulator has a corresponding top surface and a corresponding bottom surface, wherein an illumination source can be configured to guide light to the top surface of the spatial light modulator, and wherein the top surface of the spatial light modulator can be covered with a spatially varying pattern of color filters formed on the top surface. In one specific embodiment, the spatially varying pattern of color filters includes a red filter, a green filter, and a blue filter, wherein the red filter is transparent to red light but blocks / absorbs green and blue light, wherein the green filter is transparent to green light but blocks / absorbs red and blue light, and wherein the blue filter is transparent to blue light but blocks / absorbs red and green light. In one embodiment, the proportions of the red, green, and blue filters in the spatially varying pattern of the color filters can be equal. In another embodiment, the proportions of the red, green, and blue filters in the spatially varying pattern of the color filters can be unequal. In one specific embodiment, the proportion of the red filter in the spatially varying pattern of the color filters can be 1 / 4, the proportion of the green filter can be 2 / 4, and the proportion of the blue filter can be 1 / 4. In one embodiment, a higher proportion of the green filter can be given in the spatially varying pattern of the color filters because the human visual system is more sensitive to green. In another specific embodiment, the proportion of the red filter in the spatially varying pattern of the color filters can be 2 / 6, the proportion of the green filter can be 3 / 6, and the proportion of the blue filter can be 1 / 6. In yet another embodiment, the spatially varying pattern of the color filters includes a red filter, a green filter transparent to a first wavelength of green light, a green filter transparent to a second wavelength of green light different from the first wavelength of green light, and a blue filter, wherein the illumination source of the system is capable of delivering both the first wavelength of green light and the second wavelength of green light. In another example, a spatially varying pattern of color filters can be adapted to the color filters, wherein a first set of color filters may be transparent to a first wavelength of red light, and a second set of color filters may be transparent to a second wavelength of red light, different from the first wavelength; and / or adapted to the color filters, wherein a first set of color filters may be transparent to a first wavelength of blue light, and a second set of color filters may be transparent to a second wavelength of blue light, different from the first wavelength. In all examples herein, the proportions of the different sets of color filters in the spatially varying pattern of the color filters may be equal or unequal.

[0150] In another specific implementation and operational example, the optical module for displaying virtual media content on a viewing device includes an interface, an illumination source, and a wireless transceiver configured to receive data representing two-dimensional (2D) images, three-dimensional (3D) objects, and / or three-dimensional (3D) scenes and / or holographic patterns for rendering on one or more spatial light modulators to display the virtual media content via the optical module. In one example, the optical module includes a spatial light modulator and one or more holographic processors, wherein the illumination source can be configured to direct illumination to the spatial light modulator, and wherein one or more holographic processors are adapted to compute holographic patterns for rendering on the spatial light modulator. In a related example, the optical module can be adapted to use optical combiner elements to generate virtual media content for display, wherein the optical combiner elements (possibly implemented as combiner mirrors) can be configured to combine the virtual media content with a real-world environment such that the virtual media content and the real-world environment are simultaneously visible when the optical combiner elements are viewed. In various instances, the optical combiner element can take the form of a combiner mirror, a holographic optical element, or a metasurface. In one or more instances, the optical combiner element can be a semi-transparent angle-selective combiner element, which is implemented as an optical component and configured to allow light to pass through at a specific angle while simultaneously reflecting or blocking other angles. In one specific related instance, a semi-transparent angle-selective combiner can be used to selectively transmit or reflect light based on the incident angle of the incident ray. In another specific embodiment, the optical combiner element may include a semi-transparent reflective coating adapted as part of a lens of augmented reality glasses.

[0151] Figure 14A An exemplary embodiment of an augmented reality headset / glasses 172 is shown, comprising a spatial light modulator 152, a data transceiver 158, a processor 156, a motherboard 162, and a battery 164 integrated on the temples of the augmented reality glasses. In various instances, some or all of the electronic components may be implemented as integrated circuits, including the augmented reality headset / glasses 172, and additional electronic components also implemented as integrated circuits, which may be combined on a common system-on-chip (SoC) or as two or more electronic chips integrated using advanced packaging technologies. Exemplary packages include multi-chip modules, 3D integrated circuit (3D IC) packages using multi-chip stacking and through-silicon vias (TSVs), and system-in-package (SiP) and stacked-up package (PoP). Any of the foregoing can be used to reduce the footprint of various hardware components while potentially improving the efficiency of the augmented reality system and improving thermal management performance. In one instance, advanced packaging technologies can achieve one or more of lower cost, a compact design, and / or wearing comfort, while increasing the aesthetic appeal of the augmented reality glasses. Figure 5A and Figure 5B In one example, lens processing 160 may be provided to enable a user to perceive virtual media content overlaid on a real-life environment. In another example, lens processing 160 may include one of a holographic optical element or a metasurface adapted as a lens for augmented reality glasses. In one example, lens processing 160, implemented as a holographic optical element or metasurface, may be configured as a coating applied to the lens of augmented reality glasses.

[0152] Figure 14B An exemplary specific implementation of an augmented reality headset / glasses 170 is shown, illustrating an optical module (such as a reference optics module). Figure 2 The optical module 100 includes a spatial light modulator 152 coupled to an auxiliary board 154 (sub-printed circuit board), all of which are implemented in the temples of the augmented reality headset / glasses 170.

[0153] Figure 14C Provided Figure 14B A magnified view of the optical module of the augmented reality headset / glasses 170, wherein the optical module (such as a reference) Figure 11 The optical module 100 includes a spatial light modulator 152 coupled to an auxiliary board 154 (sub-printed circuit board), all of which are implemented in the temples of the augmented reality headset / glasses 170.

[0154] It should be noted that terms such as bit stream, stream, signal sequence, etc. (or their equivalents) used herein have been used interchangeably to describe digital information whose content corresponds to any of a number of desired types (e.g., data, video, speech, text, graphics, audio, etc., any of which may generally be referred to as 'data').

[0155] As may be used herein, the terms “substantially” and “approximately” provide industry-accepted tolerances for the relativity between their corresponding terms and / or items. For some industries, industry-accepted tolerances are less than one percent, and for others, they are 10 percent or higher. Other examples of industry-accepted tolerance ranges are in the range of less than one percent to fifty percent. Industry-accepted tolerances correspond to, but are not limited to, component values, integrated circuit process variations, temperature variations, rise and fall times, thermal noise, dimensions, signal conduction errors, dropped packets, temperature, pressure, material composition, and / or performance metrics. Within the industry, tolerance variations in acceptable tolerances can be greater than or less than a percentage level (e.g., a size tolerance of less than + / - 1%). A certain relativity between items can range from a difference of less than a percentage level to a range of several percentages. Other relativity between items can range from a difference of several percentages to a large difference.

[0156] As may also be used herein, the terms “configured to,” “operably coupled to,” “coupled to,” and / or “coupled” include direct coupling between items and / or indirect coupling between items via intermediate items (e.g., items include, but are not limited to, parts, elements, circuits, and / or modules), wherein, for instances of indirect coupling, the intermediate item does not modify information about the signal but may regulate its current level, voltage level, and / or power level. As may further be used herein, inferred coupling (i.e., one element is inferredly coupled to another element) includes both direct and indirect coupling between two items in the same manner as “coupled to.”

[0157] As may be used further herein, the terms “configured to,” “operable for,” “coupled to,” or “operably coupled to” indicate that an item includes one or more of electrical connections, inputs, outputs, etc., for performing one or more of their corresponding functions when activated, and may further include inferred coupling to one or more other items. As may still be used further herein, the term “associated with” includes direct and / or indirect coupling to a single item, and / or one item embedded in another.

[0158] As may be used herein, the term "favorable comparison" refers to a comparison between two or more items, signals, etc., indicating a favorable relationship that would be obvious to a person skilled in the art based on, for example, the nature of the compared signal / item. As may be used herein, the term "unfavorable comparison" indicates that a comparison between two or more items, signals, etc., fails to provide such a favorable relationship and / or provides an unfavorable relationship. Such an item / signal may correspond to one or more numerical values, one or more measurements, one or more counts and / or proportions, one or more types of data, and / or other information having attributes that can be compared with thresholds, attributes of each other, and / or other information to determine whether a favorable or unfavorable comparison exists. Examples of such a favorable relationship may include: one item / signal being greater than (or greater than or equal to) a threshold, one item / signal being less than (or less than or equal to) a threshold, one item / signal being greater than (or greater than or equal to) another item / signal, one item / signal being less than (or less than or equal to) another item / signal, one item / signal matching another item / signal, one item / signal substantially matching another item / signal within a predefined or industry-accepted tolerance (such as 1%, 5%, 10%, or other ranges), and so on. Furthermore, those skilled in the art will recognize that such comparisons between two items / signals can be performed in different ways. For example, when a favorable relationship is that signal 1 has a larger amplitude than signal 2, then a favorable comparison can be achieved when the amplitude of signal 1 is greater than the amplitude of signal 2 or when the amplitude of signal 2 is less than the amplitude of signal 1. Similarly, those skilled in the art will recognize that comparisons of the opposites or opposites of items / signals and / or other forms of mathematical or logical equivalence can also be used in an equivalent manner. For example, a comparison to determine whether signal X > 5 is equivalent to determining whether -X < -5, and a comparison to determine whether signal A matches signal B can also be performed by determining whether -A matches -B or not A matches not B. As discussed herein, determining whether a particular relationship (favorable or unfavorable) exists can be used to automatically trigger a particular action. Unless otherwise explicitly stated, it can be assumed that this particular condition does not exist, implying that a particular action will not be automatically triggered. In other instances, determining whether a particular relationship (favorable or unfavorable) exists can be used as a basis or consideration for determining whether to perform one or more actions. It should be noted that such a basis or consideration can be considered alone or in combination with one or more other basis or considerations to determine whether to perform one or more actions. In one instance, when multiple basis or considerations are used to determine whether to perform one or more actions, the respective basis or considerations are given equal weight in this determination. In another instance, when multiple basis or considerations are used to determine whether to perform one or more actions, the respective basis or considerations are given unequal weight in this determination.

[0159] As may be used herein, one or more claims may include the phrase "at least one of a, b, and c" in a specific form of this general form or the phrase "at least one of a, b, or c" in the general form, along with elements more or fewer than "a," "b," and "c." In either wording, the phrase is interpreted the same way. Specifically, "at least one of a, b, and c" is equivalent to "at least one of a, b, or c," and both mean a, b, and / or c. As an example, it means: only "a," only "b," only "c," "a" and "b," "a" and "c," "b" and "c," and / or "a," "b," and "c."

[0160] As may also be used herein, a holographic pattern or hologram refers to an interference pattern of light, and a holographic pattern is a diffraction pattern that diffracts incident light. Further, a holographic image refers to the visual result that a viewer can perceive when a holographic pattern is properly illuminated. Therefore, the visual results that a viewer can perceive include two-dimensional (2D) images, two-dimensional (2D) representations, two-dimensional (2D) information, three-dimensional (3D) objects, and three-dimensional (3D) scenes.

[0161] As may also be used herein, the terms “processing module,” “processing circuit,” “processor,” “processing circuit system,” and / or “processing unit” can refer to a single processing device or multiple processing devices. Such processing devices can be microprocessors, microcontrollers, digital signal processors, microcomputers, central processing units, field-programmable gate arrays, programmable logic devices, state machines, logic circuit systems, analog circuit systems, digital circuit systems, and / or any device that manipulates signals (analog and / or digital) based on hard-coded circuit systems and / or operating instructions. A processing module, module, processing circuit, processing circuit system, and / or processing unit can be or further include memory and / or integrated memory elements, which can be a single storage device, multiple storage devices, and / or another embedded circuit system of a processing module, module, processing circuit, processing circuit system, and / or processing unit. Such storage devices can be read-only memory, random access memory, volatile memory, non-volatile memory, static memory, dynamic memory, flash memory, cache memory, and / or any device storing digital information. It should be noted that if a processing module, module, processing circuit, processing circuit system, and / or processing unit comprises more than one processing device, the processing devices may be centrally located (e.g., directly coupled together via wired and / or wireless bus structures) or distributed (e.g., cloud computing via indirect coupling through a local area network and / or wide area network). It should further be noted that if a processing module, module, processing circuit, processing circuit system, and / or processing unit implements one or more of its functions through a state machine, analog circuit system, digital circuit system, and / or logic circuit system, then the memory and / or memory element storing the corresponding operation instructions may be embedded within or outside the circuit system including the state machine, analog circuit system, digital circuit system, and / or logic circuit system. Still, it should be further noted that the memory element may store and process hard-coded and / or operation instructions corresponding to at least some of the steps and / or functions shown in one or more of the figures, and the processing module, module, processing circuit, processing circuit system, and / or processing unit. Such a storage device or memory element may be included in the article of manufacture.

[0162] The foregoing has described one or more embodiments by means of method steps demonstrating the execution of specified functions and their relationships. For ease of description, the boundaries and order of these functional building blocks and method steps are arbitrarily defined herein. Alternative boundaries and orders may be defined, provided that the specified functions and their relationships are properly performed. Therefore, any such alternative boundaries or orders are within the scope and spirit of the claims. Furthermore, for ease of description, the boundaries of these functional building blocks are arbitrarily defined herein. Alternative boundaries may be defined, provided that certain essential functions are properly performed. Similarly, flowcharts may be arbitrarily defined herein to illustrate certain essential functions.

[0163] Within the scope of use, the flowchart boundaries and sequence can be defined in other ways while still performing certain important functions. Therefore, such alternative definitions of functional building blocks and flowchart sequences are within the scope and spirit of the claims. Those skilled in the art will also recognize that the functional building blocks and other illustrative blocks, modules, and components described herein can be implemented as shown in the figures, or by discrete components, application-specific integrated circuits, processors executing appropriate software, etc., or any combination thereof.

[0164] Additionally, a flowchart may include "Start" and / or "Continue" instructions. The "Start" and "Continue" instructions reflect that the presented steps may optionally be incorporated into or otherwise combined with one or more other routines. Furthermore, a flowchart may include "End" and / or "Continue" instructions. The "End" and / or "Continue" instructions reflect that the presented steps may end as described and shown, or optionally be incorporated into or otherwise combined with one or more other routines. In this context, "Start" indicates the beginning of the first presented step, which may precede other activities not explicitly shown. Further, the "Continue" instruction reflects that the presented steps may be performed multiple times, and / or may continue with other activities not specifically shown. Further, while the flowchart indicates a specific order of steps, other orders are equally possible, as long as the principle of cause and effect is maintained.

[0165] The one or more embodiments herein are used to illustrate one or more aspects, one or more features, one or more concepts, and / or one or more instances. Physical embodiments of devices, articles of manufacture, machines, and / or processes may include one or more of the aspects, features, concepts, instances, etc., described with reference to one or more embodiments discussed herein. Furthermore, throughout the figures, embodiments may incorporate functions, steps, modules, etc., with the same or similar names that may use the same or different reference numerals, and such functions, steps, modules, etc., may be the same or similar functions, steps, modules, etc., or may not be the same.

[0166] Unless expressly stated to the contrary, signals arriving at or from any of the elements in the accompanying figures presented herein and / or between such elements may be analog or digital, continuous-time or discrete-time, and single-ended or differential. For example, if a signal path is shown as a single-ended path, it also represents a differential signal path. Similarly, if a signal path is shown as a differential path, it also represents a single-ended signal path. Although one or more specific architectures are described herein, other architectures may also be implemented using one or more data buses not explicitly shown, direct connections between elements, and / or indirect couplings between other elements as recognized by one of ordinary skill in the art.

[0167] The term "module" is used to describe one or more embodiments in the examples. A module implements one or more functions through means such as a processor or other processing device or other hardware, which may include or operate in association with memory storing operation instructions. A module may operate independently and / or in combination with software and / or firmware. Also as used herein, a module may contain one or more submodules, each of which may be one or more modules.

[0168] As may be further used herein, a computer-readable storage device includes one or more memory elements. A memory element can be a single memory device, multiple memory devices, or a set of memory locations within a memory device. Such a memory device can be read-only memory, random access memory, volatile memory, non-volatile memory, static memory, dynamic memory, flash memory, cache memory, quantum registers or other quantum memories, and / or any other means of storing data in a non-transitory manner. Furthermore, a memory device can be in the form of solid-state memory, hard disk drive memory or other disk storage, cloud storage, thumb drives, server memory, computing device memory, and / or other non-transitory media for storing data. Data storage includes temporary storage (i.e., data is lost when power is removed from the memory element) and / or persistent storage (i.e., data is retained when power is removed from the memory element). As used herein, transient media shall mean one or more of the following: (a) wired or wireless media for transmitting data as a signal from one computing device to another for temporary or persistent storage; (b) wired or wireless media for transmitting data as a signal within a computing device from one element of the computing device to another for temporary or persistent storage; (c) wired or wireless media for transmitting data as a signal from one computing device to another for processing by that other computing device; (d) wired or wireless media for transmitting data as a signal within a computing device from one element of the computing device to another for processing by that other element. As used herein, non-transitory computer-readable storage is substantially equivalent to computer-readable storage. Non-transitory computer-readable storage may also be referred to as non-transitory computer-readable storage media.

[0169] One or more functions associated with the methods and / or processes described herein may be implemented via a processing module that operates via non-human “artificial” intelligence (AI) of the machine. Examples of this AI include machines operating via anomaly detection techniques, decision trees, association rules, expert systems and other knowledge-based systems, computer vision models, artificial neural networks, convolutional neural networks, support vector machines (SVMs), Bayesian networks, genetic algorithms, feature learning, sparse dictionary learning, preference learning, deep learning, and other machine learning techniques trained using training data via unsupervised, semi-supervised, supervised, and / or reinforcement learning, and / or other AI. The human mind cannot perform these AI techniques, not only due to their complexity but also due to the fact that artificial intelligence, as defined by itself, requires “artificial” intelligence (i.e., machine / non-human intelligence).

[0170] One or more functions associated with the methods and / or processes described herein may be implemented as a large-scale system operable to receive, transmit, and / or process data at scale. As used herein, "large-scale" refers to the large volume of data being received, transmitted, and / or processed, such as one or more kilobytes, megabytes, gigabytes, terabytes, or even more. This reception, transmission, and / or processing of data is practically impossible for the human mind to perform at scale within reasonable time periods such as one second, one millisecond, one microsecond, in real time, or other high speeds required by machines that generate, receive, transport, store, and / or use data.

[0171] One or more functions associated with the methods and / or processes described herein may require manipulating data in different ways over overlapping time spans. The human mind cannot independently, simultaneously, in parallel, and / or in a coordinated manner perform these different data manipulations within reasonable time periods such as one second, one millisecond, one microsecond, in real time, or other high speeds required by machines that generate, receive, transport, store, and / or use data.

[0172] One or more functions associated with the methods and / or processes described herein may be implemented in a system operable to electronically receive and / or transmit digital data via a wired or wireless communication network. This reception and transmission cannot actually be performed by the human mind, as the human mind cannot electronically transmit or receive digital data, let alone transmit and receive digital data via a wired or wireless communication network.

[0173] One or more functions associated with the methods and / or processes described herein may be implemented in a system operable to electronically store digital data in a memory device. This storage is practically impossible to perform by the human mind, as the human mind cannot electronically store digital data.

[0174] One or more functions associated with the methods and / or processes described herein can operate to cause a processing module to take an action directly in response to a triggering event—without any human intervention between the triggering event and the action. Any such action can be identified as being performed "automatically," "automatically based on," and / or "automatically in response to" such triggering events. Furthermore, any such behavior identified in this manner explicitly excludes the operation of human activity associated with these behaviors—even if the triggering event itself may be causally related to some human activity.

[0175] While specific combinations of various functions and features of the one or more embodiments have been explicitly described herein, other combinations of these features and functions are equally possible. This disclosure is not limited to the specific instances disclosed herein, and these other combinations are expressly incorporated. Claims (as amended under Article 19 of the Treaty) 1. A method for a display device, the method comprising: Receive data representing a set of two-dimensional (2D) scene layers, each 2D scene layer having a corresponding predetermined display depth in the focus space; Generate a first hologram pattern for each of the set of 2D scene layers to create a set of first hologram patterns, wherein each of the set of first hologram patterns is adapted to place the associated 2D scene layer at an infinite depth; Each of the first holographic patterns in the set of first holographic patterns is transformed into a second holographic pattern using a mathematical lens function to create a set of second holographic patterns, wherein each second holographic pattern is adapted to place the associated 2D scene layer at a corresponding predetermined display depth in the focal space; The set of second holographic patterns is aggregated to provide an aggregated holographic pattern. 2. The method of claim 1, wherein the data is generated from at least one of a three-dimensional (3D) object or a three-dimensional (3D) scene. 3. The method according to claim 1, further comprising: A random phase is applied to each of the set of 2D scene layers. 4. The method of claim 1, further comprising: The aggregated hologram pattern is corrected using an aberration correction function. 5. The method of claim 1, further comprising: The aggregated hologram pattern is transformed using a vision correction function, wherein the vision correction function is adapted to modify the aggregated hologram pattern to correct for visual impairment. 6. The method of claim 1, wherein the mathematical lens function includes at least one of an aberration correction function or a vision correction function. 7. The method of claim 1, further comprising: The quantization function is used to finally determine the aggregated holographic pattern. 8. The method of claim 7, wherein the quantization function is based on either error diffusion quantization or mask-based quantization. 9. The method of claim 7, wherein the quantized holographic pattern is rendered on one or more spatial light modulator devices of the display device. 10. The method of claim 1, wherein each of the set of first holographic patterns is generated using a Fourier transform. 11. A method for a display device, wherein the display device includes at least one spatial light modulator means configured to render one or more holographic patterns, the method comprising: The at least one spatial light modulator device is divided into a set of sub-regions; Generate a holographic pattern of the first sub-region within the set of sub-regions; The holographic pattern of the first sub-region is rendered on each sub-region of the set of sub-regions. 12. The method of claim 11, wherein each sub-region in the set of sub-regions has a width equal to the width of each other sub-region in the set of sub-regions, and wherein each sub-region further has a length equal to the length of each other sub-region in the set of sub-regions. 13. A method for a display device, the method comprising: Receive data representing media intended for display at a predetermined display depth in the focus space; A first holographic pattern is generated from the data, wherein the first holographic pattern has a size corresponding to a portion of a spatial light modulator, and wherein the first holographic pattern is adapted to place the medium at an infinite depth; A second holographic pattern is created by placing two or more copies of the first holographic pattern in an array, wherein the second holographic pattern is configured to have a width that is a multiple of the width of the first holographic pattern, and wherein the second holographic pattern is further configured to have a length that is a multiple of the length of the first holographic pattern. The second hologram pattern is converted into a third hologram pattern using a mathematical lensing function, wherein the third hologram is adapted to place the medium for display at a predetermined display depth in the focal space. 14. The method of claim 13, further comprising: A random phase is applied to the data representing the medium before the first hologram pattern is generated. 15. The method of claim 13, further comprising: The third hologram pattern is corrected using an aberration correction function. 16. The method of claim 13, further comprising: The third hologram pattern is transformed using a vision correction function, wherein the vision correction function is adapted to modify the third hologram pattern to correct for visual impairment. 17. The method of claim 13, wherein the mathematical lens function further comprises one of an aberration correction function or a vision correction function. 18. The method of claim 13, further comprising: The third hologram pattern is ultimately determined using a quantization function to provide a quantized hologram pattern. 19. The method of claim 18, wherein the quantization function is based on either error diffusion quantization or mask-based quantization. 20. The method for a display device according to claim 18, wherein the display device includes one or more spatial light modulator devices, and wherein the quantized holographic pattern is rendered on the one or more spatial light modulator devices. 21. The method of claim 12, wherein the first holographic pattern is generated using a Fourier transform. 22. A method for a display device, wherein the display device includes at least one spatial light modulator device, the method comprising: Receive data representing a set of two-dimensional (2D) scene layers, wherein the 2D scene layers have a corresponding predetermined display depth in the focus space; A first holographic pattern is generated for each of the set of 2D scene layers to create a set of first holographic patterns, wherein each first holographic pattern places the associated 2D scene layer at an infinite depth; wherein each first holographic pattern has a size corresponding to a portion of the at least one spatial light modulator device; For each of the first holographic patterns in the set of first holographic patterns, two or more replicas of the first holographic pattern are placed in an array to create a second holographic pattern, wherein the width of each second holographic pattern is a multiple of the width of the associated first holographic pattern, and wherein the length of each second holographic pattern is a multiple of the length of the associated first holographic pattern. Each of the second holographic patterns in the set of second holographic patterns is transformed using a mathematical lensing function to create a set of third holographic patterns, wherein each third holographic pattern is adapted to place the associated 2D scene layer at a corresponding predetermined display depth in the focal space; The set of third holographic patterns is aggregated to provide an aggregated holographic pattern. 23. The method of claim 22, wherein each of the set of first holographic patterns has a width equal to the width of each other first holographic pattern in the set of first holographic patterns, and wherein each first holographic pattern further has a length equal to the length of each other first holographic pattern in the set of first holographic patterns. 24. The method of claim 22, wherein the data is generated from at least one of a three-dimensional (3D) object or a three-dimensional (3D) scene. 25. The method of claim 22, further comprising: A random phase is applied to each of the set of 2D scene layers. 26. The method of claim 22, further comprising: The aggregated hologram pattern is corrected using an aberration correction function. 27. The method of claim 22, further comprising: The aggregated hologram pattern is transformed using a vision correction function, wherein the vision correction function is adapted to modify the aggregated hologram pattern to correct for visual impairment. 28. The method of claim 22, wherein the mathematical lens function further comprises at least one of an aberration correction function or a vision correction function. 29. The method of claim 22, further comprising: The aggregated hologram pattern is ultimately determined using a quantization function to provide a quantized hologram pattern. 30. The method of claim 29, wherein the quantization function is based on at least one of error diffusion quantization or mask-based quantization. 31. The method for a display device according to claim 29, wherein the display device includes one or more spatial light modulator devices, and wherein the quantized holographic pattern is adapted for rendering on the one or more spatial light modulator devices. 32. The method of claim 22, wherein the first holographic pattern in the set of first holographic patterns is generated using a Fourier transform. 33. A method for execution by one or more processing modules of one or more computing devices of a mobile device, the method comprising: Receive visual search requests; Capture the scene in response to the visual search request; The scene is divided into multiple elements; Determine the relative depth of one or more of the plurality of elements; Facilitate the display of indicators associated with one or more of the said one or more elements; Determine whether the indicator is associated with the element of interest; In response to the determination that the indicator is associated with an element of interest, the sensor data associated with the element of interest is transmitted to the processing device; Receive reverse image search results from the processing device; and Promote the display of indicators representing the reverse image search results. 34. The method of claim 33, wherein the segmentation of the scene into multiple elements is performed by one or more processing devices, wherein the one or more processing devices are external to the mobile device, and wherein the segmented scene data is adapted for transmission from the one or more processing devices to the mobile device. 35. The method of claim 33, wherein the indicator associated with the element of interest is the outline of the element of interest. 36. The method of claim 35, wherein the contour represents an augmented reality overlay. 37. The method of claim 33, wherein the indicator associated with the element of interest is adapted to be displayed at a predetermined depth relative to the depth of the element of interest. 38. The method of claim 37, wherein the relevant depth is the same as the depth of the element of interest. 39. The method of claim 33, wherein the indicator representing the reverse image search result includes contextual information relating to the element of interest. 40. The method of claim 33, wherein the indicator representing the reverse image search result is adapted to be displayed at a predetermined depth relative to the depth of the element of interest. 41. The method of claim 40, wherein the relevant depth is the same as the depth of the element of interest. 42. The method of claim 33, wherein the processing device is a remote server. 43. The method of claim 33, wherein the mobile device is an augmented reality device. 44. The method of claim 43, wherein the mobile device is further configured to collect metadata. 45. The method of claim 44, further comprising: transmitting the metadata to the processing device. 46. ​​The method of claim 44, wherein the metadata is captured by one or more sensors associated with the mobile device. 47. The method of claim 44, wherein the metadata includes at least one of descriptive metadata, geospatial metadata, or contextual metadata. 48. A method for execution by one or more processing modules of one or more computing devices of a mobile device, the method comprising: Receive visual search requests; Capture the scene in response to the visual search request; The scene is divided into multiple elements; Potential elements of interest are determined from the plurality of elements based on the tracking element; Determine the relative depth of the potential elements of interest; Facilitate the display of indicators associated with the potential elements of interest; Determine whether the indicator is associated with the actual element of interest; In response to the determination that the indicator is associated with the actual element of interest, sensor data associated with the actual element of interest is transmitted to the processing device; Receive reverse image search results from the processing device; and Promote the display of indicators representing the reverse image search results. 49. The method of claim 48, wherein the tracking element is at least one of an eye-tracking sensor or a gaze-tracking sensor. 50. The method of claim 48, wherein the segmentation of the scene into multiple elements is performed by one or more processing devices, wherein the one or more processing devices are external to the mobile device, and wherein the segmented scene data is adapted for transmission from the one or more processing devices to the mobile device. 51. The method of claim 48, wherein the indicator associated with any of the potential elements of interest is the outline of the potential elements of interest. 52. The method of claim 48, wherein the contour represents an augmented reality overlay. 53. The method of claim 48, wherein the indicator associated with the potential element of interest is displayed at a predetermined depth relative to the depth of the potential element of interest. 54. The method of claim 53, wherein the predetermined depth is the same depth as the potential element of interest. 55. The method of claim 48, wherein the indicator representing the reverse image search result includes contextual information relating to the element of interest. 56. The method of claim 48, wherein the indicator representing the reverse image search result is adapted to be displayed at a predetermined depth relative to the depth of the element of interest. 57. The method of claim 56, wherein the predetermined depth is the same as the depth of the element of interest. 58. The method of claim 45, wherein the processing device is a remote server. 59. The method of claim 45, wherein the mobile device is an augmented reality device. 60. The method of claim 45, wherein the mobile device is further configured to collect metadata. 61. The method of claim 60, further comprising: transmitting metadata to the processing apparatus. 62. The method of claim 60, wherein the metadata includes at least one of descriptive metadata, geospatial metadata, or contextual metadata. 63. The method of claim 60, wherein the metadata is captured by one or more sensors associated with the mobile device. 64. An augmented reality device comprising: An interface used for network connectivity; light source; One or more optical elements; Memory; A spatial light modulator, wherein the spatial light modulator is configured to render a light interference pattern to provide a rendered light interference pattern, wherein the rendered light interference pattern is adapted for displaying media content at a predetermined distance relative to a user; and A processing module that is operatively coupled to the interface and the memory. 65. The augmented reality device of claim 64, wherein the processing module is further operable to: Data representing the optical interference pattern is received via the interface; The data is provided to the spatial light modulator for rendering; and Promotes the illumination of the rendered light interference pattern by the light source. 66. The augmented reality device of claim 64, wherein the processing module is further operable to: Receive data representing media to be displayed via the interface; The data is then transferred to another processing module; Receive the optical interference pattern from the other processing module; The light interference pattern is provided to the spatial light modulator for rendering; and Promotes the illumination of the rendered light interference pattern by the light source. 67. The augmented reality device of claim 64, wherein the processing module is configured to execute a computer-generated holography (CGH) algorithm. 68. The augmented reality device of claim 67, wherein the CGH algorithm comprises at least one of the following: Fourier transform algorithm, Fresnel transform algorithm, Iterative Fourier Transform (IFTA) algorithm, point cloud-based algorithm, angular spectrum-based algorithm, or lookup table (LUT)-based algorithm. 69. The augmented reality device of claim 64, wherein the rendered light interference pattern is associated with at least one of a two-dimensional (2D) image, a two-dimensional (2D) representation, two-dimensional (2D) information, a three-dimensional (3D) object, or a three-dimensional (3D) scene. 70. An optical display system comprising: One or more spatial light modulators are configured to display holographic images that can be viewed by a user; One or more lighting sources; and One or more optical elements. 71. The optical display system of claim 70, wherein the holographic image comprises at least one of a two-dimensional (2D) image, a two-dimensional (2D) representation, two-dimensional (2D) information, a three-dimensional (3D) object, or a three-dimensional (3D) scene. 72. The optical display system according to claim 70, further comprising: One or more holographic processing modules are configured to execute one or more computer-generated holography (CGH) algorithms. 73. The optical display system according to claim 72, wherein the one or more CGH algorithms include at least one of Fourier transform algorithm, Fresnel transform algorithm, Iterative Fourier Transform (IFTA) algorithm, point cloud-based algorithm, angular spectrum-based algorithm, or lookup table (LUT)-based algorithm. 74. The optical display system of claim 70, wherein the spatial light modulator comprises an array of light modulation elements. 75. The optical display system according to claim 74, wherein the pitch between two adjacent optical modulation elements of the optical modulation element array is equal to or less than the wavelength of a predetermined visible light wavelength. 76. The optical display system of claim 74, wherein each optical modulation element of the optical modulation element array modulates at least one of the amplitude, phase, or polarization of the light incident on the optical modulation element. 77. The optical display system according to claim 70, further comprising: One or more displays, wherein the display of the one or more displays is at least one of a head-mounted display, a head-up display, a stereoscopic display, or a holographic display. 78. A method for execution by one or more processing modules of one or more computing devices, the method comprising: Generate a quantization mask, wherein the quantization mask is adapted to shift noise associated with the quantization process outside a predetermined signal window in the frequency domain. The continuous hologram is quantized based on the quantization mask to generate a quantized hologram. 79. The method of claim 76, wherein quantizing the continuous hologram comprises comparing the value of the continuous hologram pattern with the corresponding value of the quantization mask. 80. The method of claim 76, wherein the quantization mask is configured to have the same size as the continuous hologram. 81. The method of claim 76, wherein the quantization mask is configured to have a size smaller than that of the continuous hologram. 82. The method of claim 79, wherein the quantization mask is copied over the continuous hologram to quantize the continuous hologram to create a quantized hologram. 83. A method for execution by one or more processing modules of one or more computing devices, the method comprising: A first quantization mask is generated, wherein the first quantization mask has a size smaller than that of a continuous hologram, and wherein the first quantization mask is adapted to move noise associated with the quantization process outside a predetermined signal window in the frequency domain; A second quantization mask is generated, wherein the second quantization mask has a size smaller than that of the continuous hologram, and wherein the second quantization mask is adapted to move noise associated with the quantization process outside a predetermined signal window in the frequency domain; The first quantization mask and the second quantization mask are used to quantize the continuous hologram to generate a quantized hologram. 84. The method of claim 83, wherein the second quantization mask is different from the first quantization mask. 85. The method of claim 83, wherein quantizing a continuous hologram into a quantized hologram using the first quantization mask and the second quantization mask comprises comparing the value of the continuous hologram with a corresponding value of the first quantization mask or the second quantization mask. 86. The method of claim 83, wherein the second quantization mask is configured to have the same width as the first quantization mask and is configured to have the same length as the first quantization mask. 87. The method of claim 83, wherein the first quantization mask and the second quantization mask are copied alternately along the horizontal or vertical direction of the continuous hologram, or along both the horizontal and vertical directions. 88. A method for execution by one or more processing modules of one or more computing devices, the method comprising: Receive a continuous hologram, wherein the continuous hologram is divided into a pixel array, wherein each pixel is associated with a value corresponding to a value in the continuous hologram; The pixel value of each pixel in the pixel array is quantized into one of a plurality of states using a quantization mask, wherein the quantization mask is configured to facilitate the shifting of noise associated with the quantization process outside a predetermined signal window in the frequency domain. 89. The method according to claim 88, wherein the method comprises: Adjust the continuous hologram so that the value of the continuous hologram is between -1 and 1, including -1 and 1; The continuous hologram is divided to provide four quadrants sharing four points, the first quadrant including a portion of the pixel array, wherein each of the second quadrant (upper left quadrant), the third quadrant (lower left quadrant), and the fourth quadrant (lower right quadrant) includes another equally sized portion of the pixel array; Generate a first quantization mask, wherein each value of the first quantization mask is between -1 and 1, including -1 and 1; Invert the sign of each value in the first quantization mask to provide a second quantization mask; The first quantization mask is used above the second quadrant and the fourth quadrant to quantize the second quadrant and the fourth quadrant of the continuous hologram; Reverse the sign of each value in the first and third quadrants of the continuous hologram to provide a sign-reversed first quadrant and a sign-reversed third quadrant; The second quantization mask is used over the first quadrant and the third quadrant of the sign inversion of the continuous hologram to provide the quantized first quadrant and the quantized third quadrant of the sign inversion. Reversing the sign of each value in the first quadrant and the third quadrant of the quantized sign inversion provides a quantized hologram of the first and fourth quadrants of the continuous hologram. 90. The method of claim 89, wherein the first quantization mask and the second quantization mask have dimensions smaller than the dimensions of any of the four quadrants, the method comprising: The first quantization mask is replicated above the second quadrant and above the fourth quadrant to quantize the second and fourth quadrants of the continuous hologram, providing quantized holograms of the second and fourth quadrants. Reverse the sign of each value in the first and third quadrants of the continuous hologram to provide a sign-reversed first quadrant and a sign-reversed third quadrant; The second quantization mask is copied above the first quadrant of the sign inversion and above the third quadrant of the sign inversion to quantize the first quadrant of the sign inversion and the third quadrant of the sign inversion, providing the quantized first quadrant of the sign inversion and the quantized third quadrant of the sign inversion. Reversing the sign of each value in the first quadrant and the third quadrant of the quantized sign inversion provides a quantized hologram of the first and fourth quadrants of the continuous hologram. 91. A method for a display device, the method comprising: Receive data representing media intended for display at a predetermined display depth in the focus space; A first holographic pattern is generated from the data, the first holographic pattern placing the media at infinite depth; The first hologram pattern is transformed into a second hologram pattern using a mathematical lensing function, the second hologram pattern placing the medium for display from the infinite depth to the predetermined display depth in the focal space. 92. The method of claim 91, wherein the media for displaying at a predetermined display depth in the focal space comprises one of a two-dimensional (2D) image, a two-dimensional (2D) representation, or two-dimensional (2D) information. 93. The method of claim 91, further comprising: Before generating the first hologram pattern, a random phase is applied to the data representing the medium to display it at a predetermined display depth in the focal space. 94. The method of claim 91, wherein the mathematical lens function further comprises one of an aberration correction function or a vision correction function. 95. The method of claim 91, further comprising: The quantization function was used to finally determine that the second hologram pattern was a quantized hologram pattern. 96. The method of claim 95, wherein the quantization function is based on either error diffusion quantization or mask-based quantization. 97. The method for a display device according to claim 95, wherein the display device includes one or more spatial light modulator devices, and wherein the quantized holographic pattern is rendered on the one or more spatial light modulator devices. 98. The method of claim 95, wherein the first holographic pattern is generated using a Fourier transform. 99. An optical display system comprising: One or more spatial light modulators; One or more optical modules; and One or more optical combiners. 100. The optical display system of claim 99, wherein the one or more spatial light modulators comprise an array of light modulation elements, wherein each light modulation element in the array is individually addressable to control the state of the light modulation element, wherein each light modulation element can exhibit at least two different states, each state having different optical properties. 101. The optical display system of claim 100, wherein the pitch between two adjacent optical modulation elements of the optical modulation element array is equal to or less than the wavelength of the light incident on the one or more spatial optical modulators. 102. The optical display system of claim 100, wherein the pitch between two adjacent optical modulation elements of the optical modulation element array is equal to or less than half the wavelength of the light incident on the one or more spatial light modulators. 103. The optical display system of claim 100, wherein the optical modulation elements of the optical modulation element array are configured to modulate either the amplitude or the phase of the polarization of light incident on the optical modulation elements. 104. The optical display system of claim 100, wherein the optical modulation element of the optical modulation element array of the one or more spatial light modulators comprises: Phase change materials; Heater element; Two or more electrodes connected to the heater element, wherein the optical modulation element is configured to be individually addressable via the two or more electrodes; The state of the optical modulation element is configured to change by altering the state of the phase change material in response to a thermal contribution from the heater element. 105. The optical display system according to claim 99, wherein the optical module in the one or more optical modules comprises: Lighting solutions; The first set of optical elements; and The second set of optical elements. 106. The optical display system of claim 105, wherein the illumination scheme comprises: One or more illumination sources and optical components, wherein the optical components are configured to transmit light from the one or more illumination sources to the one or more spatial light modulators. 107. The optical display system of claim 106, wherein the one or more illumination sources are RGB laser sources. 108. The optical display system of claim 106, wherein the optical component comprises at least one of the following: Beam splitter; Total internal reflection (TIR) ​​prism; Freeform prism. 109. The optical display system of claim 106, wherein the optical component includes a collimator element configured to provide collimated illumination to the one or more spatial light modulators. 110. The optical display system of claim 106, wherein at least one of the first group of optical elements or the second group of optical elements of the optical module comprises one or more achromatic lenses. 111. The optical display system of claim 105, wherein at least one of the first group of optical elements or the second group of optical elements of the optical module comprises at least one of the following: lens; Mirror; Prism; Beam splitter; Optical filters; Polarizer. 112. The optical display system of claim 105, wherein the optical module in the one or more optical modules comprises: One or more optical corrector elements, wherein the one or more optical corrector elements are configured to correct aberrations introduced by optical elements associated with the optical module. 113. The optical display system of claim 112, wherein the one or more optical corrector elements comprise at least one of the following: toroidal optical elements; or Freeform surface optical elements. 114. The optical display system of claim 99, wherein the optical combiner in the one or more optical combiners comprises at least one of the following: Holographic optical elements; Metasurface; Semi-reflective element. 115. The optical display system of claim 105, wherein a first set of optical elements of the one or more optical modules forms a first lens group of a 4f optical system, and wherein a second set of optical elements, together with the one or more optical combiners, is configured to form a second lens group of the 4f optical system. 116. The optical display system of claim 115, wherein the first lens group of the 4f optical system is adapted to perform a Fourier transform on information representing a hologram at the plane of the one or more spatial light modulators by converting information at the plane of the one or more spatial light modulators from the spatial domain to the spatial frequency domain, wherein the spatial frequency information is at the Fourier plane of the 4f optical system, and wherein the second lens group of the 4f system is adapted to perform an inverse Fourier transform on the spatial frequency information at the Fourier plane by converting the spatial frequency information at the Fourier plane from the frequency domain to the spatial domain. 117. The optical display system of claim 99, wherein the one or more optical modules comprise: Lighting solutions; A first set of optical elements, wherein the first set of optical elements forms a first lens group of a 4f optical system, and wherein one or more optical combiners form a second lens group of the 4f optical system. 118. The optical display system of claim 99, wherein the one or more optical modules further comprises one or more optical filter elements. 119. The optical display system of claim 99, wherein the one or more optical filter elements are configured to filter out unwanted frequency-based information at the Fourier plane. 120. The optical display system according to claim 99, further comprising one or more processing units. 121. The optical display system of claim 120, wherein the one or more processing units are implemented on a system-on-a-chip. 122. The optical display system of claim 120, wherein the one or more processing units are configured to execute one or more computer-generated holography (CGH) algorithms to generate one or more holographic patterns for rendering using the one or more spatial light modulators. 123. The optical display system according to claim 122, wherein the CGH algorithm includes at least one of the following: Fourier transform algorithm, Fresnel transform algorithm, Iterative Fourier Transform (IFTA) algorithm, point cloud-based algorithm, angular spectrum-based algorithm, or lookup table (LUT)-based algorithm. 124. The optical display system of claim 120, wherein the one or more processing units are adapted to modify one or more holographic patterns for rendering using the one or more spatial light modulators to provide a modified holographic pattern, wherein the modified holographic pattern is adapted to correct aberrations introduced by the one or more optical modules or by the one or more optical combiners. 125. The optical display system of claim 120, wherein the one or more processing units are adapted to modify one or more holographic patterns for rendering using the one or more spatial light modulators to provide a modified holographic pattern, wherein the modified holographic pattern is adapted to correct refractive errors or visual impairment associated with a user of the optical display system.

Claims

1. A method for a display device, the method comprising: Receive data representing a set of two-dimensional (2D) scene layers, each 2D scene layer having a corresponding predetermined display depth in the focus space; Generate a first hologram pattern for each of the set of 2D scene layers to create a set of first hologram patterns, wherein each of the set of first hologram patterns is adapted to place the associated 2D scene layer at an infinite depth; Each of the first holographic patterns in the set of first holographic patterns is transformed into a second holographic pattern using a mathematical lens function to create a set of second holographic patterns, wherein each second holographic pattern is adapted to place the associated 2D scene layer at a corresponding predetermined display depth in the focal space; The set of second holographic patterns is aggregated to provide an aggregated holographic pattern.

2. The method of claim 1, wherein the data is generated from at least one of a three-dimensional (3D) object or a three-dimensional (3D) scene.

3. The method according to claim 1, further comprising: A random phase is applied to each of the set of 2D scene layers.

4. The method of claim 1, further comprising: The aggregated hologram pattern is corrected using an aberration correction function.

5. The method of claim 1, further comprising: The aggregated hologram pattern is transformed using a vision correction function, wherein the vision correction function is adapted to modify the aggregated hologram pattern to correct for visual impairment.

6. The method of claim 1, wherein the mathematical lens function includes at least one of an aberration correction function or a vision correction function.

7. The method of claim 1, further comprising: The quantization function is used to finally determine the aggregated holographic pattern.

8. The method of claim 7, wherein the quantization function is based on either error diffusion quantization or mask-based quantization.

9. The method of claim 7, wherein the quantized holographic pattern is rendered on one or more spatial light modulator devices of the display device.

10. The method of claim 1, wherein each of the set of first holographic patterns is generated using a Fourier transform.

11. A method for a display device, wherein the display device includes at least one spatial light modulator means configured to render one or more holographic patterns, the method comprising: The at least one spatial light modulator device is divided into a set of sub-regions; Generate a holographic pattern of the first sub-region within the set of sub-regions; The holographic pattern of the first sub-region is rendered on each sub-region of the set of sub-regions.

12. The method of claim 11, wherein each sub-region in the set of sub-regions has a width equal to the width of each other sub-region in the set of sub-regions, and wherein each sub-region further has a length equal to the length of each other sub-region in the set of sub-regions.

13. A method for a display device, the method comprising: Receive data representing media intended for display at a predetermined display depth in the focus space; A first holographic pattern is generated from the data, wherein the first holographic pattern has a size corresponding to a portion of a spatial light modulator, and wherein the first holographic pattern is adapted to place the medium at an infinite depth; A second holographic pattern is created by placing two or more copies of the first holographic pattern in an array, wherein the second holographic pattern is configured to have a width that is a multiple of the width of the first holographic pattern, and wherein the second holographic pattern is further configured to have a length that is a multiple of the length of the first holographic pattern. The second hologram pattern is converted into a third hologram pattern using a mathematical lensing function, wherein the third hologram is adapted to place the medium for display at a predetermined display depth in the focal space.

14. The method of claim 13, further comprising: A random phase is applied to the data representing the medium before the first hologram pattern is generated.

15. The method of claim 13, further comprising: The third hologram pattern is corrected using an aberration correction function.

16. The method of claim 13, further comprising: The third hologram pattern is transformed using a vision correction function, wherein the vision correction function is adapted to modify the third hologram pattern to correct for visual impairment.

17. The method of claim 13, wherein the mathematical lens function further comprises one of an aberration correction function or a vision correction function.

18. The method of claim 13, further comprising: The third hologram pattern is ultimately determined using a quantization function to provide a quantized hologram pattern.

19. The method of claim 18, wherein the quantization function is based on either error diffusion quantization or mask-based quantization.

20. The method for a display device according to claim 18, wherein the display device includes one or more spatial light modulator devices, and wherein the quantized holographic pattern is rendered on the one or more spatial light modulator devices.

21. The method of claim 12, wherein the first holographic pattern is generated using a Fourier transform.

22. A method for a display device, wherein the display device includes at least one spatial light modulator device, the method comprising: Receive data representing a set of two-dimensional (2D) scene layers, wherein the 2D scene layers have a corresponding predetermined display depth in the focus space; A first holographic pattern is generated for each of the set of 2D scene layers to create a set of first holographic patterns, wherein each first holographic pattern places the associated 2D scene layer at an infinite depth; wherein each first holographic pattern has a size corresponding to a portion of the at least one spatial light modulator device; For each of the first holographic patterns in the set of first holographic patterns, two or more replicas of the first holographic pattern are placed in an array to create a second holographic pattern, wherein the width of each second holographic pattern is a multiple of the width of the associated first holographic pattern, and wherein the length of each second holographic pattern is a multiple of the length of the associated first holographic pattern. Each of the second holographic patterns in the set of second holographic patterns is transformed using a mathematical lensing function to create a set of third holographic patterns, wherein each third holographic pattern is adapted to place the associated 2D scene layer at a corresponding predetermined display depth in the focal space; The set of third holographic patterns is aggregated to provide an aggregated holographic pattern.

23. The method of claim 22, wherein each of the set of first holographic patterns has a width equal to the width of each other first holographic pattern in the set of first holographic patterns, and wherein each first holographic pattern further has a length equal to the length of each other first holographic pattern in the set of first holographic patterns.

24. The method of claim 22, wherein the data is generated from at least one of a three-dimensional (3D) object or a three-dimensional (3D) scene.

25. The method of claim 22, further comprising: A random phase is applied to each of the set of 2D scene layers.

26. The method of claim 22, further comprising: The aggregated hologram pattern is corrected using an aberration correction function.

27. The method of claim 22, further comprising: The aggregated hologram pattern is transformed using a vision correction function, wherein the vision correction function is adapted to modify the aggregated hologram pattern to correct for visual impairment.

28. The method of claim 22, wherein the mathematical lens function further comprises at least one of an aberration correction function or a vision correction function.

29. The method of claim 22, further comprising: The aggregated hologram pattern is ultimately determined using a quantization function to provide a quantized hologram pattern.

30. The method of claim 29, wherein the quantization function is based on at least one of error diffusion quantization or mask-based quantization.

31. The method for a display device according to claim 29, wherein the display device includes one or more spatial light modulator devices, and wherein the quantized holographic pattern is adapted for rendering on the one or more spatial light modulator devices.

32. The method of claim 22, wherein the first holographic pattern in the set of first holographic patterns is generated using a Fourier transform.

33. A method for execution by one or more processing modules of one or more computing devices of a mobile device, the method comprising: Receive visual search requests; Capture the scene in response to the visual search request; The scene is divided into multiple elements; Determine the relative depth of one or more of the plurality of elements; Facilitate the display of indicators associated with one or more of the said one or more elements; Determine whether the indicator is associated with the element of interest; In response to the determination that the indicator is associated with an element of interest, the sensor data associated with the element of interest is transmitted to the processing device; Receive reverse image search results from the processing device; as well as Promote the display of indicators representing the reverse image search results.

34. The method of claim 33, wherein the segmentation of the scene into multiple elements is performed by one or more processing devices, wherein the one or more processing devices are external to the mobile device, and wherein the segmented scene data is adapted for transmission from the one or more processing devices to the mobile device.

35. The method of claim 33, wherein the indicator associated with the element of interest is the outline of the element of interest.

36. The method of claim 33, wherein the contour represents an augmented reality overlay.

37. The method of claim 33, wherein the indicator associated with the element of interest is adapted to be displayed at a predetermined depth relative to the depth of the element of interest.

38. The method of claim 37, wherein the relevant depth is the same as the depth of the element of interest.

39. The method of claim 33, wherein the indicator representing the reverse image search result includes contextual information relating to the element of interest.

40. The method of claim 33, wherein the indicator representing the reverse image search result is adapted to be displayed at a predetermined depth relative to the depth of the element of interest.

41. The method of claim 40, wherein the relevant depth is the same as the depth of the element of interest.

42. The method of claim 33, wherein the processing device is a remote server.

43. The method of claim 30, wherein the mobile device is an augmented reality device.

44. The method of claim 43, wherein the mobile device is further configured to collect metadata.

45. The method of claim 44, further comprising: The metadata is transmitted to the processing device.

46. ​​The method of claim 44, wherein the metadata is captured by one or more sensors associated with the mobile device.

47. The method of claim 44, wherein the metadata includes at least one of descriptive metadata, geospatial metadata, or contextual metadata.

48. A method for execution by one or more processing modules of one or more computing devices of a mobile device, the method comprising: Receive visual search requests; Capture the scene in response to the visual search request; The scene is divided into multiple elements; Potential elements of interest are determined from the plurality of elements based on the tracking element; Determine the relative depth of the potential elements of interest; Facilitate the display of indicators associated with the potential elements of interest; Determine whether the indicator is associated with the actual element of interest; In response to the determination that the indicator is associated with the actual element of interest, sensor data associated with the actual element of interest is transmitted to the processing device; Receive reverse image search results from the processing device; as well as Promote the display of indicators representing the reverse image search results.

49. The method of claim 48, wherein the tracking element is at least one of an eye-tracking sensor or a gaze-tracking sensor.

50. The method of claim 48, wherein the segmentation of the scene into multiple elements is performed by one or more processing devices, wherein the one or more processing devices are external to the mobile device, and wherein the segmented scene data is adapted for transmission from the one or more processing devices to the mobile device.

51. The method of claim 48, wherein the indicator associated with any of the potential elements of interest is the outline of the potential elements of interest.

52. The method of claim 48, wherein the contour represents an augmented reality overlay.

53. The method of claim 48, wherein the indicator associated with the potential element of interest is displayed at a predetermined depth relative to the depth of the potential element of interest.

54. The method of claim 53, wherein the predetermined depth is the same depth as the potential element of interest.

55. The method of claim 48, wherein the indicator representing the reverse image search result includes contextual information relating to the element of interest.

56. The method of claim 48, wherein the indicator representing the reverse image search result is adapted to be displayed at a predetermined depth relative to the depth of the element of interest.

57. The method of claim 56, wherein the predetermined depth is the same as the depth of the element of interest.

58. The method of claim 45, wherein the processing device is a remote server.

59. The method of claim 45, wherein the mobile device is an augmented reality device.

60. The method of claim 45, wherein the mobile device is further configured to collect metadata.

61. The method of claim 60, further comprising: The metadata is transmitted to the processing device.

62. The method of claim 60, wherein the metadata includes at least one of descriptive metadata, geospatial metadata, or contextual metadata.

63. The method of claim 60, wherein the metadata is captured by one or more sensors associated with the mobile device.

64. An augmented reality device comprising: An interface used for network connectivity; light source; Spatial light modulator; One or more optical elements; Memory; and A processing module that is operatively coupled to the interface and the memory.

65. The augmented reality apparatus of claim 64, wherein the processing module is operable to: Data representing a hologram pattern is received via the interface; The data is provided to the spatial light modulator, wherein the spatial light modulator is configured to render the hologram pattern; This facilitates the rendering of holographic patterns illuminated by the light source to provide media visible to the user.

66. The augmented reality device of claim 64, further comprising a holographic processing module, wherein the processing module is further operable to: Receive data representing media to be displayed via the interface; The data is transmitted to the holographic processing module; Receive the holographic pattern from the holographic processing module; The hologram pattern is provided to the spatial light modulator for rendering to provide a rendered hologram pattern; Facilitates the illumination of the rendered holographic pattern by the light source to provide media visible to the user.

67. The augmented reality device of claim 64, wherein the holographic processing module is configured to execute a computer-generated holography (CGH) algorithm.

68. The augmented reality device of claim 64, wherein the CGH algorithm comprises at least one of the following: Fourier transform algorithm, Fresnel transform algorithm, Iterative Fourier Transform (IFTA) algorithm, point cloud-based algorithm, angular spectrum-based algorithm, or lookup table (LUT)-based algorithm.

69. The augmented reality device of claim 64, wherein the media for display comprises at least one of a two-dimensional (2D) image, a two-dimensional (2D) representation, two-dimensional (2D) information, a three-dimensional (3D) object, or a three-dimensional (3D) scene.

70. An optical display system comprising: One or more spatial light modulators are configured to display holographic images that can be viewed by a user; One or more lighting sources; and One or more optical elements.

71. The optical display system of claim 70, wherein the holographic image comprises at least one of a two-dimensional (2D) image, a two-dimensional (2D) representation, two-dimensional (2D) information, a three-dimensional (3D) object, or a three-dimensional (3D) scene.

72. The optical display system according to claim 70, further comprising: One or more holographic processing modules are configured to execute one or more computer-generated holography (CGH) algorithms.

73. The optical display system according to claim 72, wherein the one or more CGH algorithms include at least one of Fourier transform algorithm, Fresnel transform algorithm, Iterative Fourier Transform (IFTA) algorithm, point cloud-based algorithm, angular spectrum-based algorithm, or lookup table (LUT)-based algorithm.

74. The optical display system of claim 70, wherein the spatial light modulator comprises an array of light modulation elements.

75. The optical display system according to claim 74, wherein the pitch between two adjacent optical modulation elements of the optical modulation element array is equal to or less than the wavelength of a predetermined visible light wavelength.

76. The optical display system of claim 74, wherein each optical modulation element of the optical modulation element array modulates at least one of the amplitude, phase, or polarization of the light incident on the optical modulation element.

77. The optical display system according to claim 70, further comprising: One or more displays, wherein the display of the one or more displays is at least one of a head-mounted display, a head-up display, a stereoscopic display, or a holographic display.

78. A method for execution by one or more processing modules of one or more computing devices, the method comprising: Generate a quantization mask, wherein the quantization mask is adapted to shift noise associated with the quantization process outside a predetermined signal window in the frequency domain. The continuous hologram is quantized based on the quantization mask to generate a quantized hologram.

79. The method of claim 76, wherein quantizing the continuous hologram comprises comparing the value of the continuous hologram pattern with the corresponding value of the quantization mask.

80. The method of claim 76, wherein the quantization mask is configured to have the same size as the continuous hologram.

81. The method of claim 76, wherein the quantization mask is configured to have a size smaller than that of the continuous hologram.

82. The method of claim 79, wherein the quantization mask is copied over the continuous hologram to quantize the continuous hologram to create a quantized hologram.

83. A method for execution by one or more processing modules of one or more computing devices, the method comprising: A first quantization mask is generated, wherein the first quantization mask has a size smaller than that of a continuous hologram, and wherein the first quantization mask is adapted to move noise associated with the quantization process outside a predetermined signal window in the frequency domain; A second quantization mask is generated, wherein the second quantization mask has a size smaller than that of the continuous hologram, and wherein the second quantization mask is adapted to move noise associated with the quantization process outside a predetermined signal window in the frequency domain; The first quantization mask and the second quantization mask are used to quantize the continuous hologram to generate a quantized hologram.

84. The method of claim 83, wherein the second quantization mask is different from the first quantization mask.

85. The method of claim 83, wherein quantizing a continuous hologram into a quantized hologram using the first quantization mask and the second quantization mask comprises comparing the value of the continuous hologram with a corresponding value of the first quantization mask or the second quantization mask.

86. The method of claim 83, wherein the second quantization mask is configured to have the same width as the first quantization mask and is configured to have the same length as the first quantization mask.

87. The method of claim 83, wherein the first quantization mask and the second quantization mask are copied alternately along the horizontal or vertical direction of the continuous hologram, or along both the horizontal and vertical directions.

88. A method for execution by one or more processing modules of one or more computing devices, the method comprising: Receive a continuous hologram, wherein the continuous hologram is divided into a pixel array, wherein each pixel is associated with a value corresponding to a value in the continuous hologram; The pixel value of each pixel in the pixel array is quantized into one of a plurality of states using a quantization mask, wherein the quantization mask is configured to facilitate the shifting of noise associated with the quantization process outside a predetermined signal window in the frequency domain.

89. The method according to claim 88, wherein the method comprises: Adjust the continuous hologram so that the value of the continuous hologram is between -1 and 1, including -1 and 1; The continuous hologram is divided to provide four quadrants sharing four points, the first quadrant including a portion of the pixel array, wherein each of the second quadrant (upper left quadrant), the third quadrant (lower left quadrant), and the fourth quadrant (lower right quadrant) includes another equally sized portion of the pixel array; Generate a first quantization mask, wherein each value of the first quantization mask is between -1 and 1, including -1 and 1; Invert the sign of each value in the first quantization mask to provide a second quantization mask; The first quantization mask is used above the second quadrant and the fourth quadrant to quantize the second quadrant and the fourth quadrant of the continuous hologram; Reverse the sign of each value in the first and third quadrants of the continuous hologram to provide a sign-reversed first quadrant and a sign-reversed third quadrant; The second quantization mask is used over the first quadrant and the third quadrant of the sign inversion of the continuous hologram to provide the quantized first quadrant and the quantized third quadrant of the sign inversion. Reversing the sign of each value in the first quadrant and the third quadrant of the quantized sign inversion provides a quantized hologram of the first and fourth quadrants of the continuous hologram.

90. The method of claim 89, wherein the first quantization mask and the second quantization mask have dimensions smaller than the dimensions of any of the four quadrants, the method comprising: The first quantization mask is replicated above the second quadrant and above the fourth quadrant to quantize the second and fourth quadrants of the continuous hologram, providing quantized holograms of the second and fourth quadrants. Reverse the sign of each value in the first and third quadrants of the continuous hologram to provide a sign-reversed first quadrant and a sign-reversed third quadrant; The second quantization mask is copied above the first quadrant of the sign inversion and above the third quadrant of the sign inversion to quantize the first quadrant of the sign inversion and the third quadrant of the sign inversion, providing the quantized first quadrant of the sign inversion and the quantized third quadrant of the sign inversion. Reversing the sign of each value in the first quadrant and the third quadrant of the quantized sign inversion provides a quantized hologram of the first and fourth quadrants of the continuous hologram.

91. A method for a display device, the method comprising: Receive data representing media intended for display at a predetermined display depth in the focus space; A first holographic pattern is generated from the data, the first holographic pattern placing the media at infinite depth; The first hologram pattern is transformed into a second hologram pattern using a mathematical lensing function, the second hologram pattern placing the medium for display from the infinite depth to the predetermined display depth in the focal space.

92. The method of claim 91, wherein the media for displaying at a predetermined display depth in the focal space comprises one of a two-dimensional (2D) image, a two-dimensional (2D) representation, or two-dimensional (2D) information.

93. The method of claim 91, further comprising: Before generating the first hologram pattern, a random phase is applied to the data representing the medium to display it at a predetermined display depth in the focal space.

94. The method of claim 91, wherein the mathematical lens function further comprises one of an aberration correction function or a vision correction function.

95. The method of claim 91, further comprising: The quantization function was used to finally determine that the second hologram pattern was a quantized hologram pattern.

96. The method of claim 95, wherein the quantization function is based on either error diffusion quantization or mask-based quantization.

97. The method for a display device according to claim 95, wherein the display device includes one or more spatial light modulator devices, and wherein the quantized holographic pattern is rendered on the one or more spatial light modulator devices.

98. The method of claim 95, wherein the first holographic pattern is generated using a Fourier transform.

99. An optical display system comprising: One or more spatial light modulators; One or more optical modules; and One or more optical combiners.

100. The optical display system of claim 99, wherein the one or more spatial light modulators comprise an array of light modulation elements, wherein each light modulation element in the array is individually addressable to control the state of the light modulation element, wherein each light modulation element can exhibit at least two different states, each state having different optical properties.

101. The optical display system of claim 100, wherein the pitch between two adjacent optical modulation elements of the optical modulation element array is equal to or less than the wavelength of the light incident on the one or more spatial optical modulators.

102. The optical display system of claim 100, wherein the pitch between two adjacent optical modulation elements of the optical modulation element array is equal to or less than half the wavelength of the light incident on the one or more spatial light modulators.

103. The optical display system of claim 100, wherein the optical modulation elements of the optical modulation element array are configured to modulate either the amplitude or the phase of the polarization of light incident on the optical modulation elements.

104. The optical display system of claim 100, wherein the optical modulation element of the optical modulation element array of the one or more spatial light modulators comprises: Phase change materials; Heater element; Two or more electrodes connected to the heater element, wherein the optical modulation element is configured to be individually addressable via the two or more electrodes; The state of the optical modulation element is configured to change by altering the state of the phase change material in response to a thermal contribution from the heater element.

105. The optical display system according to claim 99, wherein the optical module in the one or more optical modules comprises: Lighting solutions; The first set of optical elements; and The second set of optical elements.

106. The optical display system of claim 105, wherein the illumination scheme comprises: One or more illumination sources and optical components, wherein the optical components are configured to transmit light from the one or more illumination sources to the one or more spatial light modulators.

107. The optical display system of claim 106, wherein the one or more illumination sources are RGB laser sources.

108. The optical display system of claim 106, wherein the optical component comprises at least one of the following: Beam splitter; Total internal reflection (TIR) ​​prism; Freeform prism.

109. The optical display system of claim 106, wherein the optical component includes a collimator element configured to provide collimated illumination to the one or more spatial light modulators.

110. The optical display system of claim 106, wherein at least one of the first group of optical elements or the second group of optical elements of the optical module comprises one or more achromatic lenses.

111. The optical display system of claim 105, wherein at least one of the first group of optical elements or the second group of optical elements of the optical module comprises at least one of the following: lens; Mirror; Prism; Beam splitter; Optical filters; Polarizer.

112. The optical display system of claim 105, wherein the optical module in the one or more optical modules comprises: One or more optical corrector elements, wherein the one or more optical corrector elements are configured to correct aberrations introduced by optical elements associated with the optical module.

113. The optical display system of claim 112, wherein the one or more optical corrector elements comprise at least one of the following: toroidal optical elements; or Freeform surface optical elements.

114. The optical display system of claim 99, wherein the optical combiner in the one or more optical combiners comprises at least one of the following: Holographic optical elements; Metasurface; Semi-reflective element.

115. The optical display system of claim 105, wherein a first set of optical elements of the one or more optical modules forms a first lens group of a 4f optical system, and wherein a second set of optical elements, together with the one or more optical combiners, is configured to form a second lens group of the 4f optical system.

116. The optical display system of claim 115, wherein the first lens group of the 4f optical system is adapted to perform a Fourier transform on information representing a hologram at the plane of the one or more spatial light modulators by converting information at the plane of the one or more spatial light modulators from the spatial domain to the spatial frequency domain, wherein the spatial frequency information is at the Fourier plane of the 4f optical system, and wherein the second lens group of the 4f system is adapted to perform an inverse Fourier transform on the spatial frequency information at the Fourier plane by converting the spatial frequency information at the Fourier plane from the frequency domain to the spatial domain.

117. The optical display system of claim 99, wherein the one or more optical modules comprise: Lighting solutions; A first set of optical elements, wherein the first set of optical elements forms a first lens group of a 4f optical system, and wherein one or more optical combiners form a second lens group of the 4f optical system.

118. The optical display system of claim 99, wherein the one or more optical modules further comprises one or more optical filter elements.

119. The optical display system of claim 99, wherein the one or more optical filter elements are configured to filter out unwanted frequency-based information at the Fourier plane.

120. The optical display system according to claim 99, further comprising one or more processing units.

121. The optical display system of claim 120, wherein the one or more processing units are implemented on a system-on-a-chip.

122. The optical display system of claim 120, wherein the one or more processing units are configured to execute one or more computer-generated holography (CGH) algorithms to generate one or more holographic patterns for rendering using the one or more spatial light modulators.

123. The optical display system according to claim 120, wherein the one or more CGH algorithms include at least one of Fourier transform algorithm, Fresnel transform algorithm, Iterative Fourier Transform (IFTA) algorithm, point cloud-based algorithm, angular spectrum-based algorithm, or lookup table (LUT)-based algorithm.

124. The optical display system of claim 120, wherein the one or more processing units are adapted to modify one or more holographic patterns for rendering using the one or more spatial light modulators to provide a modified holographic pattern, wherein the modified holographic pattern is adapted to correct aberrations introduced by the one or more optical modules or by the one or more optical combiners.

125. The optical display system of claim 120, wherein the one or more processing units are adapted to modify one or more holographic patterns for rendering using the one or more spatial light modulators to provide a modified holographic pattern, wherein the modified holographic pattern is adapted to correct refractive errors or visual impairment associated with a user of the optical display system.

Images