Display device and method

Abstract

A display apparatus and method are disclosed. The display apparatus is arranged to form a visible first picture at all viewing positions within a viewing window, the display apparatus comprising: a picture generation unit arranged to output first spatially modulated light of a first colour; a waveguide pupil expander comprising: an input port; a viewing window; and a pair of parallel reflective surfaces arranged to direct the first spatially modulated light from the input port to the viewing window by a series of internal reflections from the pair of parallel reflective surfaces; a viewer tracking system arranged to determine a viewing position within the viewing window, wherein the first picture is formed by a holographic reconstruction of a first hologram of the first picture; and a controller arranged to change or recalculate the first hologram in response to a change in the viewing position determined by the viewer tracking system.

Description

[0001] This application is a divisional application of the applicant’s patent application No. 202011068554.7 entitled “Pupil Expander”, filed on September 28, 2020. Technical Field

[0002] This disclosure relates to a pupil expander, such as a waveguide pupil expander. More specifically, this disclosure relates to a multicolor waveguide pupil expander, such as a waveguide pupil expander for a two- or three-color display. This disclosure also relates to a projector and projection method, such as holographic projection and holographic projection methods. Some embodiments relate to head-up displays. Some embodiments relate to optimizing image uniformity within a viewing window. Background Technology

[0003] Light scattered from an object contains amplitude and phase information. This amplitude and phase information can be captured on, for example, a photosensitive plate using well-known interferometry techniques to form a holographic record, or "hologram," including interference fringes. The hologram can be reconstructed by illuminating it with appropriate light to form a two-dimensional or three-dimensional holographic reconstruction or playback image representing the original object.

[0004] Computer-generated holography can numerically simulate interference processes. Computer-generated holograms can be calculated using techniques based on mathematical transformations such as Fresnel or Fourier transforms. These types of holograms are called Fresnel / Fourier transform holograms or simply Fresnel / Fourier holograms. A Fourier hologram can be considered a Fourier domain / plane representation of an object or a frequency domain / plane representation of an object. For example, computer-generated holograms can also be calculated using coherent ray tracing or point cloud techniques.

[0005] Computer-generated holograms can be encoded on spatial light modulators arranged to modulate the amplitude and / or phase of incident light. For example, optical modulation can be achieved using electrically addressable liquid crystals, optically addressable liquid crystals, or micromirrors.

[0006] Spatial light modulators typically comprise multiple individually addressable pixels, which may also be referred to as cells or elements. The light modulation scheme can be two-state, multi-level, or sequential. Alternatively, the device can be sequential (i.e., excluding pixels), so light modulation can be continuous throughout the device. Spatial light modulators can be reflective, meaning the modulated light is output via reflection. Spatial light modulators can also be transmissive, meaning the modulated light is output via transmission.

[0007] The system described in this article can be used to provide holographic display devices, such as holographic projectors. Such projectors have already found applications in head-up displays (HUDs). Summary of the Invention

[0008] The aspects of this disclosure are defined in the appended independent claims.

[0009] This document discloses a display device including an image generation unit, a waveguide pupil expander, and a viewer tracking system. The image generation unit includes a first display channel, a second display channel, and a controller. The first display channel is arranged to output a first spatially modulated light of a first color. The first spatially modulated light corresponds to a first image (or picture). The second display channel is arranged to output a second spatially modulated light of a second color. The second spatially modulated light corresponds to a second image (or picture). The controller is arranged to drive the first and second display channels. The waveguide pupil expander includes a pair of parallel reflective surfaces. The waveguide pupil expander defines an input port and a viewing window. The input port is arranged to receive the first and second spatially modulated light. The viewing window is an area or volume within which a viewer can view the first image / picture and the second image / picture. The pair of parallel reflective surfaces is arranged to guide the first and second spatially modulated light from the input port to the viewing window through a series of internal reflections. The reflectivity of the first reflective surface in the pair of parallel reflective surfaces is provided by a gradient coating. The gradient coating partially transmits light of the first and second colors. The transmittance of the gradient coating is non-achromatic. The viewer tracking system is configured to determine the viewer's position within the viewing window. The controller is configured to maintain the color balance of the first and second images viewed from the viewer's position as substantially constant based on the viewer's position determined by the viewer tracking system.

[0010] A waveguide pupil expander comprises two reflective surfaces. One reflective surface—e.g., a first reflective surface—is partially transmissive and partially reflective. The partial transmissivity results in the formation of a copy of the light received by the waveguide. The other reflective surface of the waveguide—e.g., a second reflective surface—is a fully or nearly fully reflective surface—e.g., a full mirror. Spatially modulated light is received at the input port of the waveguide. The waveguide may be substantially planar, with the spacing between the two reflective surfaces less than the dimensions of the two reflective surfaces. Each reflective surface is substantially planar. The waveguide is configured such that a viewer should gaze at the larger surface of one of the reflective surfaces—e.g., a partially reflective surface. This surface may be referred to as the viewing surface. The input port may be on the same side of the planar surface as the viewer, or it may be on the opposite side. The input port may include a slit or gap in the reflective element of the fully or partially reflective surface—e.g., a reflective layer or coating. Alternatively, the input port may begin from the end of the reflective element.

[0011] The viewing window can be an eye box or an eye-tracking box. The waveguide enlarges the viewing window because the division of the amplitude of the incident wavefront produces excess light. According to this disclosure, the incident light (i.e., the light received by the waveguide) is spatially modulated light. Spatially modulated light can be encoded using a hologram of the image or the image itself. In either case, the incident light can be collimated.

[0012] Embodiments of this disclosure relate to a multicolor waveguide pupil expander. Specifically, the waveguide is configured to perform pupil expansion of a first color image (i.e., a first image formed by light of a first wavelength) and a second color image (i.e., a second image formed by light of a second wavelength, wherein the first wavelength is different from the second wavelength). The system can be configured such that the first color image and the second color image are perceived as substantially overlapping at the viewing window in a manner familiar to the reader. For example, the system can form a full-color image by combining multiple monochrome images, such as red, green, and blue images (referred to herein as a "composite color image / picture").

[0013] The first surface of the waveguide includes a partially transmissive and reflective coating. The reflectivity of the coating is not constant along the length of the waveguide (in the overall direction of light propagation). However, for optimal viewing, the brightness of each color image should not change as the viewer moves around within the viewing window. Furthermore, the color balance between different monochromatic image components should remain constant. The coating may include more than 25 dielectric layers to provide the necessary optical properties at multiple wavelengths. Significant effort can be invested in optimizing the coating, but achieving a completely achromatic coating is virtually impossible.

[0014] The non-achromatic nature of the first reflective surface / gradient mirror causes the color balance of the image to vary across the entire viewing window. The inventors have discovered that viewers are particularly sensitive to this type of imbalance, especially in real-world systems where viewing positions are unstable, such as head-up displays. The inventors have devised a method to compensate for the imperfect (i.e., non-achromatic) coating on the viewing surface / gradient mirror. Notably, the inventors have recognized the importance of chromaticity as a sensitive function of viewing position. This position-sensitive dependence is a result of guiding light through the internal reflection of a waveguide. Specifically, this is due to the perceived image formed by different light rays at different viewing positions. These different light rays may experience different numbers of bounces (i.e., reflections) within the waveguide. Different light rays will travel along different reflection paths within the plate and are therefore reflected by different local areas of the gradient coating. According to this disclosure, the viewer's position within the viewing window is used to fine-tune the brightness of individual monochrome images to compensate for the chromaticity of the gradient mirror. A viewer tracking system is used as part of a feedback system. The viewer tracking system can be an eye-tracking system or a head-tracking system. Calibration data can be used to fine-tune the brightness of one or more monochrome images in real time to maintain color balance. Calibration data can be obtained through a calibration process that involves measuring the relative brightness of each monochrome image at multiple different viewing positions within the viewing window.

[0015] A waveguide replicates the light from the incident wavefront to expand the viewing window / pupil. Each successive copy is formed by one or more bounces of light within the waveguide. Therefore, the reflectivity / transmittance of the viewing surface is gradient, i.e., it varies with distance along the waveguide to compensate for the decrease in light intensity along the waveguide due to partial escape of light at each bounce. Thus, the reflectivity of the gradient coating decreases with distance from the input port. The gradient coating may comprise a stack of thin films, optionally wherein each film is a dielectric or a metal. Those skilled in the art will be familiar with the design of such a stack and will understand the difficulties in providing the optical properties required to provide the multicolor pupil expansion as described herein.

[0016] In some embodiments, spatially modulated light is the light of an image formed on an intermediate plane between the image generation unit and the viewing window. The image formed at the intermediate plane can be formed in free space or on a screen, such as a diffuser. In these embodiments, the amplitude of the light is spatially modulated according to the image being displayed. Thus, in these embodiments, it can be said that a first display channel is arranged to display a first image, and a second display channel is arranged to display a second image. Each image can be formed using a hologram of the image by holographic projection onto the intermediate plane or the screen. That is, each display channel can display a hologram, which is illuminated and formed into an image on the intermediate plane or the screen (e.g., a diffuser) by holographic reconstruction. In embodiments where the image (i.e., holographic reconstruction) is formed on the screen, the screen may preferably be considered part of the image generation unit. In this case, it can be said that the screen is positioned between the hologram plane and the viewing plane.

[0017] In other embodiments, spatially modulated light is encoded using a hologram. In these embodiments, the hologram is displayed, and the spatially modulated light, according to the hologram pattern, can be used to reconstruct the image. It is sometimes said that in this configuration, the viewer's eye lens performs a hologram-to-image (e.g., frequency domain to spatial domain) conversion. The holographic reconstruction can thus be formed on the viewer's retina. Optionally, a lens may be included between the display / hologram and the viewer to relay the hologram reconstruction or determine the location of the reconstruction plane. The hologram may be a Fourier hologram. Embodiments where the viewer directly views the hologram (e.g., a spatial light modulator displays the hologram) may be referred to as direct viewing. Direct viewing embodiments are characterized by the absence of a screen (e.g., a diffuser) between the viewer and the hologram. Direct viewing embodiments may be advantageous because they avoid the problems associated with using a screen / diffuser.

[0018] The first / second holographic reconstruction may include a first / second image (or image) region containing a first / second picture (or image) and a first / second non-picture (or non-image) region containing first / second non-picture (or non-image) content. The projector also includes a closed-loop feedback system comprising first / second photodetectors arranged to measure the brightness of the first / second non-image regions. The hologram redirects light to different points on the playback field. The hologram may be a phase hologram, such as a phase-only hologram. Therefore, due to the nature of the holographic process, the brightness of the non-image regions can indicate the brightness of the corresponding image regions. In some embodiments, if the brightness of the non-image regions is maintained, for example by modulating the output of the light source, the brightness of the image regions will be maintained. These embodiments address the problem that the brightness of the holographic playback field is a function of the amount of image content formed by the hologram.

[0019] In some embodiments, the controller is arranged to maintain the relative brightness of the first image region and the second image region substantially constant by applying a main drive signal correction to the drive signal of the first / second light source based on the brightness of the first / second non-image region measured by the first / second detector. The main drive signal correction may be a correction of the voltage of the drive signal or a correction of the switching time gating of the drive signal.

[0020] The controller can be configured to maintain a substantially constant relative brightness between the first and second images as seen from the viewing position by applying a secondary drive signal correction to the drive signals of the first / second light source. The secondary drive signal correction is a function of the viewing position within the viewing window. The primary drive signal correction compensates for the holographic process, while the secondary drive signal correction compensates for the non-achromatic aberration of the graduated mirror. The primary drive signal correction maintains the target output of the photodetector, while the secondary drive signal correction provides for variations in the target output, such as an offset relative to the target output. The target output can be, for example, a voltage or current value. The offset value required to maintain the brightness of the first / second image can be determined through calibration. The target output and offset, based on the viewing position, can be stored in a lookup table in the projector's memory.

[0021] Alternatively or additionally, the controller may be arranged to maintain the relative brightness of the first and second images seen from the viewing position substantially constant by changing / recalculating the first / second holograms in response to a viewing position determined by a viewer tracking system. The holographic playback plane includes a zero-order playback field. The zero-order playback field includes a zero-order spot at its center. The zero-order spot includes light that has not yet been modulated by the spatial light modulator. Various methods for changing the ratio of modulated light to unmodulated light in the zero-order playback field are disclosed herein. Each of these methods can be used to fine-tune the brightness of the first / second images based on the detected viewing position.

[0022] The value of each hologram pixel can be restricted to one of several permissible light modulation levels depending on the quantization scheme. Recalculating the first / second hologram may include changing the quantization scheme to alter the light proportion of the zero-order spot that contributes to holographic reconstruction.

[0023] Changing the quantization scheme may include reducing the range of optical modulation within which permissible optical modulation levels are allocated. Alternatively or additionally, changing the quantization scheme may include altering the vector sum of the vectors representing permissible optical modulation levels on the complex plane. This changes the proportion of light contributing to the zero-order spot, thus altering the brightness of the perceived image.

[0024] The term "hologram" is used to refer to a record containing amplitude or phase information about an object, or some combination thereof. The term "holographic reconstruction" is used to refer to the optical reconstruction of an object formed by illuminating a hologram. The system disclosed herein is described as a "holographic projector" because the holographic reconstruction is a real image and spatially separate from the hologram. The term "replay field" is used to refer to the 2D region within which the holographic reconstruction is formed and fully focused. If the hologram is displayed on a spatial light modulator including pixels, the playback field will be repeated in the form of multiple diffraction orders, where each diffraction order is a copy of the zero-order playback field. The zero-order playback field generally corresponds to the preferred or principal playback field because it is the brightest playback field. Unless otherwise explicitly stated, the term "replay field" should be considered to refer to the zero-order playback field. The term "replay plane" is used to refer to a plane in space containing all the playback fields. The terms "image," "playback image," and "image region" refer to the region of the playback field illuminated by the light reconstructed by the hologram. In some embodiments, an "image" may include discrete light spots, which may be referred to as "image spots" or, for convenience only, as "image pixels".

[0025] The terms "writing" or "addressing" can be used to describe the process of providing multiple corresponding control values ​​to multiple pixels of an SLM, each determining the modulation level of the pixel. In other words, the pixels of an SLM are configured to "display" an optical modulation distribution in response to receiving multiple control values. Therefore, it can be said that an SLM "displays" a hologram, and a hologram can be considered an array of optical modulation values ​​or levels.

[0026] It has been discovered that acceptable-quality holographic reconstructions can be formed from "holograms" containing only phase information related to the Fourier transform of the original object. Such holographic records can be referred to as phase-only holograms. While embodiments relate to phase-only holograms, this disclosure is equally applicable to amplitude-only holograms.

[0027] This disclosure is equally applicable to forming holographic reconstructions using amplitude and phase information related to the Fourier transform of the original object. In some embodiments, this is achieved by using complex modulation of a so-called fully complex hologram containing amplitude and phase information related to the original object. Because the value (gray level) assigned to each pixel of the hologram has amplitude and phase components, such a hologram can be called a fully complex hologram. The value (gray level) assigned to each pixel can be represented as a complex number with amplitude and phase components. In some embodiments, a fully complex computer-generated hologram is computed.

[0028] The term "phase delay" can be a shorthand for the phase value, phase component, phase information, or simply phase of a pixel in a computer-generated hologram or spatial light modulator. That is, any phase value described is actually a number (e.g., in the range of 0 to 2) representing the amount of phase delay provided by that pixel. For example, a spatial light modulator is described as a pixel having a π / 2 phase value causing a phase delay of π / 2 radians in the received light. In some embodiments, each pixel of a spatial light modulator can operate on one of a plurality of possible modulation values ​​(e.g., phase delay values). The term "gray level" can be used to refer to a plurality of available modulation levels. For example, the term "gray level" can be used for convenience to refer to a plurality of available phase levels in a phase modulator, even if different phase levels do not provide different shades of gray. For convenience, the term "gray level" can also be used to refer to a plurality of available complex modulation levels in a complex modulator.

[0029] Therefore, a hologram comprises an array of gray levels, i.e., an array of optical modulation values, such as phase delay values ​​or complex modulation values. A hologram is also considered a diffraction pattern because it is a pattern that causes diffraction when displayed on a spatial light modulator and illuminated with light whose wavelength is relative to (typically less than) the pixel spacing of the spatial light modulator. Reference is made herein to combining holograms with other diffraction patterns, such as those used as lenses or gratings. For example, a diffraction pattern used as a grating can be combined with a hologram to translate the playback field on the playback plane, or a diffraction pattern used as a lens can be combined with a hologram to focus the holographic reconstruction onto the playback plane in the near field.

[0030] Although different embodiments and groups of embodiments may be disclosed separately in the detailed description below, any feature of any embodiment or group of embodiments may be combined with any other feature or combination of features of any embodiment or group of embodiments. That is, all possible combinations and substitutions of the features disclosed in this disclosure are contemplated. Attached Figure Description

[0031] Specific embodiments are described by way of example only with reference to the following figures:

[0032] Figure 1This is a schematic diagram illustrating a reflective SLM that generates holographic reconstruction on a screen;

[0033] Figure 2A The first iteration of the example Gerchberg-Saxton type algorithm is shown;

[0034] Figure 2B The second and subsequent iterations of the example Gerchberg-Saxton type algorithm are shown;

[0035] Figure 2C Alternative second and subsequent iterations of the example Gerchberg-Saxton type algorithm are shown;

[0036] Figure 3 This is a schematic diagram of a reflective LCOS SLM;

[0037] Figure 4 An example pupil expander including a waveguide is shown;

[0038] Figure 5 The transmittance for different wavelengths of light is shown as a function of the wavelength along the path. Figure 4 An example diagram showing the relationship between the propagation distance of the waveguide;

[0039] Figure 6 A holographic display device including a pupil expander according to an embodiment is shown;

[0040] Figure 7A -C illustrates the geometry of the pupil expander according to an embodiment;

[0041] Figure 8 A holographic playback field formed by the holographic display device according to an embodiment is shown;

[0042] Figure 9 A holographic display device including a pupil expander according to other embodiments is shown;

[0043] Figure 10 A set of uniformly distributed permissible phase / modulation levels is shown plotted in the complex plane in the range of 0 to 2π;

[0044] Figure 11 This shows a set of sum vectors that allow all available modulation levels in the phase modulation levels;

[0045] Figure 12A It shows a similar structure with balanced truncation. Figure 10 A set of cutoff ranges that allow for phase modulation levels;

[0046] Figure 12B It shows a similar one with unbalanced truncation. Figure 12AA set of cutoff ranges that allow for phase modulation levels; and

[0047] Figure 13 The diagram illustrates the formation of a virtual image in a head-up display using pupil expanders and optical combiners, such as a windshield.

[0048] The same reference numerals will be used throughout the accompanying drawings to refer to the same or similar parts. Detailed Implementation

[0049] This invention is not limited to the embodiments described below, but extends to the full scope of the appended claims. That is, the invention can be implemented in different forms and should not be construed as limited to the described embodiments, which are illustrated for illustrative purposes.

[0050] Unless otherwise stated, a singular term may include a plural term.

[0051] A structure described as being formed above or below another structure should be interpreted to include situations where structures are in contact with each other and where a third structure is situated between them.

[0052] When describing temporal relationships, such as when the chronological order of events is described as “after,” “following,” “next,” “before,” etc., this disclosure shall be considered to include both consecutive and discontinuous events, unless otherwise specified. For example, the description shall be considered to include discontinuous cases unless wording such as “only,” “immediately adjacent,” or “directly” is used.

[0053] Although the terms “first,” “second,” etc., may be used herein to describe various elements, these elements are not limited by these terms. These terms are used only to distinguish individual elements. For example, without departing from the scope of the appended claims, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element.

[0054] Features of different embodiments may be combined or integrated with each other in part or in whole, and may interoperate differently with each other. Some embodiments may be implemented independently of each other, or may be implemented together in an interdependent relationship.

[0055] Optical configuration

[0056] Figure 1An embodiment is shown in which a computer-generated hologram is encoded on a single spatial light modulator. The computer-generated hologram is a Fourier transform of the object used for reconstruction. Therefore, a hologram can be described as a Fourier domain, frequency domain, or spectral domain representation of the object. In this embodiment, the spatial light modulator is a reflective liquid crystal on silicon (LCOS) device. The hologram is encoded on the spatial light modulator, and a holographic reconstruction is formed at a playback field, such as a light-receiving surface like a screen or diffuser.

[0057] A light source 110, such as a laser or laser diode, is positioned to illuminate the SLM 140 via a collimating lens 111. The collimating lens causes the approximately plane wavefront of the light to be incident on the SLM. Figure 1 In this embodiment, the wavefront is oriented off-normally (e.g., two or three degrees away from a plane that is truly perpendicular to the transparent layer). However, in other embodiments, a generally planar wavefront is provided with normal incidence, and beam splitters are arranged to separate the input and output optical paths. Figure 1 In the illustrated embodiment, the arrangement is such that light from the light source is reflected from the mirrored rear surface of the SLM and interacts with the light modulation layer to form an outgoing wavefront 112. The outgoing wavefront 112 is applied to an optics device including a Fourier transform lens 120, the focal point of which is located on a screen 125. More specifically, the Fourier transform lens 120 receives the modulated beam from the SLM 140 and performs a frequency-space transformation to produce a holographic reconstruction on the screen 125.

[0058] It is worth noting that in this type of hologram, each pixel of the hologram contributes to the overall reconstruction. There is no one-to-one correlation between a specific point (or image pixel) in the playback field and a specific optical modulation element (or hologram pixel). In other words, the modulated light leaving the optical modulation layer is distributed across the entire playback field.

[0059] In these embodiments, the spatial position of the holographic reconstruction is determined by the diopter (focusing) of the Fourier transform lens. Figure 1 In the illustrated embodiment, the Fourier transform lens is a physical lens. That is, the Fourier transform lens is an optical Fourier transform lens, and the Fourier transform is performed optically. Any lens can act as a Fourier transform lens, but the lens's performance will limit the accuracy of the Fourier transform it performs. Those skilled in the art will understand how to use lenses to perform optical Fourier transforms.

[0060] Holographic computation

[0061] In some embodiments, the computer-generated hologram is a Fourier transform hologram, or simply a Fourier hologram or Fourier-based hologram, in which an image is reconstructed in the far field by utilizing the Fourier transform properties of a positive lens. The Fourier hologram is computed by Fourier transforming the desired light field in the playback plane back to the lens plane. The Fourier transform can be used to compute computer-generated Fourier holograms.

[0062] Algorithms such as the Gerchberg-Saxton algorithm can be used to compute Fourier transform holograms. Furthermore, the Gerchberg-Saxton algorithm can be used to compute holograms in the Fourier domain (i.e., Fourier transform holograms) based solely on amplitude-only information in the spatial domain (e.g., a photograph). Phase information relating to the object is effectively "obtained" from amplitude-only information in the spatial domain. In some embodiments, the Gerchberg-Saxton algorithm or its variants are used to compute computer-generated holograms from amplitude-only information.

[0063] The Gerchberg-Saxton algorithm considers the case where the intensity cross-sections IA(x,y) and IB(x,y) of a beam in planes A and B are known, and IA(x,y) and IB(x,y) are correlated by a single Fourier transform. For a given intensity cross-section, approximate values ​​of the phase distributions in planes A and B, ΨA(x,y) and ΨB(x,y), are obtained. The Gerchberg-Saxton algorithm finds a solution to the problem by following an iterative process. More specifically, the Gerchberg-Saxton algorithm iteratively applies spatial and spectral constraints while repeatedly transmitting datasets (amplitude and phase) representing IA(x,y) and IB(x,y) between the spatial and Fourier (spectral or frequency) domains. A corresponding computer-generated hologram in the spectral domain is obtained through at least one iteration of the algorithm. The algorithm is convergent and arranged to produce a hologram representing the input image. The hologram can be an amplitude-only hologram, a phase-only hologram, or a fully complex hologram.

[0064] In some embodiments, the phase-only hologram is computed using an algorithm based on the Gerchberg-Saxton algorithm, such as that described in British Patents 2498170 or 2501112, the entire contents of which are incorporated herein by reference. However, the embodiments disclosed herein are described by way of example only for the computation of the phase-only hologram. In these embodiments, the Gerchberg-Saxton algorithm obtains the phase information Ψ[u,v] of the Fourier transform of a dataset, which produces known amplitude information T[x,y], where the amplitude information T[x,y] represents the target image (e.g., a photograph). Since amplitude and phase are inherently combined in the Fourier transform, the transformed amplitude and phase contain useful information about the accuracy of the computed dataset. Therefore, the algorithm can be used iteratively with feedback of amplitude and phase information. However, in these embodiments, the phase-only information Ψ[u,v] is used as a hologram to form a holographic representation of the target image on the image plane. The hologram is a dataset of phase values ​​(e.g., a 2D array).

[0065] In other embodiments, an algorithm based on the Gerchberg-Saxton algorithm is used to compute a fully complex hologram. A fully complex hologram is a hologram having amplitude and phase components. A hologram is a dataset (e.g., a 2D array) comprising an array of complex data values, where each complex data value includes an amplitude component and a phase component.

[0066] In some embodiments, the algorithm processes complex data, and the Fourier transform is a complex Fourier transform. Complex data can be viewed as comprising (i) real and imaginary components, or (ii) amplitude and phase components. In some embodiments, the two components of the complex data are processed differently at different stages of the algorithm.

[0067] Figure 2A A first iteration of an algorithm for computing a phase-only hologram, according to some embodiments, is shown. The input to the algorithm is an input image 210 comprising a 2D array of pixel or data values, where each pixel or data value is an amplitude or oscillation value. That is, each pixel or data value of the input image 210 does not have a phase component. Therefore, the input image 210 can be considered as an amplitude-only, oscillation-only, or intensity-only distribution. An example of such an input image 210 is a photograph or a frame of a video comprising a temporal sequence of frames. The first iteration of the algorithm begins with a data formation step 202A, which includes assigning random phase values ​​to each pixel of the input image using a random phase distribution (or random phase seed) 230 to form an initial complex dataset, where each data element of the dataset includes both amplitude and phase. In other words, the initial complex dataset represents the input image in the spatial domain.

[0068] First processing block 250 receives an initial complex dataset and performs a complex Fourier transform to form a complex dataset of Fourier transforms. Second processing block 253 receives the complex dataset of Fourier transforms and outputs a hologram 280A. In some embodiments, hologram 280A is a phase-only hologram. In these embodiments, second processing block 253 quantizes each phase value and sets each amplitude value to a unit value to form hologram 280A. Each phase value is quantized according to a phase level that can be represented on a pixel of a spatial light modulator, which will be used to “display” the phase-only hologram. For example, if each pixel of the spatial light modulator provides 256 different phase levels, each phase value of the hologram is quantized to one of the 256 possible phase levels. Hologram 280A is a phase-only Fourier hologram representing an input image. In other embodiments, hologram 280A is a fully complex hologram comprising an array of complex data values ​​(each including an amplitude component and a phase component) derived from a received complex dataset of Fourier transforms. In some embodiments, a second processing block 253 constrains each complex data value to one of a plurality of permissible complex modulation levels to form hologram 280A. The constraint step may include setting each complex data value to the closest permissible complex modulation level in the complex plane. Hologram 280A can be said to represent an input image in the spectral, Fourier, or frequency domain. In some embodiments, the algorithm stops at this point.

[0069] However, in other embodiments, the algorithm continues, such as Figure 2A As shown by the dashed arrow in the image. In other words, follow... Figure 2A The steps indicated by the dashed arrows are optional (i.e., not essential for all embodiments).

[0070] The third processing block 256 receives the modified complex dataset from the second processing block 253 and performs an inverse Fourier transform to form a complex dataset with an inverse Fourier transform. The complex dataset with the inverse Fourier transform can be said to represent the input image in the spatial domain.

[0071] The fourth processing block 259 receives the complex dataset of the inverse Fourier transform and extracts the distribution of amplitude values ​​211A and the distribution of phase values ​​213A. Optionally, the fourth processing block 259 evaluates the distribution of amplitude values ​​211A. Specifically, the fourth processing block 259 can compare the distribution of amplitude values ​​211A of the complex dataset of the inverse Fourier transform with the input image 510, which itself is, of course, the distribution of amplitude values. If the difference between the distribution of amplitude values ​​211A and the input image 210 is sufficiently small, the fourth processing block 259 can determine that the hologram 280A is acceptable. That is, if the difference between the distribution of amplitude values ​​211A and the input image 210 is sufficiently small, the fourth processing block 259 can determine that the hologram 280A is a sufficiently accurate representation of the input image 210. In some embodiments, for comparison purposes, the distribution of phase values ​​213A of the complex dataset of the inverse Fourier transform is ignored. It should be understood that any number of different methods can be used to compare the distribution of amplitude value 211A with the input image 210, and this disclosure is not limited to any particular method. In some embodiments, the mean squared error is calculated, and if the mean squared error is less than a threshold, the hologram 280A is considered acceptable. If the fourth processing block 259 determines that the hologram 280A is unacceptable, further iterations of the algorithm can be performed. However, this comparison step is not required, and in other embodiments, the number of iterations of the algorithm performed is predetermined, preset, or user-defined.

[0072] Figure 2B This represents the second iteration of the algorithm and any further iterations of the algorithm. The distribution of the phase values ​​213A from previous iterations is fed back through the algorithm's processing block. The distribution of the amplitude values ​​211A is discarded to favor the distribution of the amplitude values ​​of the input image 210. In the first iteration, the data formation step 202A forms a first complex dataset by combining the distribution of the amplitude values ​​of the input image 210 with the random phase distribution 230. However, in the second and subsequent iterations, the data formation step 202B includes forming a complex dataset by combining (i) the distribution of the phase values ​​213A from previous iterations of the algorithm with (ii) the distribution of the amplitude values ​​of the input image 210.

[0073] Then, with reference Figure 2A The same method described is handled by Figure 2BThe complex dataset formed in step 202B is used to form the second iterative hologram 280B. Therefore, the description of this process will not be repeated here. The algorithm can stop when the second iterative hologram 280B has been computed. However, any number of further iterations of the algorithm can be performed. It will be understood that the third processing block 256 is only needed if a fourth processing block 259 is required or further iterations are needed. The output hologram 280B generally improves with each iteration. However, in practice, a point is usually reached where measurable improvement is no longer observable, or the positive benefits of performing further iterations are offset by the negative impact of the additional processing time. Therefore, the algorithm is described as iterative and convergent.

[0074] Figure 2C This represents an alternative embodiment for the second and subsequent iterations. The distribution of the phase value 213A from the previous iteration is fed back through the algorithm's processing block. The distribution of the amplitude value 211A is discarded to favor an alternative distribution of the amplitude values. In this alternative embodiment, the alternative distribution of the amplitude values ​​is derived from the distribution of the amplitude values ​​211 from the previous iteration. Specifically, processing block 258 subtracts the distribution of the amplitude values ​​of the input image 210 from the distribution of the amplitude values ​​211 from the previous iteration, scales the difference by a gain factor α, and subtracts the scaled difference from the input image 210. This is expressed mathematically by the following equation, where the subscript text and numbers represent the iteration number:

[0075] R n+1 [x, y] = F′{exp(iψ n [u, v])}

[0076] ψ n [u, v] = ∠F{η·exp(i∠R}) n [x, y])}

[0077] η=T[x,y]-α(|R n [x, y]|-T[x, y])

[0078] in:

[0079] F′ is the inverse Fourier transform;

[0080] F is the forward Fourier transform;

[0081] R[x, y] is the complex dataset output by the third processing block 256;

[0082] T[x, y] is the input or target image;

[0083] ∠ is the phase component;

[0084] Ψ is a phase-only hologram 280B;

[0085] η is a new distribution of amplitude value 211B; and

[0086] α is the gain factor.

[0087] The gain factor α can be fixed or variable. In some embodiments, the gain factor α is determined based on the size and rate of the input target image data. In some embodiments, the gain factor α depends on the number of iterations. In some embodiments, the gain factor α is only a function of the number of iterations.

[0088] In all other respects, Figure 2C Implementation examples and Figure 2A and Figure 2B The implementation is the same. It can be said that only the phase hologram Ψ(u, v) includes the phase distribution in the frequency domain or Fourier domain.

[0089] In some embodiments, a spatial light modulator is used to perform a Fourier transform. Specifically, holographic data is combined with second data that provides optical power. That is, the data written into the spatial light modulator includes holographic data representing an object and lens data representing a lens. When displayed on the spatial light modulator and illuminated with light, the lens data simulates a physical lens, i.e., it focuses light in the same way as a corresponding physical optical element. Therefore, the lens data provides optical or focusing power. In these embodiments, [the following can be omitted] Figure 1A physical Fourier transform lens 120 is used. Data representing the lens is known. This data can be referred to as a software lens. For example, a phase-only lens can be formed by calculating the phase delay caused by the optical path length at each point of the lens due to its refractive index and spatial variations. For example, the optical path length at the center of a convex lens is greater than the optical path length at the edges of the lens. An amplitude-only lens can be formed from Fresnel zone plates. In the field of computer-generated holography, it is also known how to combine data representing the lens with a hologram to perform a Fourier transform of the hologram without requiring a physical Fourier lens. In some embodiments, the lensed data is combined with the hologram by simple addition, such as simple vector addition. In some embodiments, a physical lens is used in combination with a software lens to perform a Fourier transform. Alternatively, in other embodiments, the Fourier transform lens is omitted entirely, allowing holographic reconstruction to occur in the far field. In further embodiments, the hologram can be combined with grating data in the same manner, i.e., data arranged to perform grating functions (such as image steering). Again, how to calculate such data is known in the art. For example, a phase-only grating can be formed by modeling the phase delay caused by each point on the surface of the blazed grating. An amplitude-only grating can be simply superimposed on an amplitude-only hologram to provide angular steering for holographic reconstruction. The second data providing lensing and / or steering may be referred to as an optical processing function or optical processing pattern to distinguish it from the holographic data, which may be referred to as an image forming function or image forming pattern.

[0090] In some embodiments, the Fourier transform is performed jointly by a physical Fourier transform lens and a software lens. That is, the software lens provides some of the optical power that contributes to the Fourier transform, while one or more physical optics provide the remaining optical power that contributes to the Fourier transform.

[0091] In some embodiments, a real-time engine is provided, arranged to receive image data using an algorithm and compute holograms in real time. In some embodiments, the image data is video comprising a sequence of image frames. In other embodiments, the holograms are pre-computed, stored in computer memory, and retrieved as needed for display on the SLM. That is, in some embodiments, a library of predetermined holograms is provided.

[0092] The embodiments described herein are by way of example only and relate to Fourier holography and Gerchberg-Saxton type algorithms. This disclosure is equally applicable to Fresnel holography and Fresnel holograms that can be computed using similar methods. This disclosure is also applicable to holograms computed using other techniques, such as point cloud-based methods.

[0093] Optical modulation

[0094] Spatial light modulators can be used to display diffraction patterns, including computer-generated holograms. If the hologram is a phase-only hologram, a spatial light modulator is needed to modulate the phase. If the hologram is a fully complex hologram, a spatial light modulator that modulates both the phase and amplitude can be used, or a first spatial light modulator that modulates the phase and a second spatial light modulator that modulates the amplitude can be used.

[0095] In some embodiments, the light modulation element (i.e., pixel) of the spatial light modulator is a cell comprising liquid crystal. That is, in some embodiments, the spatial light modulator is a liquid crystal device in which the optically active component is liquid crystal. Each liquid crystal cell is configured to selectively provide multiple light modulation levels. That is, each liquid crystal cell is configured at any time to operate at one light modulation level selected from multiple possible light modulation levels. Each liquid crystal cell can be dynamically reconfigured from multiple light modulation levels to different light modulation levels. In some embodiments, the spatial light modulator is a reflective liquid crystal on silicon (LCOS) spatial light modulator, but this disclosure is not limited to this type of spatial light modulator.

[0096] LCOS devices provide a dense array of light-modulating elements or pixels within a small aperture (e.g., a few centimeters wide). Pixels are typically about 10 micrometers or smaller, resulting in a diffraction angle of a few degrees, meaning the optical system can be compact. The small aperture of an LCOS SLM is much easier to fully illuminate than the larger aperture of other liquid crystal devices. LCOS devices are typically reflective, meaning the circuitry driving the LCOS SLM pixels can be buried under the reflective surface. This results in a higher aperture ratio. In other words, the pixels are densely packed, meaning there are virtually no dead zones between them. This is advantageous because it reduces optical noise in the playback field. LCOS SLMs use a silicon substrate, which has the advantage of making the pixels optically flat. This is particularly important for phase modulation devices.

[0097] The following are just examples for reference. Figure 3 To describe a suitable LCOS SLM, an LCOS device is formed using a single-crystal silicon substrate 302. It has a 2D array of square planar aluminum electrodes 301, spaced apart by gaps 301a, arranged on the upper surface of the substrate. Each electrode 301 can be addressed by circuitry 302a buried in the substrate 302. Each electrode forms its own planar mirror. An alignment layer 303 is disposed on the electrode array, and a liquid crystal layer 304 is disposed on the alignment layer 303. A second alignment layer 305 is disposed on a planar transparent layer 306, for example, made of glass. A single transparent electrode 307, for example, made of ITO, is disposed between the transparent layer 306 and the second alignment layer 305.

[0098] Each square electrode 301, together with the area covered by the transparent electrode 307 and the liquid crystal material in between, defines a controllable phase modulation element 308, commonly referred to as a pixel. Taking into account the space between pixels 301a, the effective pixel area, or fill factor, is the percentage of the total number of optically active pixels. By controlling the voltage applied to each electrode 301 relative to the transparent electrode 307, the properties of the liquid crystal material of each phase modulation element can be altered, thereby providing a variable delay for light incident upon it. The effect is to provide phase modulation only to the wavefront, i.e., no amplitude effect occurs.

[0099] The described LCOS SLM outputs spatially modulated light via reflection. The advantage of a reflective LCOS SLM is that the signal lines, grating lines, and transistors are located below the mirror, resulting in a high fill factor (typically greater than 90%) and high resolution. Another advantage of using a reflective LCOS spatial light modulator is that the thickness of the liquid crystal layer can be half that required when using a transmissive device. This significantly improves the switching speed of the liquid crystal (a key advantage for projecting moving video images). However, the teachings of this disclosure can be achieved similarly using a transmissive LCOS SLM.

[0100] Waveguide pupil / viewing window extender

[0101] In holographic projection systems such as head-up displays (HUDs), it is desirable to expand the exit pupil corresponding to the eyebox area or viewing window. Specifically, the viewer needs to be able to move their head left and right to view the complete image from any position within the limited area of ​​the eyebox / viewing distance. This is known as the eye motion box (EMB) or viewing window. Therefore, a pupil expander can be used to enlarge the EMB or viewing window. Typically, a pupil expander enlarges the EMB by dividing the amplitude of the incident wavefront to generate additional light.

[0102] Figure 4An example pupil expander including a waveguide is shown. The general principles of waveguides are known in the art and will not be described in detail here. The waveguide guides light within a layer between a pair of parallel reflective surfaces by internal reflection. The pupil expander is formed by a waveguide including a first graded / partially reflective surface 420 (e.g., a graded mirror with a reflectivity that varies with distance) and a second total reflective surface 410 (e.g., a mirror with essentially 100% reflectivity). In particular, the first reflective surface 420 includes a reflective coating whose reflectivity decreases along the length of the plate. This layer can be glass or plexiglass. Therefore, the waveguide can be a block or plate of glass or plexiglass. The first reflective surface can be a first surface of a glass block, and the second reflective surface can be a second surface of a glass block, wherein the first surface is opposite to and parallel to the second surface. Alternatively, the layer can be air, and the first and second reflective surfaces can be separate components, such as first and second mirrors spatially separated, to form an air gap within which light propagates by internal reflection.

[0103] Therefore, as Figure 4 As shown, an input beam 402, including input light (which may include spatially modulated light encoded with a picture (i.e., light from a picture / image or just a picture) or spatially modulated light encoded with a hologram, as described below), enters the waveguide through its input port. The waveguide is arranged to guide the light received at the input port to a viewing window. In the illustrated arrangement, the input port includes a gap in a first partial reflective surface 420 near one end of the waveguide, but other locations for the input port are also possible. The viewing window is the area or volume within which a viewer can view an image, as described herein. The angle of incidence of the input beam 402 causes the light to propagate along the length of the waveguide due to internal reflections from the first partial reflective surface 420 and the second total internal reflection surface 410. An exemplary light beam in... Figure 4 As shown in the diagram. Due to the gradient reflectivity of the first reflective surface 420, a certain proportion of light is transmitted through the first reflective surface 420 to provide multiple output light rays 404a-f (referred to here as "replicas" because they copy the input light rays) along the length of the waveguide. Thus, the first reflective surface 420 forms a viewing surface. The pupil (or viewing window) is expanded by the replicas formed by the waveguide. In particular, the size of the viewing window is increased by forming multiple replicas 404a-f along the length of the waveguide. Each replica 404a-f corresponds to a certain proportion of the amplitude (intensity or brightness) of the input beam 402. It is desired that the gradient along the length of the waveguide provides a decrease in the reflectivity of the first reflective surface 420 (or conversely, an increase in the transmittance) such that each replica 404a-f has substantially the same amplitude. Therefore, a viewer 430 at the eyebox at a viewing distance from the first reflective surface 420 can view the image at any position within the expanded viewing window, as indicated by arrow 440.

[0104] like Figure 4 The waveguide shown extends the viewing window in one dimension, corresponding to the longitudinal direction along which the beam propagates within the waveguide, as indicated by arrow 440. As those skilled in the art will understand, the viewing window can be extended in two dimensions if desired by using two orthogonal waveguides.

[0105] The first reflective surface 420 of the waveguide may be coated with a layer comprising a large number of thin films (e.g., 25 or more films) to provide the necessary gradient reflectivity. Specifically, as described above, such thin films or similar coatings need to provide a decreasing reflectivity with propagation distance, thereby increasing transmittance so that the brightness (light intensity) of each copy 404a-f is substantially constant. The amplitude of the propagating beam decreases with propagation distance due to the output of copies 404a-f and due to any other optical losses (e.g., imperfect reflection from the second reflective surface 410). Therefore, the gradient of the first reflective surface 420 is designed to account for the decrease in the intensity of the propagating beam with propagation distance, while ensuring that each copy 404a-f has substantially the same intensity, so that the viewed image has uniform brightness across the entire viewing window (i.e., at all viewing positions).

[0106] In this implementation, a waveguide pupil expander is used to expand the viewing window of an input beam 402 of different wavelengths. For example, a composite color image can be formed from multiple individual color (monochrome) images, such as red, green, and blue images. Therefore, Figure 4 The waveguide may need to provide the same performance for multiple individual wavelengths of input beam 402. In this embodiment, the individual wavelengths are within the visible spectrum. Therefore, the graded reflectivity of the first reflective surface 420 needs to output copies 404a-f with substantially constant intensity with propagation distance for each wavelength (e.g., for red, green, and blue incident beams 402). However, it is difficult, if not impossible, to form a first reflective surface 420 providing the same performance / optical response for two or more wavelengths along the entire length of the waveguide using thin films and coatings. This problem is illustrated in Figure 5 The diagram illustrates the relationship between the transmittance of the corresponding red 510 and green 520 input beams along the example first reflective surface 420 and the distance. Figure 5As shown, the transmittance of the first reflective surface varies considerably with distance for both red and green wavelengths. Specifically, the first reflective surface exhibits different reflectance responses to red and green wavelengths. It can be said that the first reflective surface is “non-achromatic”—the reflectance / transmittance of the first reflective surface (e.g., a thin film or coating) is wavelength-dependent and therefore varies depending on the incident wavelength of light. This means that copies 404a-f have different intensities at different wavelengths (colors). When viewed from different positions within the viewing window, this results in inhomogeneity between different monochrome images. In other words, the color balance of the composite color image across the entire viewing window is uneven.

[0107] Exemplary embodiments of this disclosure include holographic display apparatus and methods that use waveguides as pupil expanders to provide a more uniform color image across the entire expanded viewing window.

[0108] First Embodiment

[0109] Figure 6 A holographic display device comprising a waveguide forming a waveguide pupil expander, according to a first exemplary embodiment of the present disclosure, is shown.

[0110] The holographic display device includes an image generation unit arranged to form a first image (also referred to as a "first picture") and a second image (also referred to as a "second picture"). A first monochromatic channel (also referred to as a "first display channel") is arranged to form the first picture and includes a first light source 610, a first collimating lens 612, and a first dichroic mirror 614. The first dichroic mirror 614 is arranged to reflect light of a first wavelength along a common optical path to illuminate a spatial light modulator (SLM) 640. The first wavelength of the light corresponds to the first display channel of a first color (e.g., red). A second monochromatic channel (also referred to as a "second display channel") is arranged to form the second picture and includes a second light source 620, a second collimating lens 622, and a second mirror 624. The second mirror 624 is arranged to reflect light of a second wavelength along a common optical path to illuminate the SLM 640. The second wavelength of the light corresponds to the second monochromatic channel of a second color (e.g., green). As described below, in other embodiments, the image generation unit may include a third monochrome / display channel (equivalent to the first and second channels) arranged to form a third image, wherein the third color channel corresponds to the wavelength of light of a third color (e.g., blue). In the illustrated embodiment, the SLM640 includes a single array of light-modulated pixels (e.g., LCOS) illuminated by light of the first and second wavelengths. In other embodiments, the SLM640 may include separate arrays of light-modulated pixels illuminated by corresponding first and second wavelengths.

[0111] The holographic display device also includes a holographic controller 602, which is arranged to control the image generation unit, particularly the light output by the image generation unit as described herein. An SLM 640 outputs first spatially modulated light of a first color corresponding to a first image to form a first monochrome image (e.g., a red image) on a light-receiving surface 670, such as a screen or diffuser. A first monochrome computer-generated hologram is calculated by the holographic controller 602 and encoded on the SLM 640, for example, by a display driver 642. The SLM 640 displays the first hologram and is illuminated by light of the first color from a first color / display channel to form a first holographic reconstruction on the light-receiving surface 670 located at the playback plane. Similarly, the SLM 640 outputs second spatially modulated light of a second color corresponding to a second image to form a second monochrome image (e.g., a green image) on the light-receiving surface 670. A second monochrome computer-generated hologram is encoded on the SLM 640 by the holographic controller 602. The SLM640 displays a second hologram and is illuminated by light of a second color from a second color / display channel to form a second holographic reconstruction on a light-receiving surface at the playback plane. In the illustrated arrangement, a beam splitter cube 630 is arranged to separate the input light to the SLM640 and the spatially modulated light output from the SLM640. A Fourier lens 650 and a mirror 660 are provided in the optical path of the output spatially modulated light to the light-receiving surface 670. The first / second images can be said to be formed on the light-receiving surface 670. The first / second images are first / second holographic reconstructions of the corresponding first / second holograms. Therefore, a composite color image can be formed on the light-receiving surface 670 by combining the first and second images. A projection lens 680 is arranged to project the first and second images formed on the light-receiving surface 672 onto the input port of a pupil expander in the form of a waveguide 690. A viewer 608 can view a magnified image of the picture from an expanded eyebox (“viewing window”) formed by the waveguide 690 due to the optical power of the projection lens 680. Waveguide 690 includes an optically transparent medium separated by first and second reflective surfaces, as referenced above. Figure 4 Therefore, the holographic display device has an "indirect view" configuration—that is, the viewer does not directly view the holographic reconstruction, but rather views the image formed on the light-receiving surface 670.

[0112] The holographic display device also includes a viewer tracking system comprising an eye-tracking camera 606 and an eye-tracking controller 604. As known in the art, the eye-tracking camera is arranged to capture images of the viewer's eyes to track eye position and thus viewing position within a viewing window. The eye-tracking controller 604 provides feedback to the holographic controller 602 indicating the current viewing position. In an example embodiment, the holographic controller 602 is arranged to dynamically adjust the relative brightness of the first and second images based on the current viewing position. Specifically, the relative brightness of the first and second images can be adjusted to compensate for the difference in reflectivity of light of the first and second wavelengths on the first (partial) reflective surface of the planar waveguide at a propagation distance corresponding to the current viewing position. In other words, the holographic controller 602 is arranged to adjust the relative brightness of the first and second images seen at the current viewing position to compensate for the difference in reflectivity response of the second reflective surface to light of the corresponding first and second wavelengths. This maintains perceived color balance at different viewing positions within the viewing window. Calibration data can be used to fine-tune the brightness of one or more monochrome images in real time to maintain color balance. Calibration data can be obtained through a calibration process that includes measuring the relative brightness of each monochrome image at multiple different viewing positions within the viewing window.

[0113] In some embodiments, the holographic controller 602 may be arranged to adjust the relative brightness of the first and second images based on the current viewing position by adjusting one or more drive signals (e.g., provided by a light source controller) to the first light source 610 and the second light source 620. The drive signals to the light sources control the power to the light sources and thus the optical power of the output light. In other embodiments, the holographic controller 602 may be arranged to adjust the relative brightness of the first and second images by adjusting one or more of the first and second computer-generated holograms. For example, the quantization scheme used to calculate the first and / or second holograms may be changed based on the current viewing position. The quantization scheme may be changed to reduce the range of optical modulation within which permissible levels of optical modulation are allocated, which may change the intensity of the pixels in the calculated hologram. An example of adjusting the quantization scheme for this purpose is described below.

[0114] Additionally, a photodetector (not shown) is arranged to measure the optical power of a non-image region of the playback field formed on the first / second holographic reconstruction on the light-receiving surface 670. The photodetector can be located in front of or behind the light-receiving surface 670. The photodetector can provide the optical detection signal to the holographic controller 602 as part of a closed-loop feedback system to maintain the brightness of the image content when the image content changes, as referenced below. Figure 8 Further description.

[0115] In addition to feedback inputs from the viewer tracking system and the closed-loop feedback system (not shown), the holographic controller 602 may also receive other external and internal inputs 600 for generating computer-generated holograms as known in the art. Such inputs can determine the image content to be displayed by the holographic display device.

[0116] Figure 6 The illustrated holographic display device has an image generation unit comprising a first color (e.g., red) display channel arranged to display a first monochrome hologram and a second color (e.g., green) display channel arranged to display a second monochrome hologram, as an example only. In the example embodiment, three or more display channels may be provided, configured to display corresponding monochrome holograms. For example, a full-color composite image / picture can be formed by displaying corresponding red, green, and blue monochrome holograms.

[0117] waveguide geometry

[0118] Figure 7A -C illustrates the geometry of the waveguide pupil expander according to an embodiment. Specifically, Figure 7B and 7C The positioning and propagation of the principal ray of the input image beam 702 (as described above) along the waveguide, and the output of a copy visible to the viewer 730, are shown. The waveguide includes a first partial reflective surface 720 and a second total reflection surface 710 as described above.

[0119] Figure 7C The geometry shown can be represented by the following equation.

[0120] w1 = 2dtan(θ - φ)

[0121] w² = 2dtan(θ + φ)

[0122] p1 = 2dsin(θ - φ)

[0123] p2 = 2dsin(θ + φ)

[0124] tan(θ-φ)=s1 / d=(s1+w1-s) / d1

[0125] tan(θ+φ)=s2 / d=(s2+w2-s) / d1

[0126] s1d1=(s1+w1-s)d

[0127] s1 Δd = 2d 2 tan(θ-φ)-sd (1)

[0128] s2d1=(s2+w2-s)d

[0129] s2 Δd = 2d 2 tan(θ+φ)-sd (2)

[0130] Subtract (1) from (2)

[0131] Δd=2d 2 (tan(θ+φ)-tan(θ-φ)) / (s2-s1)

[0132] Subtract s2(1) from s1(2)

[0133] 0 = 2s²d 2 tan(θ-φ)-ss2d-2s1d 2 tan(θ+φ)-ss1ds=(2s2dtan(θ-φ)-2s1dtan(θ+φ)) / (s1+s2)

[0134] =(s2w1-s1w2) / (s1+s2)

[0135] Closed-loop feedback system

[0136] Figure 8 An example holographic playback field 800 according to an embodiment is shown. For example, the illustrated holographic playback field includes a synthetic color image comprising three-color holographic reconstructions formed by displaying three corresponding monochrome holograms (e.g., red, green, and blue) using a holographic display device according to an embodiment. As described below, the synthetic color image can be perceived by a viewer either by simultaneously forming the three-color holographic reconstructions or by sequentially forming the three-color holographic reconstructions within the integration time of the human eye.

[0137] The holographic playback field 800 includes the entire playback field area of ​​the playback plane. As previously described, the playback field is the area on the playback plane where light can be redirected using a hologram. According to an embodiment, a main region 810 is designated for displaying image content, corresponding to the desired synthetic color image that the viewer wants to see. The main region 810 can be considered as a first / second / third picture region (or image region). Additionally, first, second, and third sub-regions 820a-c, spatially separated from the main region 810, are designated for displaying non-image content. Sub-regions 820a-c can be considered as first / second / third non-picture (or non-image) regions. Specifically, the first monochrome holographic reconstruction includes the first non-image region 820a, the second monochrome holographic reconstruction includes the second non-image region 820b, and the third monochrome holographic reconstruction includes the third non-image region 820c. Each of the first, second, and third non-image regions 820a-c displays corresponding monochrome non-image content, such as defined color patches or patterns of image pixels. Non-image regions 820a-c are spatially separated from image region 810, for example, they are arranged around the periphery of image region 810. Figure 8As shown, the first, second, and third non-image regions 820a-c are arranged near each boundary of the holographic playback field 800. As those skilled in the art will understand, each monochrome hologram is calculated to include the desired image content and non-image content. The purpose of the non-image content is described below.

[0138] As previously described, the holographic reconstruction formed by displaying a hologram includes image pixels formed within a playback field at the playback plane. As described herein, when a hologram is displayed using a pixelated spatial light modulator (SLM), each hologram pixel calculated contributes to each image pixel formed in the playback field. Furthermore, the output power of the SLM depends on the power of the incident light because the hologram only redirects the light. The power of the incident light is typically kept constant. As a result, the intensity or brightness of each image pixel, and thus the resulting image, depends on the number of bright image pixels in the playback field. However, the number of displayed image pixels varies dynamically depending on the displayed image. Therefore, visible variations in image brightness may occur during the display of an image that dynamically changes over time.

[0139] Therefore, the embodiments include a closed-loop feedback system, as referenced above. Figure 6 Brief description for use Figure 8 The brightness of the image is maintained by the non-image content shown. Specifically, the closed-loop feedback system ensures that the brightness of each monochrome image remains substantially constant based on the measurement of the optical power of the corresponding monochrome sub-image / non-image regions 820a-c. As those skilled in the art will understand, the brightness of the image pixels in the non-image content in each non-image region 820a-c varies proportionally to the brightness of the image pixels in the corresponding monochrome image content in image region 810. Therefore, the measurement of the optical power of the non-image content indicates the brightness of the image content. Thus, the feedback of the measured optical power can be used to maintain the brightness of each monochrome image in the main (image) region 810 substantially constant by correspondingly setting the desired optical power of the sub-(non-image) image regions 820a-c as a parameter of the error correction feedback loop. In some embodiments, the optical controller and / or holographic controller for each monochrome light source (e.g., laser) is arranged to maintain the optical power of the monochrome image in the main image region 810 at a substantially constant value in response to the detected optical power of the non-image content displayed in the sub-image regions 820a-c.

[0140] For a given monochrome holographic reconstruction / image, the brightness of the main (image) region 810 can be maintained by using a reference optical power for the secondary (non-image) region 820. Specifically, the brightness of the image region 810 can be maintained at a reference level where the optical power of the non-image region 820 is at the reference value. In some embodiments, the light controller is arranged to decrease the optical power of the light output from the light source if the detected optical power of the non-image region 820 is greater than the reference optical power and / or increase the optical power of the light output from the light source if the detected optical power of the non-image region 820 is less than the reference optical power.

[0141] A reference optical power for the non-image region 820 can be determined in any way to correspond to a desired brightness, for example, based on luminance measurements calculated by taking into account the area of ​​the non-image region 820 (e.g., the number of image pixels) and possible other factors (e.g., sensing wavelength scaling). For example, the reference optical power can be hardwired in the system, predetermined by the user, or selected by the user. In some embodiments, the reference optical power is determined from the optical power of the non-image region 820 at an earlier time, for example, to keep the image brightness constant over time.

[0142] In some embodiments, feedback control regarding a reference value may include a feedback loop, for example, where the optical controller proportionalizes the output optical power to a value obtained by subtracting the detected optical power from the reference optical power to provide a proportional control term. A gain factor may be used to multiply this difference to convert the feedback signal to units related to brightness (e.g., luminance). Other control terms may also be used, such as differential and integral terms or other, for example, nonlinear terms. In some embodiments, feedback control may include a hysteresis element, such that the output optical power may be increased if the detected optical power is less than the reference optical power by more than a first threshold amount, and the output optical power may be decreased if the detected optical power exceeds the reference optical power by more than a second threshold amount.

[0143] Embodiments of this disclosure are arranged to dynamically adjust the brightness of each monochrome image based on the viewing position detected by the viewer tracking system, so that the relative brightness of the monochrome images is substantially constant for all viewing positions within the extended viewing window. Therefore, embodiments can implement a closed-loop feedback system arranged to dynamically adjust the brightness of each monochrome image based on the amount of image content displayed (e.g., the number of image pixels formed in a designated image region 810). The closed-loop feedback system can be configured to provide primary feedback to maintain the brightness of the image content as the image content changes over time. The viewer tracking system can be configured to provide secondary feedback as a function of the detected viewing position to compensate for the non-achromatic aberration of the partial reflective layer of the waveguide pupil expander as described above. For example, the secondary feedback can provide an offset value as a function of the viewing position. Therefore, the primary feedback can provide primary drive signal correction to the drive signal to the light source, which corrects for brightness changes caused by the amount of variation in the holographic image content. The secondary feedback can provide secondary drive signal correction, which compensates for the non-achromatic graduated filter. The secondary drive signal correction for the detected viewing position can be determined by a calibration process as described herein.

[0144] As previously mentioned, various techniques can be used to adjust the brightness of each monochrome image. In some embodiments, a light controller can be used to adjust the brightness of the monochrome image by changing the drive signal of the corresponding monochrome light source used to control the optical power of the output light. By dynamically changing the drive signal, the optical power of the display device (i.e., SLM) can be dynamically increased or decreased. In other embodiments, holographic calculations can be modified to adjust the brightness of the monochrome image, as further described below.

[0145] Second Embodiment

[0146] Figure 9 A holographic display device including a waveguide pupil expander is shown according to a second exemplary embodiment of the present disclosure.

[0147] Figure 9 The holographic display device shown is similar to Figure 6A holographic display device. Specifically, the holographic display device includes an image generation unit arranged to form a first image (or first picture) and a second image (or second picture). A first monochrome / display channel (e.g., a red channel) includes a first light source 910, a first collimating lens 912, and a first dichroic mirror 914, arranged to illuminate the SLM 940 with light of a first wavelength. A second monochrome / display channel (e.g., a green channel) includes a second light source 920, a second collimating lens 922, and a second reflecting mirror 924, arranged to illuminate the SLM 940 with light of a second wavelength. The first display channel is arranged to form a first image (e.g., a red image) on a light-receiving surface 970. A first monochrome computer-generated hologram is encoded on the SLM 940 by a holographic controller 902. The SLM 940 displays the first hologram and is illuminated by light from the first color channel to form a first holographic reconstruction on the light-receiving surface 970 located at the playback plane. Similarly, the second display channel is arranged to form a second image (e.g., a green image) on the light-receiving surface 970. The second monochrome computer-generated hologram is encoded on the SLM940 by the holographic controller 902. The SLM940 displays the second hologram and is illuminated by light from the second color channel to form a second holographic reconstruction on the light-receiving surface at the playback plane.

[0148] The holographic display device also includes a beam splitter cube 930, which is arranged to separate the input light to the SLM 940 and the output light from the SLM 940. However, with Figure 6 Conversely, the holographic display device is a direct viewing system. In the illustrated arrangement, lens 950 is located in the optical path of the spatially modulated light output by SLM 940. Lens 950 is optional. Viewer 908 can directly view the spatially modulated light from the spatial light modulator. As described above, in some embodiments, the lens of the viewer's eye forms a holographic reconstruction on the retina of the eye. In these embodiments, it can be said that the viewer receives spatially modulated light encoded with a hologram. In other embodiments, the viewer receives light of a picture or light encoded with a picture. The picture can be formed at an intermediate plane in free space. Waveguide 990 includes an optically transparent medium separated by the first and second reflective surfaces as described above. Therefore, the holographic display device has a "direct viewing" configuration, i.e., the viewer directly views the display device (i.e., the spatial light modulator), and Figure 6 The light-receiving surface is optional.

[0149] Holographic calculations for adjusting the brightness of individual monochrome images

[0150] The hologram encoded on the SLM can be altered to change the proportion of incident light spatially modulated by the SLM according to the hologram. The remainder of the incident light is unmodulated and contributes to the zero-order spot, referred to herein as the “DC spot”. Therefore, by changing the proportion of incident light spatially modulated by the SLM, the intensity of the corresponding holographic reconstruction can be altered, since the hologram only redirects the spatially modulated component of the incident light.

[0151] When calculating a computer-generated hologram displayed by a holographic display device, as described herein, the value of each hologram pixel is restricted to one of a plurality of permissible light modulation levels. Permissible light modulation levels (also known as “grayscale levels”) are defined in a quantization scheme.

[0152] One technique that can be used to alter the proportion of incident light contributing to the zero-order / DC spot and thus the proportion of incident light spatially modulated by the SLM is to change the permissible light modulation level. The permissible modulation level can be represented as a vector, as a graph on a complex plane. The step of “quantization” (as described above) can also be called “constraint” because the calculated values ​​are effectively constrained to permissible values. In the case of phase-only holography, the multiple permissible modulation levels are multiple permissible phase values ​​(with unit amplitude) on a circle in the complex plane. In other words, the number of modulation levels used to encode the hologram is changed (e.g., based on or according to the detected viewing position).

[0153] Figure 10 An example of a typical quantization scheme including multiple permissible optical modulation levels is shown. Specifically, Figure 10 A set of permissible optical modulation levels is shown, plotted in the complex plane in the range of 0 to 2π.

[0154] As referenced above Figures 2A to 2C The hologram calculated using Fourier transform comprises a 2D array of complex numbers, and in the case of phase-only holography, each complex number is converted into an allowable modulation level (or gray level) by setting the amplitude component to a unit value and quantizing the phase value according to one of the gray levels, which can represent the gray level on the pixel of the spatial light modulator (e.g., ...). Figure 2A (See processing block 253).

[0155] Figure 10 An example with 16 permissible modulation levels 1003 (labeled 0 to 15) is shown, represented as a vector on a unit amplitude phase circle 1001 in a complex plane within a “balanced” hologram, as further described below. Although in principle the quantization point can be... Figure 10 The unit amplitude phase circle is randomly distributed, but it was found that the best quality holographic image can be obtained when the quantization points are uniformly distributed.

[0156] Therefore, in Figure 10 In the example, during the quantization step, each calculated complex number in the complex array of the calculated hologram is assigned to one of the permissible modulation levels from 0 to 15. For example, any mathematical method (such as using a set of comparators) can be used to identify the permissible modulation level that is closest to the calculated complex value in the complex plane.

[0157] Conventionally, for high-quality holograms in phase-only holography, the entire range of permissible phase values ​​(2π) is used as the range of permissible phase values. That is, the permissible phase values ​​(e.g., Figure 10 The values ​​from 0 to 15 in the spectrum should span 2π in the complex plane. This method minimizes the zeroth-order undiffracted / unmodulated light, which represents unwanted "optical noise" in the holographic image, as described above. Conversely, embodiments of this disclosure can dynamically reduce or "truncate" the range of permissible phase values ​​in an "unbalanced" manner to increase the zeroth-order undiffracted / unmodulated light and thus reduce the optical power of the spatially modulated light. Reducing the optical power of the spatially modulated light reduces the intensity of the displayed image.

[0158] Specifically, holographic controllers 602 and 902 can dynamically compute (or modify) the monochromatic hologram encoded on the SLM640 and 940 to change the distribution and / or range of allowable phase values ​​in response to feedback from the closed-loop feedback system. This can be achieved in various ways. For example, during real-time hologram computation, holographic controllers 602 and 902 can dynamically change the range of allowable modulation levels to dynamically change (decrease or increase) the proportion of output light from the SLM640 and 940 corresponding to the spatially modulated light forming the holographic reconstruction. In another example, holographic controllers 602 and 902 can adjust the weights of the hologram values ​​using a weighting or scaling factor after hologram computation. Specifically, holographic controllers 602 and 902 can multiply each quantized phase value by a scaling factor to reduce the range of allowable phase values ​​(e.g., multiplying each quantized phase value by 1 / 2 reduces the range from 0 to 2π to 0 to π). In another example, holographic controllers 602, 902 can alter the mapping of allowable modulation levels (i.e., gray levels) to voltages of the pixels of SLMs 640, 940. Specifically, a lookup table (LUT) can be used to associate each gray level with an analog voltage (if the device has an analog substrate) or a so-called bit-plane sequence (if the SLM has a digital substrate). Values ​​in the LUT can be manipulated to provide a reduced phase range. In another example, holographic controllers 602, 902 can change the distribution of phase values ​​within a range of 0 to 2π (or a more limited range) to provide a non-uniform distribution. For example, the interval between allowable modulation levels (e.g., phase values) can depend on the modulation level (e.g., phase). In another example, each allowable modulation level can be limited by a bin size, which can depend on the modulation level. These and other examples are described in more detail below.

[0159] Figure 10 The modulated level 1003 shown is uniformly distributed around the center of the phase circle. When combined, Figure 10 All vectors represented in the diagram have no net effect. In this arrangement, the modulation scheme can be said to be "balanced". Therefore, references to "balanced hologram", "balanced modulation scheme", and "balanced system" herein refer to a vector sum representing the permissible modulation level of a hologram, modulation scheme, or holographic system that is zero. Conversely, references to "unbalanced hologram", "unbalanced modulation scheme", and "unbalanced system" refer to the corresponding vector sum being non-zero.

[0160] It has been found that phase calculation algorithms (such as those based on the Gerchberg-Saxton algorithm) inherently balance the use of modulation levels in the hologram. That is, they use gray levels equally. A system is said to be balanced when the modulation scheme and the hologram are balanced. Specifically, if the complex value of each hologram pixel is represented by a vector in the complex plane, the sum of all vectors will be zero. Therefore, in implementing... Figure 10 In the balanced modulation scheme shown and the balanced system calculated using the hologram with the Gerchberg-Saxton algorithm, the intensity of the zero-order unmodulated light (i.e., the DC spot) is minimized.

[0161] Therefore, by dynamically altering the hologram encoded on a phase-only LCOS SLM to have different distributions of permissible phase values, an "unbalanced" system can be achieved to change (increase / decrease) the intensity of the zero-order unmodulated light and thus form the intensity of the spatially modulated light in the holographic reconstruction. Specifically, this can be achieved by changing from a balanced system where the vector sum of the hologram pixels is zero to one where the vector sum of the hologram pixel values ​​is non-zero and therefore represents... Figure 11 The unbalanced system of the DC light spot shown is used to reduce the intensity of spatially modulated light.

[0162] The imbalance of the system can be achieved in a variety of ways, such as by making the modulation scheme unbalanced and / or by making the hologram unbalanced.

[0163] Unbalanced truncation of the phase circle

[0164] In some embodiments of phase holography, an unbalanced "truncation" of the phase circle used for the allowable modulation level can be used. As mentioned above, typically in phase-only holography, the entire 2π allowable phase value is required to achieve a high-quality holographic image. That is, the allowable phase value (e.g., Figure 10 The gray levels (0 to 15) should span 2π in the complex plane. If the range of phase values ​​is allowed to decrease from 2π, the phase circle is said to be "truncated". If the hologram pixel values ​​still sum to zero in the complex plane, it can be said to have "balanced truncation". An example of balanced truncation is shown in... Figure 12A As shown in [the image]. Figure 12A In the diagram, the arcs of the phase circle, represented by solid lines, indicate the regions for permissible phase values, while the arcs, represented by dashed lines, represent the truncation regions for disallowed phase values. When the truncation is balanced, each permissible modulation level on the phase circle has equal and opposite points on opposite sides of the phase circle's diameter, for example... Figure 12A The pair of points marked "X" are shown in the image. Specifically, in... Figure 12A A pair of points labeled X in the diagram have equal and opposite real and imaginary parts. However, when the truncation is balanced, such as... Figure 12AAs shown, the zero-order undiffracted / unmodulated light is minimized in the same way as the allowable phase value in the normal 2π range, since the sum vector is still zero.

[0165] Therefore, an "unbalanced" truncation is needed to generate non-zero sum vectors. An example of unbalanced truncation is... Figure 12B As shown in the diagram. It can be seen that certain modulation levels on the phase circle (e.g., points marked "X" on the real arc) do not have the same and opposite modulation levels on the other side of the phase circle (because points on the phase circle with diameters opposite to point X fall within the imaginary arc). Therefore, in cases such as... Figure 12B As shown, under the condition of an unbalanced distribution of permissible phase values, the sum vector is non-zero, and the intensity of the zero-order undiffracted / unmodulated light (i.e., DC spot) increases. As a result, the intensity of the spatially modulated light is reduced, thereby reducing holographic reconstruction.

[0166] By dynamically altering the imbalance truncation, the intensity of the spatially modulated light used to form the holographic reconstruction can be dynamically changed. Specifically, in embodiments, holographic controllers 602 and 902 can dynamically calculate (or modify pre-calculated holograms) the holograms encoded on the SLMs 640 and 940 in real time to provide an imbalance distribution or range of permissible phase values ​​(i.e., gray levels) in real time. The degree of gray level imbalance, in turn, determines the proportion of incident light that contributes to the zero-order spot.

[0167] As those skilled in the art will understand, imbalance can be achieved in various ways, such as by modifying the holographic data after or during its calculation, or by making the modulation scheme unbalanced (altering the distribution of the permissible phase values) through voltage conversion or other means.

[0168] Therefore, in embodiments, holographic controllers 602, 902 can recalculate the first / second hologram in response to a viewing position determined by a viewer tracking system and / or in response to a closed-loop feedback system. Specifically, the first / second hologram is recalculated by changing the quantization scheme to alter the proportion of light in the zero-order spot that contributes to holographic reconstruction, thereby changing the proportion of light spatially modulated and contributing to the first / second picture (or image). As described above, changing the quantization scheme can include reducing the range of light modulation levels distributed within it. For example, changing the quantization scheme can include changing the vector sum of the vectors representing the allowable light modulation levels on a complex plane.

[0169] Head-up display

[0170] A commercially viable display for the automotive industry has been proposed using a dynamic spatial light modulator that includes a phase modulator. The main advantages of phase modulation are optical efficiency and fault tolerance.

[0171] Figure 13A system for forming a virtual image using the pupil expander 1305 of this disclosure is shown. The physical extent of the holographic playback field 1301 is limited by the maximum diffraction angle θm of the SLM. Associated with each point in the playback field 1301 is a ray cone that diverges at an angle φ, and the principal ray also diverges. In Fourier holographic playback, the angle φ gives the reconstructed field of view (FOV). A small FOV is associated with a large playback field size. The angle of the principal ray is determined by the curvature of the playback field. Projection optics 1303 (e.g., a lens or powered mirror) are used to view the virtual image of the playback field 1301 at large distances (e.g., 2m, 5m, 15m, etc.). This is in Figure 13 As shown in the image. Figure 13 The combiner 1307 shown is typically a car windshield. If the principal rays from each light cone are projected backward, they will meet at a distance u from the lens, determined by the curvature of the projection field surface. An eye pupil placed at a distance v from the projection lens 1303 (where v is given by the lens equation) will view the complete image. Now, the field of view (FOV) is... Figure 13 The total field of view encompassed by the pupil of the middle eye (1309).

[0172] In addition to viewing the full image, the driver must also be able to turn their head within a limited area at a designed distance v. In the automotive industry, this is known as the "eye-tracking box" (EMB)—but in this document, it is more commonly referred to as the viewing window. To maintain a full view of the image while the eyes move around the EMB, a pupil expander 1305 is employed. The pupil expander 1305 expands the EMB by generating excess light by dividing the amplitude of the incident wavefront, as described above. This excess light enhances the field of view directly below the eyes. A further result is that a larger image size can be supported, and therefore a larger FOV can be supported.

[0173] The applicant has fabricated a waveguide based on glass and an air cavity for use as a pupil expander. The uniformity of the expander can be customized using gradient reflectors or custom diffraction gratings on the waveguide surface or throughout. Alternative techniques can be used for the pupil expander. For example, a microlens array can cause the virtual image to be positioned closer to the back of the expander. A key advantage of the HUD including the display device of this disclosure is the reduction in eye focus correction between the virtual image and the road viewed by the driver.

[0174] Additional features

[0175] The embodiments relate to electrically activated LCOS spatial light modulators by way of example only. The teachings of this disclosure can be equivalently implemented on any spatial light modulator capable of displaying computer-generated holograms according to this disclosure, such as any electrically activated SLM, optically activated SLM, digital micromirror device, or microelectromechanical device.

[0176] In some embodiments, the light source is a laser, such as a laser diode. In some embodiments, the detector is a photodetector, such as a photodiode. In some embodiments, the light-receiving surface is a diffuser surface or screen, such as a diffuser. The holographic projection system of this disclosure can be used to provide an improved head-up display (HUD) or head-mounted display. In some embodiments, a vehicle is provided that includes a holographic projection system mounted in the vehicle to provide a HUD. The vehicle can be a motor vehicle, such as a car, truck, van, delivery truck, motorcycle, train, airplane, boat, or ship.

[0177] The quality of holographic reconstruction can be affected by the so-called zero-order problem, which is a result of the diffraction properties of pixelated spatial light modulators. This zero-order light can be considered "noise" and includes, for example, specular reflections and other unwanted light from SLMs.

[0178] In examples of Fourier holography, this "noise" is concentrated at the focal point of the Fourier lens, resulting in a bright spot at the center of the holographic reconstruction. Zero-order light can be simply blocked out; however, this means replacing the bright spot with a dark spot. Some embodiments include angle-selective filters to remove only collimated rays of the zeroth order. Embodiments also include methods for managing the zeroth order as described in European Patent 2030072, which is incorporated herein by reference in its entirety.

[0179] In this embodiment, only the primary playback field is used, and the system includes physical blocks, such as baffles, arranged to restrict the propagation of higher-level playback fields through the system.

[0180] In the above embodiments, holographic reconstruction is the synthesis of a color image. In some embodiments, a method known as Spatially Separated Color (SSC) is used to provide color holographic reconstruction. In other embodiments, a method known as Frame Order Color (FSC) is used.

[0181] The SSC method uses three spatially separated arrays of light-modulated pixels for three monochrome holograms. The advantage of the SSC method is that the images can be very bright because all three holographic reconstructions can be formed simultaneously. However, if three spatially separated arrays of light-modulated pixels are provided on a shared SLM due to spatial constraints, the quality of each monochrome image will be suboptimal because only a subset of the available light-modulated pixels is used for each color. Therefore, relatively low-resolution color images are provided.

[0182] The FSC method can use all pixels of a shared spatial light modulator to sequentially display three monochrome holograms. The monochrome reconstruction loop (e.g., red, green, blue, red, green, blue, etc.) is fast enough that a human viewer perceives a multicolor image from the integration of the three monochrome images. An advantage of FSC is that the entire SLM can be used for each color. This means the quality of the resulting three-color images is optimal because all pixels of the SLM are used for each color image. However, a disadvantage of the FSC method is that the brightness of the synthesized color image is lower than that of the SSC method—about one-third—because each monochrome illumination event only occurs for one-third of the frame time. This deficiency can be addressed by overdriving the laser or using a more powerful laser, but this requires more power, leading to higher costs and increased system size.

[0183] The methods and processes described herein can be embodied on a computer-readable medium. The term "computer-readable medium" includes media arranged for temporary or permanent storage of data, such as random access memory (RAM), read-only memory (ROM), buffer memory, flash memory, and cache memory. The term "computer-readable medium" should also be considered to include any medium or combination of media capable of storing instructions for machine execution, such that when the instructions are executed by one or more processors, the machine performs, wholly or partially, any or all of the methods described herein.

[0184] The term "computer-readable medium" also covers cloud-based storage systems. The term "computer-readable medium" includes, but is not limited to, one or more tangible and non-transitory data repositories (e.g., data volumes) in the example forms of solid-state storage chips, optical discs, disks, or any suitable combinations thereof. In some example embodiments, instructions for execution may be transmitted by a carrier medium. Examples of such carrier media include transient media (e.g., propagation signals for transmitting instructions).

[0185] It will be apparent to those skilled in the art that various modifications and variations can be made without departing from the scope of the appended claims. This disclosure covers all modifications and variations within the scope of the appended claims and their equivalents.

Claims

1. A display device arranged to form a first visible image at all viewing positions within a viewing window, the display device comprising: An image generation unit is configured to output a first spatially modulated light of a first color, wherein the first spatially modulated light corresponds to a first image. The image generation unit includes a controller configured to control the first spatially modulated light. Waveguide pupil expander, comprising: An input port is arranged to receive first spatially modulated light from the image generation unit; A viewing window in which the first image can be viewed; and A pair of parallel reflective surfaces are arranged to guide first spatially modulated light from the input port to the viewing window via a series of internal reflections from the pair of parallel reflective surfaces. The first reflective surface of the pair of parallel reflective surfaces partially transmits light of a first color, the transmittance of which is non-achromatic. The reflectivity / transmittance of the first reflective surface varies with distance along the waveguide. A viewer tracking system is configured to determine the viewer's position within the viewing window. Wherein, the first image is formed by holographic reconstruction of the first hologram of the first image; and The controller is configured to change or recalculate the first hologram in response to a change in viewing position determined by the viewer tracking system, wherein the first hologram is changed or recalculated to adjust the brightness of the first image.

2. The display device of claim 1, wherein the display device is further arranged to form a visible second image at all viewing positions within the viewing window; wherein the image generation unit is further arranged to output a second spatially modulated light corresponding to the second image; wherein the controller is further arranged to control the second spatially modulated light; the input port of the waveguide pupil expander is further arranged to receive the second spatially modulated light from the image generation unit; and the pair of parallel reflective surfaces is further arranged to guide the second spatially modulated light from the input port to the viewing window through a series of internal reflections from the pair of parallel reflective surfaces.

3. The display device as claimed in claim 2, wherein, The second spatially modulated light is a second color, and the first reflective surface portion of the pair of parallel reflective surfaces transmits light of the second color.

4. The display device as claimed in claim 2 or 3, wherein, The second image is formed by holographic reconstruction of a second hologram of the second image; and wherein the controller is arranged to change or recalculate the second hologram in response to a change in viewing position determined by the viewer tracking system.

5. The display device as claimed in claim 1, wherein, The first reflective surface of the pair of parallel reflective surfaces includes a partially transmissive coating.

6. The display device as claimed in claim 1, wherein, The value of each hologram pixel is restricted to one of a number of permissible light modulation levels according to a quantization scheme, and recalculating the first hologram involves changing the quantization scheme to alter the light proportion of the zero-order spot that contributes to holographic reconstruction.

7. The display device as claimed in claim 6, wherein, Changing the quantization scheme includes reducing the range of optical modulation in which the permissible optical modulation level is allocated; or Changing the quantization scheme involves altering the vector sum of the vectors representing the permissible optical modulation level in the complex plane.

8. The display device as claimed in claim 1, wherein, The first spatially modulated light received by the waveguide pupil expander from the image generation unit includes spatial modulation encoded with the first hologram.

9. The display device as claimed in claim 1, wherein, The first spatially modulated light received by the waveguide pupil expander is the light of the first image, wherein the first image is formed on the intermediate plane between the image generation unit and the viewing window.

10. The display device as claimed in claim 9, wherein, The image generation unit includes a first light source arranged to illuminate a first hologram so as to form a first image on the intermediate plane by holographic reconstruction, wherein the controller is arranged to apply a main drive signal correction to the drive signal of the first light source based on the brightness of the first image measured by a first photodetector.

11. A method for displaying a first image within a viewing window, the method comprising: Output a first spatially modulated light of a first color, wherein the first spatially modulated light is spatially modulated according to a first hologram of a first image; The first spatially modulated light is received at the input port of the waveguide pupil expander; Received light is directed to a viewing window between a pair of parallel reflective surfaces of a waveguide pupil expander, wherein a portion of the first reflective surface of the pair of parallel reflective surfaces transmits light of a first color, wherein the transmittance of the first reflective surface is non-achromatic, and wherein the reflectance / transmittance of the first reflective surface varies with distance along the waveguide. The received light is output to the viewing window through the first reflective surface of the waveguide pupil expander to form / display a first image; Determine the viewing position or change the viewing position within the viewer's viewing window, and The first hologram is modified / recalculated in response to a change in the determined viewing position to adjust the brightness of the first image.

12. The method of claim 11, further comprising the step of outputting second spatially modulated light of a second color, wherein, The second spatially modulated light is spatially modulated according to the second hologram of the second image; The method further includes the step of receiving second spatially modulated light at the input port of the waveguide pupil expander, wherein a first surface portion of the waveguide pupil expander transmits light of a second color. And further includes holographic reconstruction of receiving light through the first reflective surface of the waveguide pupil expander to the viewing window to form a second image.

13. The method of claim 11 or 12, further comprising calibrating the required brightness variation for the light of the first color output to the viewing window based on the viewing position to maintain the brightness of the light substantially constant.

Images