Image projection

Abstract

A light engine is arranged to provide spatially modulated light to a viewing system having an entrance pupil. The display system includes a display device arranged to display a hologram and spatially modulate light according to the hologram. The display system also includes a hologram engine arranged to receive contribution information identifying contributing and non-contributing regions of the display device based on the position of the entrance pupil. The contributing regions of the display device substantially propagate light that passes through the entrance pupil at the determined position. The non-contributing regions of the display device substantially propagate light that is blocked by the entrance pupil at the determined position. The contribution information further identifies: (i) at least one primary contributing region of the display device that propagates light that contributes to a primary image to the viewing system, and (ii) at least one secondary contributing region of the display device that propagates light that contributes to a secondary image to the viewing system. The hologram engine is further arranged to determine a hologram based on the at least one primary contributing region of the display device identified by the processing engine. The hologram engine is further arranged to output the hologram to the display device for display.

Description

Technical Field

[0001] The present disclosure relates to image projection. More specifically, the present disclosure relates to holographic projection and methods for determining diffraction structures such as holograms or kinoforms. Some embodiments relate to real-time hologram computation based on eye tracking information. Some embodiments relate to virtual image projection. Other embodiments relate to projection of real images. Embodiments relate to viewing projected images through waveguides. Some embodiments relate to light engines such as picture generation units. Some embodiments relate to heads-up displays. Background Art

[0002] Light scattered from an object contains both amplitude and phase information. This amplitude and phase information can be captured, for example, on a photographic plate, using well-known interference techniques to form a holographic recording or "hologram" consisting of interference fringes. The hologram can be reconstructed by illuminating it with appropriate light to form a two- or three-dimensional holographic reconstruction or replay image representing the original object.

[0003] Computer-generated holography can numerically simulate interference processes. Computer-generated holograms can be calculated using techniques based on mathematical transformations, such as Fresnel or Fourier transforms. These types of holograms are referred to as Fresnel / Fourier transform holograms or simply Fresnel / Fourier holograms. Fourier holograms can be considered as Fourier domain / planar representations of an object or frequency domain / planar representations of an object. For example, computer-generated holograms can also be calculated using coherent ray tracing or point cloud techniques.

[0004] The computer generated hologram may be encoded on a spatial light modulator arranged to modulate the amplitude and / or phase of the incident light. For example, light modulation may be achieved using electrically addressable liquid crystals, optically addressable liquid crystals, or micromirrors.

[0005] A spatial light modulator typically comprises a plurality of individually addressable pixels, which may also be referred to as cells or elements. The light modulation scheme may be binary, multi-level, or continuous. Alternatively, the device may be continuous (i.e., comprise no pixels), so that the light modulation may be continuous across the device. A spatial light modulator may be reflective, meaning that the light is modulated to reflect the output. A spatial light modulator may also be transmissive, meaning that the light is modulated to transmit the output.

[0006] The systems described herein can be used to provide holographic projectors, which have applications in, for example, heads-up displays (HUDs) and light detection and ranging (LIDAR). Summary of the Invention

[0007] Aspects of the present disclosure are defined in the accompanying independent claims.

[0008] The present disclosure relates to image projection. It relates to a method of image projection and an image projector including a display device. The present disclosure also relates to a projection system including an image projector and a viewing system. The present disclosure is equally applicable to monocular and binocular viewing systems. The viewing system may include one or both eyes of a viewer. The viewing system includes optical elements having an optical focal length (e.g., a lens of a human eye) and a viewing plane (e.g., a retina of a human eye). The projector may be referred to as a "light engine." The display device and the image formed (or perceived) using the display device are spatially separated from each other. The image is formed on or perceived by the viewer on the display plane. In some embodiments, the image is a virtual image, and the display plane may be referred to as a virtual image plane. The image is formed by illuminating a diffraction pattern (e.g., a hologram) displayed on the display device.

[0009] A display device includes pixels. The pixels of the display device diffract light. According to well-known optical principles, the maximum diffraction angle depends on the size of the pixel (and other factors, such as the wavelength of the light).

[0010] In an embodiment, the display device is a spatial light modulator, such as a liquid crystal on silicon ("LCOS") spatial light modulator (SLM). Light propagates from the LCOS toward a viewing entity / system (such as a camera or eye) within a range of diffraction angles (e.g., from zero to a maximum diffraction angle). In some embodiments, amplification techniques can be used to increase the range of available diffraction angles beyond the conventional maximum diffraction angle of the LCOS.

[0011] In embodiments, the image is a real image. In other embodiments, the image is a virtual image perceived by one (or both) human eyes. The projection system or light engine can therefore be configured so that the viewer looks directly at the display device. In such embodiments, the light encoded with the hologram travels directly to the eye, and no intermediate holographic reconstruction is formed in the free space between the display device and the viewer or on a screen or other light-receiving surface. In such embodiments, the pupil of the eye can be considered the entrance aperture of the viewing system, and the retina of the eye can be considered the viewing plane of the viewing system. It is sometimes also said that in this configuration, the lens of the eye performs the conversion of the hologram to the image.

[0012] According to well-known optical principles, the angular range of light propagating from a display device that can be viewed by an eye or other viewing entity / system varies with the distance between the display device and the viewing entity. For example, at a viewing distance of 1 meter, only a small range of angles from the LCOS can propagate through the pupil of the eye to form an image at the retina for a given eye position. The angular range of light propagating from the display device that can successfully propagate through the pupil of the eye to form an image at the retina for a given eye position determines the portion of the image that is "visible" to the viewer. In other words, not all portions of the image are visible from any one point on the viewing plane (e.g., from any one eye position within a viewing window such as an eye movement box).

[0013] In some embodiments, the image perceived by the viewer is a virtual image that appears upstream from the display device, meaning the viewer perceives the image as being farther away from them than the display device. Conceptually, one can consider multiple different virtual image points of the virtual image. The distance from a virtual image point to the viewer is referred to herein as the virtual image distance (for that virtual image point). Of course, different virtual points may have different virtual image distances. Individual light rays within the light bundle associated with each virtual point may take different corresponding optical paths through the display device to reach the viewer. However, only some portions of the display device, and therefore only some light rays from one or more virtual points in the virtual image, may be within the user's field of view. In other words, only some light rays from some virtual points on the virtual image will propagate through the display device to the user's eye and, therefore, will be visible to the viewer. Conceptually, the viewer can be thought of as viewing the virtual image through a "display-sized window," which may be very small, e.g., 1 cm in diameter, at a relatively large distance (e.g., 1 meter). Furthermore, the user will view the display-sized window through the pupil of their eye, which may also be very small. Consequently, the field of view is reduced at any given time, and the specific angular range that can be seen depends heavily on the position of the eye.

[0014] The present disclosure solves the technical problem of how to increase the field of view when the display device is (relatively) small and the projection distance is (relatively) large, that is, how to increase the angular range of light that propagates from the display device and can successfully propagate through the pupil of the eye to form an image. In some embodiments, the projection distance is at least one (such as at least two) orders of magnitude greater than the diameter or width of the aperture of the display device (i.e., the size of the pixel array). More specifically, the present disclosure solves the technical problem of how to do this using so-called direct-view holography, in which a hologram of the image is transmitted to the human eye, rather than the image itself. In other words, the light received by the viewer is modulated according to the hologram of the image.

[0015] Waveguides are used to expand the field of view, thereby increasing the maximum propagation distance of the full diffraction angle at which the display device can be used. The use of waveguides can also increase the user's eye box laterally, thereby allowing the eye to move somewhat while still allowing the user to see the image. The waveguide may therefore be referred to as a waveguide pupil expander. However, the inventors have discovered that for non-infinite virtual image distances (i.e., near-field virtual images), so-called "ghost images" appear due to the different possible light propagation paths through the waveguide. The ghost image is a low-intensity copy of the main image. The main highest intensity image may be referred to as the primary image. Each ghost image may be referred to as a secondary image. The presence of ghost images can significantly reduce the quality of the perceived virtual image. The ghost images may make the main image appear blurry.

[0016] This disclosure relates to various methods for addressing the problems caused by ghost images. Some of the solutions disclosed herein have been shown to successfully remove ghost images. Some of the solutions disclosed herein have been shown to modify / manipulate ghost images to enhance or intensify the primary / non-ghost image.

[0017] According to one aspect, a light engine is arranged to provide spatially modulated light to a viewing system having an entrance pupil. The display system includes a display device arranged to display a hologram and spatially modulate light according to the hologram. The display system also includes a hologram engine arranged to receive contribution information identifying contributing and non-contributing regions of the display device based on the position of the entrance pupil. The contributing regions of the display device substantially propagate light that passes through the entrance pupil at the determined position. The non-contributing regions of the display device substantially propagate light that is blocked by the entrance pupil at the determined position. The hologram engine is further arranged to determine a hologram based on at least one primary contributing region of the display device identified by a processing engine. The hologram engine is further arranged to output the hologram to the display device for display.

[0018] In at least some embodiments, the contribution information also identifies: (i) at least one primary contributing region of the display device that propagates light contributing to the primary image toward the viewing system, and (ii) at least one secondary contributing region of the display device that propagates light contributing to the secondary image toward the viewing system.

[0019] For the avoidance of doubt, the image formed or perceived is a holographic reconstruction of the target image. The holographic reconstruction is formed from a hologram based on the target image. In some embodiments, the hologram is determined (e.g., calculated) from the target image.

[0020] By identifying contributing and non-contributing areas of the display device, the light engine can determine which portion or portions of the display device, for a given position of the viewing system's entrance aperture, can be usefully encoded with a hologram, so as to contribute actively to the formation of the primary image. This can correspond to the position of the viewer's eyes at a given time, for example. Furthermore, the light engine can determine which portions of the display device are unable to propagate light through the entrance aperture and, therefore, are unworthy of being populated with hologram value. Furthermore, the light engine can distinguish between portions of the display device that contribute actively to the "primary" target image and portions that contribute to copies / replicas, or "ghost" versions, of the primary image. Consequently, holograms can be omitted in these so-called secondary contributing areas to eliminate ghosting.

[0021] Alternatively, in a significant further refinement, the hologram displayed in the additional contributing area can be determined based on a displaced or modified position of an image point (i.e., a point within the desired image to be holographically reconstructed). This modified position may be referred to as a "secondary image point," but this is shorthand for the secondary (i.e., altered) position of the (primary) image point. In short, the modeled / calculated position of the image point can be modified (e.g., translated in the image plane) so that light propagating from the modified position reaches the desired position on the viewing plane via the additional contributing area on the display device, effectively enhancing the primary image. Thus, in this alternative approach, the hologram for the additional contributing area is determined based on a different position from the image point used to identify the primary contributing area on the display device. The optical path length from the primary image point to the corresponding image formed on the viewing plane is typically different from the optical path length from the secondary image point to the corresponding image formed on the viewing plane. It can therefore be said that the hologram determination process associated with the additional contributing area includes translating or moving the image point used in the hologram determination process.

[0022] Thus, an intelligent and efficient light engine is provided which can be configured and operated to provide a clear, accurate image corresponding to a hologram that has been determined in a streamlined and computationally efficient manner.

[0023] According to one aspect, a method for determining a hologram for display on a display device is provided. The method includes determining a position of an entrance pupil of a viewing system arranged to view the hologram and identifying a contributing region and a non-contributing region of the display device, wherein the contributing region of the display device substantially propagates light that passes through the entrance pupil at the determined position of the viewing system, and the non-contributing region of the display device substantially propagates light that is blocked by the entrance pupil at the determined position of the viewing system. The method also includes determining a hologram based on at least one primary contributing region of the display device.

[0024] The method may further include identifying at least one primary contributing region of the display device that provides light contributing to the primary image and at least one secondary contributing region of the display device that provides light contributing to the secondary image.

[0025] According to one aspect, a diffractive structure is provided that is arranged to spatially modulate light that can be transformed by a viewing system into a target image, wherein the diffractive structure is configured to generate a plurality of discrete light patterns, each light pattern corresponding to a different portion of the target image, wherein the shape of each discrete light pattern substantially corresponds to the shape of an entrance aperture of the viewing system.

[0026] According to one aspect, a diffractive structure is provided that is arranged to spatially modulate light that can be transformed by a viewing system (including a lens) into an image, wherein the diffractive structure is arranged to guide the light into a plurality of discrete light channels, wherein each light channel has a cross-sectional shape that substantially corresponds to an entrance pupil of the viewing system, and each light channel substantially corresponds to a different portion of the image.

[0027] According to one aspect, a method is provided for determining a hologram for display on a display device and viewing the hologram displayed on the display device through a waveguide to form a virtual image perceptible from a viewing plane. The method comprises, for each virtual image point of the virtual image, determining the coordinates of the virtual image point [x virtual ,y virtual ,z virtual ], determining a viewing position on a viewing plane, and determining a number B of light reflections within the waveguide associated with a primary image formed by the waveguide. The method further includes ray tracing from the virtual image point to the viewing plane for "B" light reflections within the waveguide, and ray tracing from [x virtual ,y virtual ,z virtual ] to the viewing plane, determine the coordinates of the main ray at the display device [x LCOS (B),y LCOS (B)] The method further includes determining the display device in the [x LCOS (B),y LCOS (B)] within the active pixel defined by the region; and by virtual ,y virtual ,z virtual ] propagates to the active pixels to determine a sub-hologram comprising amplitude and / or phase hologram components of the active pixels.

[0028] Chief rays may include rays that are determined (eg, calculated or modeled) to be the main or "primary" image point rays that travel from the virtual image point via the display device to the virtual image point on the viewing plane.

[0029] The method may further comprise combining the sub-holograms respectively calculated for two or more corresponding virtual image points in order to form the hologram.

[0030] The method may further comprise determining the position of the primary image of the virtual image point on the viewing plane [x sensor ,y sensor ].

[0031] The method may further include, for each value of ΔB allowed by the waveguide, calculating the light bounced for B+ΔB from [x sensor ,y sensor ] Track back to the virtual image plane z virtual , and determine the virtual point coordinates [x virtual (ΔB), y virtual (ΔB),z virtual ], which will image the reflection for B+ΔB to [x sensor ,y sensor The method may further include the following steps: virtual (ΔB),y virtual (ΔB),z virtual ] to the viewing plane, determine the coordinates of the main ray at the display device [x LCOS (B+ΔB),y LCOS (B+ΔB)], and identify the display device in [x LCOS (B+ΔB),y LCOS The method may further comprise the step of providing an additional active pixel within a second region (ie, an additional region) defined by [X(B+ΔB)]. virtual (ΔB),y virtual (ΔB), z virtual ] propagates to additional active pixels to determine an additional sub-hologram, which includes amplitude and / or phase hologram components of the additional active pixels.

[0032] Aspects of the present disclosure also relate to a hologram or kinoform featuring the guidance or routing of holographic light. Specifically, a diffractive structure is disclosed herein that is arranged to spatially modulate light that can be transformed into an image by a viewing system, wherein the diffractive structure is configured to route the light into a plurality of holographic channels, each holographic channel corresponding to a different portion of the image.

[0033] The diffractive structure may be arranged such that the holographic channels propagate from the diffractive structure at different angles.

[0034] Each holographic channel may include spatially modulated light according to a hologram of a corresponding different portion of the image.

[0035] The diffractive structures can be arranged to spatially modulate the phase of light.

[0036] The diffractive structure may be arranged to route light through a waveguide.The waveguide may be arranged for pupil expansion.

[0037] The cross-sectional shape of the light pattern that may be formed by each holographic channel may substantially correspond to the shape of an entrance aperture of the viewing system.

[0038] The holographic channels may be spatially separated or at least partially spatially separated.

[0039] Also disclosed herein is a system comprising a diffractive structure, a waveguide arranged to receive spatially modulated light from the diffractive structure, and a viewing system arranged to receive the spatially modulated light via the waveguide.

[0040] The system may be arranged so that the light of each holographic channel follows a different optical path from the diffractive structure to the viewing system.

[0041] Different optical paths can include different numbers of reflections within the waveguide. Different optical paths can have different lengths. Different optical paths can pass through the viewing system's entrance aperture at different angles.

[0042] The waveguides can be arranged so that at any viewing position on the viewing plane all holographic channels are routed through the entrance aperture of the viewing system. For each allowed viewing position, the waveguides route each holographic channel to the viewing system through only one optical path.

[0043] At least two holographic channels of the plurality of holographic channels may partially overlap at an entrance aperture of the viewing system.

[0044] The diffractive structure can be a kinoform or a hologram.

[0045] The term "hologram" is used to refer to a recording that contains amplitude information or phase information, or some combination thereof, about an object. The term "holographic reconstruction" is used to refer to an optical reconstruction of an object formed by illuminating a hologram. The system disclosed herein is described as a "holographic projector" because the holographic reconstruction can be a real image and spatially separated from the hologram. The term "replay field" is used to refer to the 2D region within which the holographic reconstruction is formed and is perfectly focused. If the hologram is displayed on a spatial light modulator comprising pixels, the replay field will be repeated in the form of multiple diffraction orders, where each diffraction order is a copy of the zeroth-order replay field. The zeroth-order replay field generally corresponds to the preferred or primary replay field because it is the brightest replay field. Unless explicitly stated otherwise, the term "replay field" should be taken to refer to the zeroth-order replay field. The term "replay plane" is used to refer to the plane in space that contains all replay fields. The terms "image," "replay image," and "image area" refer to the area of the replay field that is illuminated by the light of the holographic reconstruction. In some embodiments, an "image" may include discrete spots of light, which may be referred to as "image spots," or simply as "image pixels" for convenience.

[0046] The terms "encoding," "writing," and "addressing" are used to describe the process of providing a plurality of control values, respectively determining the modulation level of each pixel, to a plurality of pixels of the SLM. It can be said that the pixels of the SLM are configured to "display" a light modulation profile in response to receiving the plurality of control values. Thus, the SLM can be said to "display" a hologram, and a hologram can be considered an array of light modulation values or levels.

[0047] It has been found that a holographic reconstruction of acceptable quality can be formed from a "hologram" that contains only phase information about the original target (i.e., the target image used for reconstruction). Such a holographic recording may be referred to as a phase-only hologram. The embodiments relate to phase-only holograms, but the disclosure is equally applicable to amplitude-only holograms. The disclosure is not limited to any particular method of hologram calculation. Some embodiments relate only to point cloud holograms, i.e., holograms created using point cloud methods, by way of example. However, the disclosure is equally applicable to Fourier or Fresnel type holograms, as well as holograms calculated according to other techniques such as coherent ray tracing.

[0048] The present disclosure is also applicable to forming a holographic reconstruction using amplitude and phase information related to the original target (i.e., the target image). In some embodiments, this is achieved by using complex modulation of a so-called fully complex hologram that contains amplitude and phase information related to the original target. Because the value (gray level) assigned to each pixel of the hologram has an amplitude and a phase component, such a hologram can be referred to as a fully complex hologram. The value (gray level) assigned to each pixel can be represented as a complex number with an amplitude and a phase component. In some embodiments, a fully complex computer-generated hologram is calculated.

[0049] Reference may be made to the phase value, phase component, phase information, or simply phase of a pixel of a computer-generated hologram or spatial light modulator as shorthand for "phase delay." That is, any phase value described is actually a number (e.g., in the range of 0 to 2π) representing the amount of phase delay provided by that pixel. For example, a pixel of a spatial light modulator described as having a phase value of π / 2 will delay the phase of received light by π / 2 radians. In some embodiments, each pixel of the spatial light modulator can operate at one of a plurality of possible modulation values (e.g., phase delay values). The term "grayscale" may be used to refer to a plurality of available modulation levels. For example, the term "grayscale" may be used for convenience to refer to a plurality of available phase levels in a phase modulator alone, even though the different phase levels do not provide different shades of gray. For convenience, the term "grayscale" may also be used to refer to a plurality of available complex modulation levels in a complex modulator.

[0050] Thus, a hologram comprises an array of gray levels, i.e., an array of light modulation values, such as an array of phase delay values or complex modulation values. A hologram is also considered a diffraction pattern, since it is a pattern that causes diffraction when displayed on a spatial light modulator and illuminated with light of a wavelength relative to (usually less than) the pixel pitch of the spatial light modulator. Reference is made herein to combining a hologram with other diffraction patterns, such as diffraction patterns acting as lenses or gratings. For example, a diffraction pattern acting as a grating can be combined with a hologram to shift the replay field on the replay plane, or a diffraction pattern acting as a lens can be combined with a hologram to focus the holographic reconstruction on the replay plane in the near field.

[0051] Although different embodiments and groups of embodiments may be disclosed separately in the detailed description below, any feature of any embodiment or group of embodiments may be combined with any other feature or combination of features of any embodiment or group of embodiments. That is, all possible combinations and permutations of the features disclosed in this disclosure are contemplated. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Certain embodiments are described, by way of example only, with reference to the following drawings:

[0053] Figure 1 is a schematic diagram showing a reflective SLM producing a holographic reconstruction on a screen;

[0054] Figure 2A A first iteration of an example Gerchberg-Saxton type algorithm is shown;

[0055] Figure 2B The second and subsequent iterations of an example Gerchberg-Saxton type algorithm are shown;

[0056] Figure 2C An alternative second and subsequent iteration of an example Gerchberg-Saxton type algorithm is shown;

[0057] Figure 3 is a schematic diagram of a reflective LCOS SLM;

[0058] Figure 4 The angular content of the virtual image effectively propagating from the display device to the aperture is shown;

[0059] Figure 5a shows a viewing system with a relatively small propagation distance;

[0060] FIG5 b shows a viewing system with a relatively large propagation distance;

[0061] FIG6 a shows a viewing system with a relatively large propagation distance, which includes a waveguide for forming a virtual image at infinity;

[0062] FIG6 b shows an enlarged view of the optical path of FIG6 a ;

[0063] Figure 7 shows how a ghost image can be formed using a finite virtual image and a waveguide pupil expander;

[0064] Figure 8 A virtual image comprising a main image and two ghost images is shown;

[0065] Figures 9A to 9C An example is shown in which the entire LCOS is used to form one primary image point and two corresponding ghost image points;

[0066] Figures 10A to 10C showing first, second, and third propagation paths through the waveguide, resulting in a second ghost point, a primary image point, and a first ghost point, respectively;

[0067] Figures 11A to 11C Three propagation paths and LCOS utilization associated with three different fields / image points are shown;

[0068] Figure 12A showing a viewing system comprising a virtual image point and an image of the virtual image point formed by the viewing system and the waveguide;

[0069] Figure 12B Shown with Figure 12A The main contributing area of the LCOS associated with the example;

[0070] Figure 13 A flow chart illustrating an improved method of deriving an improved data structure according to an embodiment;

[0071] Figure 14 A flow chart illustrating a further improved method of deriving an improved data structure according to an embodiment;

[0072] Figure 15A shows an image comprising a plurality of image regions (bottom) and a corresponding hologram comprising a plurality of hologram components (top);

[0073] Figure 15B A hologram according to the present disclosure is shown, characterized in that holographically encoded light is routed or directed into a plurality of discrete hologram channels; and

[0074] Figure 15C An optimized system arranged to route the light content of each hologram channel to the eye through a different optical path is shown.

[0075] The same reference numbers will be used throughout the drawings to refer to the same or like parts. DETAILED DESCRIPTION

[0076] The present invention is not limited to the embodiments described below, but extends to the full scope of the appended claims. That is, the present invention can be embodied in different forms and should not be construed as limited to the described embodiments, which are set forth for illustrative purposes.

[0077] Unless expressly stated otherwise, any numerical examples should be regarded as illustrative rather than limiting.

[0078] Unless otherwise stated, terms in the singular may include plural forms.

[0079] A structure described as being formed on / under or above / below another structure should be construed to include a case where the structures are in contact with each other and, further, a case where a third structure is provided therebetween.

[0080] When describing a temporal relationship, for example, when the temporal order of events is described as "after," "followed," "next," "before," etc., the present disclosure should be considered to include both consecutive and non-consecutive events unless otherwise specified. For example, unless terms such as "just," "immediately," or "directly" are used, the description should be considered to include non-consecutive situations.

[0081] Although the terms "first," "second," etc. may be used herein to describe various elements, these elements are not limited by these terms. These terms are only used to distinguish between the various elements. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element without departing from the scope of the appended claims.

[0082] Features of different embodiments may be coupled or combined with each other in part or in whole, and may interoperate differently with each other. Within the scope of the appended claims, some embodiments may be performed independently of each other, or may be performed together in an interdependent relationship.

[0083] Optical configuration

[0084] Figure 1 An embodiment is shown in which a computer-generated hologram is encoded on a single spatial light modulator. The computer-generated hologram is a Fourier transform of the target used for reconstruction. It should be understood that this is merely an example and that other methods for computer-generated holograms are contemplated in this disclosure. Thus, it can be said that the hologram is a Fourier domain or frequency domain or spectral domain representation of the target. In this embodiment, the spatial light modulator is a reflective liquid crystal on silicon "LCOS" device. The hologram is encoded on the spatial light modulator and a holographic reconstruction is formed at a replay field (e.g., a light receiving surface such as a screen or diffuser).

[0085] A light source 110 (e.g. a laser or laser diode) is arranged to illuminate the SLM 140 via a collimating lens 111. The collimating lens causes a substantially planar wavefront of light to be incident on the SLM. Figure 1 In embodiments, the direction of the wavefront is off-normal (e.g., two or three degrees from a plane that is truly normal to the transparent layer). However, in other embodiments, a substantially planar wavefront is provided at normal incidence, and a beam splitter is arranged to separate the input and output optical paths. Figure 1 In the embodiment shown, the arrangement is such that light from the light source reflects from the mirrored back surface of the SLM and interacts with the light modulating layer to form an exit wavefront 112. The exit wavefront 112 is applied to an optical device including a Fourier transform lens 120, the focus of which is located at a screen 125. More specifically, the Fourier transform lens 120 receives the modulated light beam from the SLM 140 and performs a frequency-to-space transform to produce a holographic reconstruction at the screen 125.

[0086] It's important to note that in this type of hologram, every pixel of the hologram contributes to the entire reconstruction. There is no one-to-one correlation between a specific point on the replay field (or image pixel) and a specific light modulation element (or hologram pixel). In other words, the modulated light leaving the light modulation layer is distributed across the entire replay field.

[0087] In these embodiments, the position of the holographic reconstruction in space is determined by the power (focus) of the Fourier transform lens. Figure 1In the illustrated embodiment, the Fourier transform lens is a physical lens. That is, the Fourier transform lens is an optical Fourier transform lens and optically performs a Fourier transform. Any lens can function as a Fourier transform lens, but the accuracy of the Fourier transform it performs will be limited by the lens's performance. Those skilled in the art understand how to use lenses to perform an optical Fourier transform.

[0088] Hologram computing

[0089] In some embodiments, the computer-generated hologram is a Fourier transform hologram, or simply a Fourier hologram or a Fourier-based hologram, in which the image is reconstructed in the far field by utilizing the Fourier transform properties of a positive lens. The Fourier hologram is calculated by Fourier transforming the desired light field in the replay plane back to the lens plane. The computer-generated Fourier hologram can be calculated using the Fourier transform.

[0090] Algorithms such as the Gerchberg-Saxton algorithm can be used to calculate Fourier transform holograms. Furthermore, the Gerchberg-Saxton algorithm can be used to calculate a hologram in the Fourier domain (i.e., a Fourier transform hologram) from amplitude-only information in the spatial domain (such as a photograph). Phase information about the object is effectively "retrieved" from amplitude-only information in the spatial domain. In some embodiments, computer-generated holograms are calculated from amplitude-only information using the Gerchberg-Saxton algorithm or a variation thereof.

[0091] The Gerchberg-Saxton algorithm considers that when the intensity cross section I of the beam in planes A and B is known, A (x,y) and I B (x,y) and I A (x,y) and I B (x,y) is related by a single Fourier transform. For a given intensity cross section, the phase distribution approximation Ψ in planes A and B is obtained separately A (x, y) and Ψ B (x,y). The Gerchberg-Saxton algorithm finds the solution to this problem by following an iterative process. More specifically, the Gerchberg-Saxton algorithm iteratively applies spatial and spectral constraints while repeatedly transferring representations of I between the spatial domain and the Fourier (spectral or frequency) domain. A (x,y) and I BA data set (amplitude and phase) of (x, y) is obtained. A corresponding computer-generated hologram in the spectral domain is obtained by at least one iteration of the algorithm. The algorithm is convergent and is arranged to produce a hologram representing the input image. The hologram can be an amplitude-only hologram, a phase-only hologram, or a full complex hologram.

[0092] In some embodiments, a phase-only hologram is calculated using an algorithm based on the Gerchberg-Saxton algorithm, such as the algorithm described in UK Patent 2498170 or 2501112, the entire contents of which are incorporated herein by reference. However, the embodiments disclosed herein describe calculating a phase-only hologram by way of example only. In these embodiments, the Gerchberg-Saxton algorithm retrieves the phase information Ψ[u,v] of the Fourier transform of a data set, which produces known amplitude information T[x,y], where the amplitude information T[x,y] represents a target image (e.g., a photograph). Since amplitude and phase are inherently combined in the Fourier transform, the transformed amplitude and phase contain useful information about the accuracy of the calculated data set. Therefore, the algorithm can be used iteratively with feedback of the amplitude and phase information. However, in these embodiments, only the phase information Ψ[u,v] is used as a hologram to form a holographic representation of the target image at the image plane. The hologram is a data set (e.g., a 2D array) of phase values.

[0093] In other embodiments, an algorithm based on the Gerchberg-Saxton algorithm is used to calculate a full complex hologram. A full complex hologram is a hologram having an amplitude component and a phase component. A hologram is a data set (e.g., a 2D array) comprising an array of complex data values, where each complex data value comprises an amplitude component and a phase component.

[0094] In some embodiments, the algorithm processes complex data and the Fourier transform is a complex Fourier transform. The complex data can be viewed as comprising (i) a real component and an imaginary component, or (ii) an amplitude component and a phase component. In some embodiments, the two components of the complex data are processed differently at various stages of the algorithm.

[0095] Figure 2AA first iteration of an algorithm for calculating a phase-only hologram according to some embodiments is shown. The input to the algorithm is an input image 210 comprising a 2D array of pixels or data values, wherein each pixel or data value is an amplitude or magnitude value. That is, each pixel or data value of the input image 210 does not have a phase component. Therefore, the input image 210 can be regarded as an amplitude-only or amplitude-only or intensity-only distribution. An example of such an input image 210 is a frame of a photograph or a video comprising a time sequence of frames. The first iteration of the algorithm begins with a data formation step 202A, which comprises assigning a random phase value to each pixel of the input image using a random phase distribution (or random phase seed) 230 to form a starting complex data set, wherein each data element of the data set comprises an amplitude and a phase. It can be said that the starting complex data set represents the input image in the spatial domain.

[0096] First processing block 250 receives an initial complex data set and performs a complex Fourier transform to form a Fourier-transformed complex data set. Second processing block 253 receives the Fourier-transformed complex data set and outputs hologram 280A. In some embodiments, hologram 280A is a phase-only hologram. In these embodiments, second processing block 253 quantizes each phase value and sets each amplitude value to 1 to form hologram 280A. Each phase value is quantized according to a phase level that can be represented at a pixel of a spatial light modulator that will be used to "display" a phase-only hologram. For example, if each pixel of the spatial light modulator provides 256 different phase levels, each phase value of the hologram is quantized to one of the 256 possible phase levels. Hologram 280A is a phase-only Fourier hologram representing the input image. In other embodiments, hologram 280A is a full complex hologram, comprising an array of complex data values (each including an amplitude component and a phase component) derived from the received Fourier-transformed complex data set. In some embodiments, second processing block 253 constrains each complex data value to one of a plurality of permissible complex modulation levels to form hologram 280A. This constraining step may include setting each complex data value to the closest permissible complex modulation level in the complex plane. Hologram 280A can be said to represent the input image in the spectral, Fourier, or frequency domain. In some embodiments, the algorithm stops at this point.

[0097] However, in other embodiments, the algorithm continues as Figure 2A In other words, following Figure 2A The steps indicated by dashed arrows are optional (ie, not essential for all embodiments).

[0098] The third processing block 256 receives the modified complex data set from the second processing block 253 and performs an inverse Fourier transform to form an inverse Fourier transformed complex data set. The inverse Fourier transformed complex data set can be said to represent the input image in the spatial domain.

[0099] Fourth processing block 259 receives the inverse Fourier transformed complex data set and extracts the distribution of amplitude values 211A and the distribution of phase values 213A. Optionally, fourth processing block 259 evaluates the distribution of amplitude values 211A. Specifically, fourth processing block 259 may compare the distribution of amplitude values 211A of the inverse Fourier transformed complex data set with the input image 510, which itself is a distribution of amplitude values. If the difference between the distribution of amplitude values 211A and the input image 210 is sufficiently small, fourth processing block 259 may determine that hologram 280A is acceptable. In other words, if the difference between the distribution of amplitude values 211A and the input image 210 is sufficiently small, fourth processing block 259 may determine that hologram 280A is a sufficiently accurate representation of the input image 210. In some embodiments, the distribution of phase values 213A of the inverse Fourier transformed complex data set is ignored for comparison purposes. It will be appreciated that any number of different methods may be employed to compare the distribution of amplitude values 211A to the input image 210, and the present disclosure is not limited to any particular method. In some embodiments, a mean square error is calculated, and if the mean square error is less than a threshold, the hologram 280A is deemed acceptable. If the fourth processing block 259 determines that the hologram 280A is unacceptable, further iterations of the algorithm may be performed. However, this comparison step is not required, and in other embodiments, the number of iterations of the algorithm performed is predetermined, preset, or user-defined.

[0100] Figure 2B 2 represents the second iteration of the algorithm, as well as any further iterations of the algorithm. The distribution 213A of phase values from the previous iteration is fed back through the processing blocks of the algorithm. The distribution 211A of amplitude values is rejected in favor of the distribution of amplitude values of the input image 210. In the first iteration, the data forming step 202A forms a first complex data set by combining the distribution of amplitude values of the input image 210 with the random phase distribution 230. However, in the second and subsequent iterations, the data forming step 202B includes forming a complex data set by combining (i) the distribution 213A of phase values from the previous iteration of the algorithm with (ii) the distribution of amplitude values of the input image 210.

[0101] Then, with reference to Figure 2A The same method as described is handled by Figure 2BThe data forming step 202B forms the complex data set formed to form a second iterative hologram 280B. Therefore, the description of the process is not repeated here. When the second iterative hologram 280B has been calculated, the algorithm can stop. However, any number of further iterations of the algorithm can be performed. It will be understood that the third processing block 256 is only required when the fourth processing block 259 is required or further iterations are required. The output hologram 280B generally gets better with each iteration. However, in practice, a point is usually reached at which no measurable improvement can be observed or the positive benefits of performing further iterations are offset by the negative effects of the additional processing time. Therefore, the algorithm is described as iterative and convergent.

[0102] Figure 2C 213A of the phase values of the previous iteration is fed back through the processing blocks of the algorithm. The distribution of amplitude values 211A is rejected in favor of the alternative distribution of amplitude values. In this alternative embodiment, the alternative distribution of amplitude values is derived from the distribution of amplitude values 211 of the previous iteration. Specifically, the processing block 258 subtracts the distribution of amplitude values of the input image 210 from the distribution of amplitude values 211 of the previous iteration, scales the difference by a gain factor α, and subtracts the scaled difference from the input image 210. This is mathematically expressed by the following equation, where the subscript text and number represent the number of iterations:

[0103] R n+1 [x,y]=F'{exp(iψ n [u,v])}

[0104] ψ n [u,v]=∠F{η·exp(i∠R n [x,y])}

[0105] η=T[x,y]-α(|R n [x,y]|-T[x,y])

[0106] in:

[0107] F' is the inverse Fourier transform;

[0108] F is the forward Fourier transform;

[0109] R[x,y] is the complex data set output by the third processing block 256;

[0110] T[x,y] is the input or target image;

[0111] ∠ is the phase component;

[0112] Ψ is the phase-only hologram 280B;

[0113] η is the new distribution of amplitude values 211B; and

[0114] α is the gain factor.

[0115] The gain factor α may be fixed or variable. In some embodiments, the gain factor α is determined based on the size and rate of the input target image data. In some embodiments, the gain factor α depends on the number of iterations. In some embodiments, the gain factor α is only a function of the number of iterations.

[0116] In all other respects, Figure 2C Examples and Figure 2A and Figure 2B It can be said that only the phase hologram Ψ(u,v) comprises the phase distribution in the frequency or Fourier domain.

[0117] In some embodiments, a spatial light modulator is used to perform a Fourier transform. Specifically, the hologram data is combined with a second data that provides optical power. That is, the data written to the spatial light modulation includes the hologram data representing the target and the lens data representing the lens. When displayed on the spatial light modulator and illuminated with light, the lens data simulates a physical lens, that is, it focuses light in the same way as a corresponding physical optical element. Therefore, the lens data provides optical power or focusing power. In these embodiments, the lens data can be omitted. Figure 1A physical Fourier transform lens 120 is shown. It is known how to calculate data representing a lens. This data representing a lens can be referred to as a software lens. For example, a phase-only lens can be formed by calculating the phase delay caused by each point of the lens due to its refractive index and spatially varying optical path length. For example, the optical path length at the center of a convex lens is greater than the optical path length at the edge of the lens. An amplitude-only lens can be formed by a Fresnel zone plate. In the field of computer-generated holography, it is also known how to combine data representing a lens with a hologram, thereby performing a Fourier transform of the hologram without requiring a physical Fourier lens. In some embodiments, the lensing data is combined with the hologram by a simple addition, such as simple vector addition. In some embodiments, a physical lens is used in conjunction with a software lens to perform the Fourier transform. Alternatively, in other embodiments, the Fourier transform lens is omitted entirely, allowing holographic reconstruction to occur in the far field. In further embodiments, the hologram can be combined with grating data (i.e., data arranged to perform a grating function, such as image steering) in the same manner. Again, it is known in the art how to calculate such data. For example, a phase-only grating can be formed by modeling the phase delay caused by each point on the surface of the blazed grating. An amplitude-only grating can simply be superimposed with an amplitude-only hologram to provide angular steering for the holographic reconstruction. The second data providing lensing and / or steering can be referred to as a light processing function or light processing pattern to distinguish it from the hologram data, which can be referred to as an image forming function or image forming pattern.

[0118] In some embodiments, the Fourier transform is performed jointly by a physical Fourier transform lens and a software lens. That is, some of the optical power that contributes to the Fourier transform is provided by the software lens, while the remaining optical power that contributes to the Fourier transform is provided by one or more physical optical devices.

[0119] In some embodiments, a real-time engine is provided that receives image data and uses an algorithm to calculate holograms in real time. In some embodiments, the image data is a video comprising a sequence of image frames. In other embodiments, the holograms are pre-calculated, stored in computer memory, and retrieved as needed for display on the SLM. That is, in some embodiments, a repository of pre-determined holograms is provided.

[0120] The embodiments relate to Fourier holography and Gerchberg-Saxton type algorithms by way of example only. The present disclosure is equally applicable to Fresnel holography and Fresnel holograms that can be calculated by similar methods. The present disclosure is also applicable to holograms calculated by other techniques, such as those based on point cloud methods. As will be seen, the subsequent figures herein are described as including point cloud methods for hologram calculation. However, other hologram calculation methods may also be used, including those described above with respect to Figures 2A to 2C Describe the Fourier method.

[0121] Optical Modulation

[0122] A spatial light modulator can be used to display diffraction patterns, including computer-generated holograms. If the hologram is a phase-only hologram, a spatial light modulator that modulates the phase is required. If the hologram is a fully complex hologram, a spatial light modulator that modulates both phase and amplitude can be used, or a first spatial light modulator that modulates the phase and a second spatial light modulator that modulates the amplitude can be used.

[0123] In some embodiments, the light modulation elements (i.e., pixels) of the spatial light modulator are cells comprising liquid crystals. That is, in some embodiments, the spatial light modulator is a liquid crystal device in which the optically active component is liquid crystal. Each liquid crystal cell is configured to selectively provide multiple light modulation levels. That is, each liquid crystal cell is configured to operate at a light modulation level selected from a plurality of possible light modulation levels at any time. Each liquid crystal cell can be dynamically reconfigured to a light modulation level that is different from the plurality of light modulation levels. In some embodiments, the spatial light modulator is a reflective liquid crystal on silicon (LCOS) spatial light modulator, but the present disclosure is not limited to this type of spatial light modulator.

[0124] LCOS devices provide a dense array of light modulating elements, or pixels, within a small aperture (e.g., a few centimeters wide). Pixels are typically around 10 microns or smaller, which results in a diffraction angle of a few degrees, meaning the optical system can be compact. It is much easier to fully illuminate the small aperture of an LCOS SLM than the large apertures of other liquid crystal devices. LCOS devices are typically reflective, which means that the circuitry that drives the LCOS SLM pixels can be buried beneath the reflective surface. This results in a higher aperture ratio. In other words, the pixels are tightly packed, meaning there is almost no dead space between pixels. This is advantageous because it reduces optical noise in the playback field. LCOS SLMs use a silicon backplane, which has the advantage that the pixels are optically flat. This is particularly important for phase modulation devices.

[0125] The following is just an example, Figure 3A suitable LCOS SLM is described below. An LCOS device is formed using a single-crystal silicon substrate 302. It has a 2D array of square planar aluminum electrodes 301, separated by gaps 301a, arranged on the upper surface of the substrate. Each electrode 301 can be addressed by circuitry 302a embedded in the substrate 302. Each electrode forms its own plane mirror. An orientation layer 303 is disposed on the electrode array, and a liquid crystal layer 304 is disposed on the orientation layer 303. A second orientation layer 305 is disposed on a planar transparent layer 306, for example, made of glass. A single transparent electrode 307, for example, made of ITO, is disposed between the transparent layer 306 and the second orientation layer 305.

[0126] Each square electrode 301, together with the footprint of the transparent electrode 307 and the intervening liquid crystal material, defines a controllable phase-modulating element 308, often referred to as a pixel. The effective pixel area, or fill factor, is the percentage of the total pixel that is optically active, taking into account the spaces between pixels 301a. By controlling the voltage applied to each electrode 301 relative to the transparent electrode 307, the properties of the liquid crystal material of each phase-modulating element can be varied, thereby providing variable retardation to light incident thereon. The effect is to provide only phase modulation of the wavefront, i.e., no amplitude effects occur.

[0127] The described LCOS SLM outputs spatially modulated light in a reflective manner. A reflective LCOS SLM has the advantage that the signal lines, gate lines, and transistors are located below the mirror surface, which results in a high fill factor (typically greater than 90%) and high resolution. Another advantage of using a reflective LCOS spatial light modulator is that the thickness of the liquid crystal layer can be half that required when using a transmissive device. This greatly increases the switching speed of the liquid crystal (a key advantage for projecting moving video images). However, the teachings of the present disclosure can also be implemented using a transmissive LCOS SLM.

[0128] Image projection using small display devices and long viewing distances

[0129] The present disclosure relates to image projection in which the separation between the display device and the viewer is much greater than the size of the display device. The viewing distance (i.e., the distance between the viewer and the display device) can be at least one order of magnitude greater than the size of the display device. The viewing distance can be at least two orders of magnitude greater than the size of the display device. For example, the pixel area of the display device can be 10 mm x 10 mm, and the viewing distance can be 1 meter. The image projected by the system is formed on a display plane that is spatially separated from the display device.

[0130] According to the present disclosure, an image is formed by holographic projection. A hologram is displayed on a display device. The hologram is illuminated by a light source (not shown), and the image is perceived on a display plane that is spatially separated from the hologram. The image can be real or virtual. For the purposes of the following explanation, it is helpful to consider a virtual image formed upstream of the display device. That is, it appears behind the display device. However, it is not necessary for the image to be a virtual image, and the present disclosure is equally applicable to real images formed between the display device and the viewing system.

[0131] The display device includes pixels that display the hologram. The pixel structure of the display device is diffractive. Therefore, the size of the holographic image is determined by the diffraction law. Figure 4 Interpret the results showing the diffraction properties of the device.

[0132] Figure 4 A pixelated display device 402 is shown, arranged to display a hologram forming a virtual image 401 upstream of the display device 402. The diffraction angle θ of the display device determines the size of the virtual image 401. The virtual image 401, the display device 402 and the viewing system 405 are arranged on an optical axis Ax.

[0133] Viewing system 405 has an entrance aperture 404 and a viewing plane 406. Viewing system 406 can be a human eye. Thus, entrance aperture 404 can be the pupil of the eye, and viewing plane 406 can be the retina of the eye.

[0134] The light traveling between display device 402 and viewing system 405 is modulated with a hologram of the image (not the image itself). Each illustrated bundle of light rays relates to a different portion of virtual image 401. More specifically, the light in each bundle of light rays is encoded by the hologram with information about a portion of the virtual image. Figure 4 Five example light beams are shown, each characterized by a respective angle relative to the optical axis Ax, and each representing a respective portion of the virtual image. In this example, one of the light beams passes through the pupil 404, while the other four light beams are blocked by the pupil 404. Again, the five different light beams correspond to five different portions of the virtual image 401. The complete image content of the virtual image is effectively divided angularly. The light beam propagating along the optical axis Ax carries the central portion of the image information, i.e., information relating to the center of the image. The other light beams carry other portions of the image information. The two light beams shown at the extreme ends of the light cone carry edge portions of the image information. A consequence of this angular division of the image information is that, at a given viewing position, not all of the image content can pass through the entrance aperture 404 of the viewing system. In other words, not all of the image content is received by the eye. Figure 4In the example of FIG4 , only one of the five beams shown at any viewing position passes through pupil 404. The reader will understand that five beams are shown by way of example only, and the process described is not limited to splitting the image information of a virtual image into only five beams.

[0135] In this example, the center portion of the image information is received by the eye. The edge of the image information is obscured by the eye's pupil. The reader will understand that if the viewer moves up or down, the eye may receive different beams of light, and, for example, the center portion of the image information may be obscured. Consequently, the viewer sees only a portion of the complete image. The remaining image information is obscured by the entrance pupil. The viewer's field of view is severely limited because they are effectively viewing the image through the small aperture of the display device itself.

[0136] In summary, light propagates within a range of diffraction angles from the display device. At a viewing distance of 1 meter, for a given eye position, only a small range of angles from the display device can propagate through the pupil of the eye to form an image on the retina. The portion of the virtual image that is visible is only that which falls on the Figure 4 The small angular range shown shows the portion of the image that passes through the entrance aperture. Therefore, the field of view is very small, and the exact angular range depends strongly on eye position.

[0137] refer to Figure 4 The problems of small field of view and sensitivity to eye position explained are a consequence of the large viewing distance and small aperture of the display device. The importance of viewing distance will be further explained with reference to Figures 5 to 7.

[0138] Figure 5A A display device 502 is shown arranged to display a hologram and to propagate light modulated according to the hologram to a viewing system comprising an entrance aperture 504 and a viewing plane 506. The virtual image 501 is at infinity, so light rays traced between the virtual image and the display device are collimated. Figure 5A The lower part of the diagram shows an enlarged view of the viewing system. The diagram is schematic and therefore does not show the physiological details of the eye. In practice, there is of course a light source ( Figure 5A ) is arranged to illuminate the display device 502.

[0139] Figure 5A Only those light rays that can propagate through the aperture 504 are shown; any other light rays that cannot pass through the aperture 504 are omitted. However, it will be appreciated that, in practice, these other light rays propagate from the display device 502. Figure 5AIn the optimal position, the field of view is equal to the diffraction angle of the display device. Interestingly, different image points on the retina are formed by light propagating from different areas on the display device 502, such as the area closest to the retina. Figure 5A The top image point is formed only by light propagating from the lower part of the display device. Light propagating from other areas of the display device does not contribute to this image point.

[0140] Figure 5B This is shown as the viewing distance increases.

[0141] In more detail, Figure 5B A display device 502' is shown, which is arranged to display a hologram and transmit light modulated according to the hologram to a viewing system comprising an entrance aperture 504' and a viewing plane 506'. The virtual image 501' is at infinity, so the light rays traced between the virtual image and the display device are collimated. The lower part of FIG5B shows an enlarged view of the viewing system. This figure is schematic and therefore does not show the physiological details of the eye. In practice, of course, there is a light source ( Figure 5B ) is arranged to illuminate the display device 502'.

[0142] Figure 5B Only those rays that can propagate through the aperture 504' are shown. Figure 5B At larger viewing distances, some light beams are blocked by entrance aperture 504'. Specifically, light beams associated with the edge portions of the virtual image are blocked by entrance pupil 504'. Consequently, the entire virtual image is invisible, and the portion of the virtual image that is visible depends heavily on eye position. This demonstrates that large distances between the display device and the viewing system are problematic due to the (relatively) small size of the display device.

[0143] Figure 6A An improved system is shown comprising a display device 602, with light encoded with a hologram displayed on the display device 602 being transmitted to a viewing system comprising an entrance aperture 604 and a viewing plane 606. In practice, a light source (not shown) is of course arranged to illuminate the display device 602. The improved system also comprises a waveguide 608 located between the display device 602 and the entrance aperture 604. Figure 6A The lower part of FIG shows a magnified view of the entrance pupil 604 and the viewing plane 606. The figure is schematic and therefore does not show the physiological details of the eye.

[0144] The viewing distance of Figure 6 Figure 5B However, in Figure 5B The blocked light beams are effectively restored by waveguide 608, so that the viewing system receives the complete image information despite the longer viewing distance.

[0145] The presence of waveguide 608 enables all angular content from display device 602 to be received by the eye, even at this relatively large projection distance. This is because waveguide 608 acts as a pupil expander in a well-known manner and will therefore only be briefly described here.

[0146] Briefly, waveguide 608 comprises a substantially elongated structure. In this example, it comprises an optical plate of refractive material, but other types of waveguides are well known and can be used. Waveguide 608 is positioned to intersect the light cone projected from display device 602, for example at an oblique angle. The size, orientation, and position of waveguide 608 are configured to ensure that light from each of the five light beams within the light cone enters waveguide 608. Light from the light cone enters waveguide 608 via its first planar surface 610 (located closest to display device 602) and is guided at least partially along the length of waveguide 608 before being emitted via its second planar surface 612, which is substantially opposite to first surface 610 (located closest to the eye). As will be well understood, second planar surface 612 is partially reflective and partially transmissive. In other words, as each light ray travels within waveguide 608 from first planar surface 610 of waveguide 608 to second planar surface 612, some of the light will be transmitted out of waveguide 608, and some will be reflected by second planar surface 612 back toward first planar surface 610. First planar surface 610 is reflective, such that all light directed toward it from within waveguide 608 will be reflected back toward second planar surface 612. Thus, some light may simply refract between the two planar surfaces 610, 612 of waveguide 608 before being transmitted, while other light may be reflected, and thus may experience one or more reflections (or "bounces") between the planar surfaces 610, 612 of waveguide 608 before being transmitted. Thus, the net effect of waveguide 608 is that the transmission of light is effectively extended to multiple locations on second planar surface 612 of waveguide 608. Consequently, all angular content output by display device 602 can appear at a greater number of locations on the display plane (and at a greater number of locations on the aperture plane) than would be the case without waveguide 608. This means that light from every beam can enter entrance aperture 604 and contribute to the image formed by viewing plane 606, despite the relatively large projection distance. In other words, the eye can receive content from all angles of display device 602. Therefore, the full diffraction angle of display device 602 is utilized, and the viewing window for the user is maximized. This, in turn, means that all light rays contribute to the perceived virtual image 601.

[0147] Figure 6B Shows the Figure 6A601 are labeled R1 through R5, from top to bottom, for each of the five light beams contributing to the five corresponding image points within the virtual image 601 formed in FIG. As can be seen in the figure, the light from each of R1 and R2 is simply refracted by waveguide 608 and then transmitted. On the other hand, the light from R4 experiences a single bounce before being transmitted. The light from R3 includes some light from the corresponding first portion of the display device 602 that is simply refracted by waveguide 608 before being transmitted, and some light from a different corresponding second portion of the display device 602 that experiences a single bounce before being transmitted. Similarly, the light from R5 includes some light from the corresponding first portion of the display device 602 that experiences a single bounce before being transmitted, and some light from a different corresponding second portion of the display device 602 that experiences two bounces before being transmitted. For each of R3 and R5, two different portions of the LCOS propagate the light corresponding to that portion of the virtual image.

[0148] The present inventors have recognized that, at least in some applications, the virtual image distance (i.e., the distance from the viewer to the virtual image) is preferably finite, as opposed to forming a virtual image at infinity. In certain applications, there will be a preferred virtual image distance at which it is desirable or necessary for virtual image content to appear. This may be the case, for example, in a heads-up display, such as in an automotive setting, where, for example, virtual image content is to be superimposed on real content viewed by a viewer through the vehicle's windshield. For example, a desired virtual image distance may include virtual image content formed a few meters (e.g., 3 or 5 meters) in front of the viewer's vehicle or windshield.

[0149] Figure 7 6 a . The top portion of FIG. 6 shows a system comprising a display device 702 that propagates light 703 that has been encoded with (i.e., modulated according to) a hologram displayed on the display device 702 toward an eye comprising an entrance aperture 704 and a viewing plane 706. A light source (not shown) is arranged to illuminate the display device 702. The system also comprises a waveguide 708 located between the display device 702 and the entrance aperture 704 to act as a pupil expander as described in detail above with respect to FIG. 6 a . Figure 7 The middle portion of FIG. 700 shows a magnified view of the entrance aperture 704 and the viewing plane 706 . Figure 7 The lowest portion of FIG shows a further magnified view of viewing plane 706. This figure is schematic and therefore does not illustrate the physiological details of the eye. In this arrangement, the eye perceives virtual image 701 as being located at a finite distance upstream of display device 702. The light rays between virtual image 701 and the display device diverge because the virtual image distance is finite.

[0150] As above based on Figure 6A As stated, Figure 7 The presence of the medium waveguide 708 effectively enables the full diffraction angle of the display device 702 to be accessed at a relatively large projection distance, so that the user at the viewing position shown can see the full image content.

[0151] However, another technical problem is introduced. When a virtual image is formed with a limited virtual image distance, for some light bundles, different optical paths from different parts of the display device 702 may cause these light bundles to each form multiple image points on the retina 706. This is different from Figure 7 The image is associated with the bundles of rays labeled R3' and R5' in the image. The additional image points formed by the primary image point at a given point within the virtual image can be referred to as "ghost image points," which together form a "ghost image" or simply "ghost." Those skilled in the art of image formation will understand that the formation of ghosts can lead to blurring of the virtual image and an overall decrease in perceived quality from the viewer's perspective. This is particularly true when the "ghost" image partially overlaps with the "primary" image.

[0152] Figure 8 An example of a virtual image of the numbers "5" and "9" is shown, which is created using a method similar to Figure 7 The viewing system shown creates a ghost image in addition to the main image. The main image can be seen as the brightest center image of each digit, with ghost images to the left and right. Figure 8 In the example of FIG, “9” is formed at a viewing distance greater than that for “5”, so the blur is more obvious for it. However, this is only an illustrative example and should not be considered as limiting the present disclosure.

[0153] The inventors have solved the problem of ghost images. The inventors have recognized that it is desirable to provide a viewing system in which a virtual image can be formed at a limited virtual image distance, the virtual image including all angular image content output by a display device, and in which the formation of ghost images is reduced or eliminated. Furthermore, the inventors have found that as the size of the viewing aperture in conventional viewing systems increases, the risk of forming ghost image points increases because the aperture can admit additional light, which may form additional image points on the display plane. Therefore, it is desirable to provide an improved viewing system that can accommodate apertures of different sizes while still reducing or eliminating the formation of ghost images. The solution provided by the inventors, described in detail below, is applicable to a range of different sizes and arrangements of apertures, waveguides, and display devices, and can be applied to different propagation distances for which one or more ghost images may typically be formed.

[0154] In summary, the inventors have recognized that it is possible to provide a light engine for generating a hologram that effectively identifies one or more regions of a display device that would contribute to one or more ghost images in a conventional arrangement, wherein the hologram is derived so as to control the contribution from these one or more regions of the display device and thereby avoid or reduce the formation of ghost image points when the hologram is displayed on the display device and illuminated. The inventors have also recognized that it is possible to provide a hologram engine for providing such a hologram even when the projection distance in the viewing system is relatively large and the display device and / or viewing aperture is relatively small, and to provide an improved viewing system for displaying and illuminating the improved hologram to form an improved image.

[0155] The inventors have recognized that by having a waveguide with a relatively small viewing aperture (such as in the present invention) Figure 6A and 7 ) and optionally also including the angular limitations imposed by the viewing system of the relatively small display device, so that different possible propagation paths within the waveguide can be considered separately. Moreover, they have recognized that as a result of this consideration it is possible to identify each of: areas of the display device that are light sources contributing to a desired "primary" image; areas of the display device that are light sources contributing to an undesired "ghost" image; and areas of the display device that are light sources that are blocked by the aperture and therefore do not contribute to either the primary or ghost image. The inventors have also recognized that it is possible to restrict the hologram calculation only to areas of the display device that contribute to the primary image. They have also recognized that in a further improvement, in some embodiments, a modified hologram can be provided, which can in fact result in one or more ghost images being translated so as to be superimposed on the primary image.

[0156] The insights made by the inventors and the improved systems and methods embodying these insights may be further understood with reference to the accompanying drawings, which are described in detail below.

[0157] Figures 9A to 9C A display device 902 is shown, which in this example is an LCOS spatial light modulator. "LCOS" referred to below is shorthand for "display device." The teachings of the present disclosure are not limited to LCOS display devices. Figure 9B The trajectories of light rays are depicted relative to a virtual image point from the LCOS 902 through the waveguide 908 toward the viewing entity / system 905, which in this example comprises the eye of a viewer. Figure 9CAlso included is a magnified view of the eye 905, showing light rays at the pupil 904 (i.e., the entrance aperture) and the retina 906 (i.e., the sensor or viewing plane). In this example, the entire LCOS area contributes to forming the image point on the retina 906. In other words, the entire LCOS 902 is "visible" to the viewer. This contribution of the entire LCOS 902 to the image is illustrated by the entire LCOS being shaded, denoting its entire surface area as the "contributing area."

[0158] It can be seen that for this specific virtual image point, Figure 9B and 9C Light tracked from the LCOS 902 results in the formation of three image points (labeled G1, M, and G2) on the retina 906. The middle image point "M" comprises the primary image point, which contributes to the primary / primary virtual image perceived by the viewer. The top image point G1 comprises a first ghost image point, while the bottom image point G2 comprises a second, different ghost image point of the same virtual image point. Note that in further development, the inventors have recognized that it is possible to identify regions of the LCOS 902 that contribute to the primary image point M and / or the ghost image points G1, G2.

[0159] Figures 10A to 10C Shown Figures 9A to 9C LCOS902 and light diagram, which is divided into three corresponding propagation paths, the first propagation path includes light contributing to the bottom ghost image point G2, the second propagation path includes light contributing to the main image point M, and the third propagation path includes light contributing to the top ghost image point G1. Figure 10A As shown, the light contributing to G2 bounces three times before being transmitted by waveguide 908. Figure 10B As shown, the light contributing to M bounces twice before being transmitted by waveguide 908. Figure 10C As shown, the light contributing to G1 bounces once before being transmitted by waveguide 908.

[0160] Each figure (10A, 10B, 10C) also shows the portion of the LCOS 902 that contributes to the corresponding image point, shown as shaded. Thus, it can be seen that the bottom ghost image point G2 is contributed by the area toward the bottom of the LCOS 902, the top ghost image point G1 is contributed by the area toward the top of the LCOS 902, and the main image point is contributed by the entire LCOS 902.

[0161] 9 and 10a to 10c, the aperture 904 (ie, the viewer's pupil) is relatively wide, which explains why the entire LCOS 902 contributes to the principal image point. In other words, in this example, the f-number of the viewing system is relatively low. Figures 10A to 10C, although portions of LCOS 902 also contribute to one or the other ghost images G1, G2, a region of LCOS 902 does not contribute to ghost images G1 and G2, but rather contributes only to primary image point M. The inventors have recognized that this region can be identified as a contributing region for LCOS 902 in this example, and more specifically, as a "primary contributing region," as will be further understood from the subsequent description of the figures. It can be seen that, in this case, the primary contributing region is not limited to a circular or elliptical shape and can take other more complex shapes.

[0162] Figures 11A to 11C The corresponding ray diagrams for different points of the virtual image are shown when the incident aperture is relatively small (ie, the f-number is relatively high). Figure 11A The first field point of the virtual image (i.e. the first virtual image point) is involved, Figure 11B The second field point involving the virtual image, Figure 11C A third field point involving a virtual image. Figure 11A 11C show that not all LCOS 902 contribute to the primary image point. In fact, Figure 11A 11C show that the first region of the LCOS corresponds to the primary image point (referred to herein as the "primary contributing region") and the second region of the LCOS corresponds to the ghost image point (referred to herein as the "secondary contributing region").

[0163] The inventors have recognized that under certain conditions, different corresponding areas of the LCOS 902 (or other display device in the viewing system) will contribute to the primary image, the ghost image, or will not contribute to any visible portion of the image. They have further recognized that this information can be used to optimize the hologram determination process. For example, light from certain portions of the display device can be omitted, or in some cases, the way the hologram encodes certain portions of the display device can be changed so that it actively contributes to the primary image rather than to the ghost image. In addition, additional areas of the display device can be identified that can be configured to actively contribute to the primary image.

[0164] The inventors' findings will be described below using an example involving point cloud holograms. However, they can be applied to other types of holograms, such as Fourier or Fresnel holograms. That is, the LCOS information determined according to the present disclosure can be used to optimize other hologram calculation methods.

[0165] As will be well understood, typically in order to compute a point cloud hologram of an image (such as a virtual image), the image is decomposed into a plurality of individual points (i.e., the image is represented by a plurality of individual points), referred to here as "virtual points," since we are describing the formation of a virtual image. A spherical wave (or "wavelet") is then propagated computationally (i.e., using a model or other theoretical tool) from each virtual point within the virtual image at its intended or desired position to the plane of a display device, such as the plane of the LCOS in the example above. The way in which such wavelets interfere with each other is taken into account, and the resultant amplitude and / or phase of the wavelets that will be received at each pixel of the display device is calculated. The display device can then be tuned in a well-known manner (which will not be described here) so as to exhibit the desired amplitude and / or phase modulation at each pixel location in order to simulate the calculated wavelet and thereby create a hologram of the image.

[0166] The inventors have recognized that for a viewing system having a waveguide and a large viewing distance as described herein, if the entire display device is populated with the net amplitude and phase of the corresponding wavelets for all virtual points, then the hologram to be created may generate one or more ghost images as well as the primary image when displayed and illuminated. This may occur, for example, when the viewing system is configured so that the virtual image is perceived at a limited distance from the viewer. Furthermore, in many cases, light emitted from pixels in some parts of the device will be wasted (i.e., they will not contribute to the image seen or perceived by the viewer) because the physical limitations of the viewing system (such as a small aperture and / or a small display device and / or a large projection distance) will dictate that light from those parts of the device will not enter the viewer's eyes. Therefore, the inventors have recognized that intelligent selection can be applied with respect to which parts of the display device are tuned to provide a hologram. Specifically, if only those portions (or components or regions) of the LCOS that contribute to the primary image are selected, and if the wavelet is computationally propagated from the virtual point of the intended virtual image only to those portions of the LCOS, and not to other portions of the LCOS that do not contribute to the primary image, then the composite amplitude and / or phase of the wavelet received at each pixel within the selected region of the display device can be calculated without performing calculations for any other portion of the display device.

[0167] Then, based on the improved calculations, the display device can be tuned to exhibit the desired amplitude and phase modulation at each pixel location within the selected portion in order to simulate the calculated wavelet, thereby creating a hologram of the primary image. When this is done, when the calculated hologram is displayed on the display device and illuminated, no other portions of the LCOS will be tuned, and therefore no image information will be transmitted from these other portions to the viewer's eye (or other viewing entity). As a result, the viewer will have no usable information, which may cause it to form undesirable "ghost" image points. As a result, the ghost images are eliminated or "extinguished." Furthermore, no computational or image information is wasted because, for a given set of conditions (such as a specific aperture width and position for the eye), only those pixels of the display device known to provide light that will be allowed to pass through the viewer's pupil (or through the aperture of the corresponding other viewing entity) will be tuned.

[0168] Figure 12A and 12B A system 1200 is shown that forms a virtual image including an example virtual point 1201. Viewing system 1200 includes a display device 1202, which in this example is an LCOS SLM, including a contributing region 1203 and a non-contributing region 1207 identified in accordance with the present disclosure. Display device 1202 is arranged to display a hologram of the virtual image and project light encoded according to the hologram into an eye 1205, which includes a pupil (not shown) acting as an aperture, a lens 1209, and a retina 1206 acting as a viewing plane. Lens 1209 and the retina are separated by a separation distance "A." A light source (not shown) is arranged to illuminate display device 1202. Viewing system 1200 also includes a waveguide 1208 located between LCOS 1202 and eye 1205. The image is schematic and, therefore, does not illustrate physiological details of the eye.

[0169] Virtual point 1201 is located upstream of display device 1202. Figure 12A and 12B1 is depicted as a virtual point 1201 located to the left of the display device 1202. The virtual point 1201 has a position defined by spatial coordinates, which in this example include Cartesian (x, y, z) coordinates, but other coordinate systems or other means of identifying the position of the virtual point may also be used. A distance "z" is defined between the virtual point 1201 and the display device 1202 in a direction substantially parallel to the optical axis of the display device 1202. A display-to-lens distance "l" is also defined between the display device 1201 and the eye lens 1209 in a direction substantially parallel to the optical axis of the display device 1201. At a given time, including the viewer position, the values of "z" and "l" will vary depending on the specific arrangement of the viewing system 1200. For example, the display-to-lens distance "l" can be on the order of about 1 meter, and the display-to-image distance "z" can be larger, such as on the order of several meters. However, these numerical examples are purely illustrative and should not be considered limiting.

[0170] The inventors have realized that if a virtual image including virtual image point 1201 is Figure 12A To perceive the position shown, the corresponding image point 1211 must be formed on the retina 1206. Light can be tracked from the virtual point 1201 of the virtual image to the corresponding point 1211 on the retina 1206 via the LCOS 1202.

[0171] It should be understood that due to the possible paths generated / produced by waveguide 1208, more than one possible optical path can be taken between virtual point 1201 and its corresponding point 1211 on the retina via LCOS 1202. According to embodiments, a principal ray can be determined, comprising a ray path among the multiple ray paths between virtual image point 1201 and corresponding point 1211 on the viewing plane (i.e., retina 1206). When this principal ray path is identified, the number of bounces the light undergoes in the waveguide is determined. This number of bounces (B) can be set to the number of bounces that the ray should be traced between the virtual image and the viewing plane. According to embodiments, as an initial step, the principal ray and the associated number of bounces (B) can be identified.

[0172] In this example, ray tracing can determine the portion of the “chief ray” light in the LCOS 1202 that travels between each virtual image point 1201 and the corresponding point 1211 on the retina in order to identify the “contributing region” 1203 for that virtual image point 1201. Figure 12A, ray "r" is depicted as propagating between a virtual image point 1201 and a contributing area 1203 of a display device 1202. According to the inventors, only the wavelets contributing to the contributing area of the LCOS need be modeled (or otherwise computationally accounted for) based on the virtual image point 1201 and the display device 1202. In other words, only the identified contributing area 1203 of the display device 1202 needs to be encoded (or "tuned") in order to generate an appropriate hologram. Such a hologram, when encoded on a display device and properly illuminated, will enable a viewer to perceive the virtual image point 1201 without any ghosting of the virtual image point 1201. This can be seen from the discussion below Figure 13 and 14 Further understanding.

[0173] Figure 12A and 12B Contribution area 1203 in the Figure 13 and 14 The size and shape of the contributing area discussed can be determined based on the size and shape of the entrance aperture of the corresponding viewing entity and associated optical devices (e.g., waveguide geometry, any reflections within the larger optical system, etc.). Thus, when the viewing entity is a human eye, in some cases, the contributing area on the display device may include a basic circle or ellipse similar in size to the receiving pupil, or any other suitable shape, such as a complex shape. However, the present disclosure encompasses more complex shapes for the contributing area. The pupil diameter can be measured or estimated in any suitable manner. For example, measurement of the pupil diameter of the eye can be performed by an eye tracking system. Alternatively, it can be estimated based on a known range of pupil diameters for the eye (e.g., 2-6 mm) or based on another estimate of given ambient light conditions at a given time.

[0174] The contributing area can be set to intentionally contribute an area slightly larger than the pupil (in the aperture plane), and / or contribute an area that is slightly different in shape from the pupil (or other aperture) (in the aperture plane). In this case, not all light from the "contributing area" can pass all the way through the pupil, but the eye will be able to move slightly while still collecting enough light to form a good image on the retina.

[0175] Figure 13 A method for determining contributing and non-contributing areas of a display device according to a main aspect of the present disclosure is shown. Optionally, these determinations can then be used to optimize the generation of one or more holograms to be displayed by a display device such as Figure 12A and 12B The viewing system 1200 is displayed and illuminated. In reference Figure 13In the described method, a viewing system includes a lens having an "f" number (i.e., focal length and aperture) and a camera. The light-sensitive component of the camera can be, for example, a CCD array and is located in the viewing plane. Functionally, the lens and camera replace the eye lens and retina of the viewer's human eye and are only used in the process of determining contributing and non-contributing areas of the display device. These areas of the display device can be determined for multiple viewing positions (e.g., eye position within an eye movement box) and / or multiple image distances (e.g., a virtual image distance in front of a vehicle). In some aspects, reference Figure 13 The disclosed method can be considered a precursor to hologram computing. The method can be considered an optimization or even calibration process.

[0176] As will be well understood, each virtual image to be generated may be represented by one or more virtual image points, each having a corresponding position, eg defined by (x, y, z) coordinates. Figure 13 Steps 1302 through 1312 of method 1300 (described in detail below) can be applied individually to each virtual image point within the virtual image to be created. Furthermore, method 1300 is adapted to a specific set of conditions (i.e., specific measurements and constraints) for the viewing system. Thus, any given iteration (or "run") of method 1300 is adapted to establish a specific image to be created (virtual image point by virtual image point) and is adapted to a system with a specific display-to-image distance "z," a specific distance "d" between the display device and the retina, a specific aperture (pupil) width, and a specific virtual image distance at which the eye is focused. Iterations of method 1300 are also specific to a specific size and type of display device, and to an allowed viewing window for a specific eye position. There may be additional measurements and / or constraints specific to each iteration of the method. Depending on the embodiment, if any of those measurements or constraints change, method 1300 can be rerun to redefine the contributing area of the display device under the changed circumstances. However, it should be understood that, depending on the embodiment, certain tolerances may be applied to one or more of those measurements or constraints so that if they change by less than a predetermined amount and / or for less than a predetermined length of time, the method may not have to be rerun. The rules for when the method should be re-iterated may be determined on a per-system basis.

[0177] Method 1300 may be performed by a suitable processor. The processor may include a hologram engine, or be included within or in communication with a hologram engine. The processor or hologram engine may be included in a light engine.

[0178] Prior to executing method 1300, the processor may obtain or receive boundary information about the viewing system. For example, it may obtain or receive information about the dimensions of components such as the display device, information about the absolute and / or relative positions of various components and the viewing system (e.g., a potential human viewer), information about light sources, etc.

[0179] According to the method 1300, in a first step 1302, the position of a virtual image point (also referred to herein as a "virtual point") is obtained based on the position where the virtual image is to be perceived, for example, the coordinates [x virtual ,y virtual ,Z virtual ]. A virtual image distance between lens 1209 and the virtual point is then obtained or determined. The virtual image distance may be set or determined by the processor executing method 1300, or may be set or determined by another entity and communicated to the processor. In some arrangements, it may be pre-set or selected from a plurality of possible virtual image distances. In actual operation, when the viewing system is an eye, eye tracking or head tracking information may be used to determine the virtual image distance.

[0180] In a second step 1304, the required distance "A" between the lens and the sensor is determined to focus on the virtual image point. Each virtual image point can also be defined by an angle, see Figure 4 The “angular content” mentioned in this article is relative to the virtual image point of the virtual image.

[0181] In a third step 1306, the number of reflections or bounces "B" of light within the waveguide associated with the primary or main image formed by the viewing system is determined. Those skilled in the art of optics will appreciate that the waveguide produces multiple replicas of light associated with a virtual image point, and each replica may be associated with a different number of light bounces / reflections within the waveguide. By way of example only, one way to determine B is to determine the intersection of the chief ray of each possible light propagation path in the waveguide with the display device and select the number of reflections / bounces that places the chief ray closest to the center of the display device. Advantageously, this approach maximizes the area of the display device contributing to the viewing system.

[0182] Alternatively, another way of calculating the number of bounces used in the third step 1306 includes the following sub-steps 1 to 5:

[0183] 1. Eye position is known and used as input

[0184] 2. For a first number of bounces B, a ray is traced from the center of the display device to the determined eye position. The extrapolation of this ray to the virtual image defines the field of view angle (θ) for this number of bounces (B). B ).

[0185] 3. For a second number of bounces B+1, a ray is traced from the center of the display device to the determined eye position. Extrapolation of this ray to the virtual image defines the field of view angle (θ) for this number of bounces (B+1). B+1 ).

[0186] 4.B is for θ B and θ B +(θ B+1 -θ B ) / 2 the number of bounces of the angle content between

[0187] 5.B+1 is used for θ B +(θ B+1 -θ B ) / 2 and θ B Number of bounces between angular content

[0188] The outputs from the first step 1302 (ie the coordinates of the virtual image point) and the third step 1306 (parameter B) are used in the fourth step 1308 to determine the corresponding image position / point [x sensor ,y sensor , Z sensor ]. That is, the fourth step 1308 determines the point on the sensor that receives the light of the virtual image point. In other words, the point on the sensor where the virtual image point is imaged. This point on the sensor is described below with respect to Figure 14 is called the principal image point [x sensor ,y sensor ,z sensor ]. By way of example only, computational ray tracing from a virtual point to the sensor for B bounces within the waveguide may be used, but the present disclosure is not limited to this approach for the fourth step.

[0189] Those skilled in the art will appreciate that it is possible to identify the virtual point [x virtual ,y virtual ,z virtual ] to the point [x virtual ,y virtual ,z virtual ] principal ray (or simply principal line). Again, computational ray tracing can be used to identify or trace the principal ray, but other methods are equally applicable. In the fifth step 1310, the display device intersection [x LCOS (B),y LCOS (B),z LCOS (B)], where the display device intersection point is the location on the display device where the primary ray intersects the display device. The display device intersection point can be determined, calculated, or measured, for example, by computational ray tracing.

[0190] In a sixth step 1312, the intersection point [xLCOS (B),y LCOS (B),z LCOS (B)] The area of the display device associated with it. The area of the display device can be geometrically concentrated at the point [x LCOS (B), y LCOS (B),z LCOS (B)] on. For example, the area can be circular or elliptical, but other more complex shapes are also conceivable. If the area is a regular shape, such as a circle or an ellipse, the radius of the area can be determined, for example, depending on the f-number of the lens of the viewing system. This area is referred to here as the "main contribution area" because it corresponds to the main image formed by the viewing system. The word "contribution" reflects that the pixels of the display device within the recognition area of the display device are the pixels that provide the necessary information content to the sensor. Other areas of the display device (i.e. other pixels of the display device) do not contribute to the formation of the image point on the sensor. Of course, other pixels can contribute to other image points on the sensor that are related to other virtual image points.

[0191] The method according to the main aspect of the present disclosure ends with determining the main contributing area of the display device. Optionally, the hologram can be determined based on the main contributing area rather than the entire area of the display device.

[0192] Therefore, in an optional seventh step 1314, hologram components are determined for the primary contributing region based on the virtual point. Specifically, optical parameters of the primary contributing region are determined. The optical parameters can be the amplitude and / or phase of each pixel in the primary contributing region. For example, point cloud methods familiar to those skilled in the art can be used to determine the light amplitude and phase for each pixel within the primary contributing region based on the propagation of light from the virtual point to the primary contributing region. The hologram component of the virtual point can be stored and combined with hologram components of other virtual points as part of the iterative process described in the next paragraph to create a complete hologram for the entire virtual image.

[0193] In summary, in the seventh step 1314, a light modulation value (eg, amplitude and / or phase value) is assigned to each pixel value of the display device within the main contribution area. This is done by considering the light wave from [x virtual ,y virtual , z virtual ] to the main contributing region and adds the amplitude and / or phase to [x LCOS (B),y LCOS (B), z LCOS(B)] is achieved on pixels of a display device within a desired radius of the virtual image point. That is, the amplitude and / or phase of light originating from the virtual image point and arriving at each point (i.e., pixel) of the primary contribution area is determined by considering the propagation of the light wave, i.e., the amplitude and / or phase of the light wave after traveling the distance from the virtual image point to each pixel. This determination can be performed by any of a variety of different techniques known to those skilled in the art of optics. Such determination can be performed by experimental measurement.

[0194] Steps 1 through 7 can be repeated for each virtual point within the virtual image to be projected using the hologram. For example, multiple hologram components can be summed together to produce a composite hologram for each pixel of a display device. For example, the complex amplitudes propagated from all virtual image points can be summed at each pixel. If the hologram is to be displayed on a phase-only modulator, the amplitude component of the resulting complex amplitude sum can be ignored, retaining only the phase. More generally, the result is a diffraction structure corresponding to the virtual image, which, if displayed and illuminated on a display device within a viewing system, would form that virtual image.

[0195] The hologram can be displayed or encoded on a display device, which will then be tuned to modulate the light so that the viewer can perceive the virtual image at the desired virtual image distance.

[0196] Method 1300 can be performed substantially simultaneously (or in very rapid succession) for each of a plurality of virtual points within a virtual image, so that for a given viewing setting and specific numerical measurements and constraints, a suitable hologram of the entire virtual image can be derived very quickly and encoded onto a display device. The method can be rerun if anything changes that may affect the identification of the contributing area and / or the desired tuning of the display device. The processor can be configured to rerun the method on a time-controlled loop, and / or in response to a signal indicating that a change has occurred, and / or when the content or identity of the desired virtual image changes. The processor can include or be in communication with a memory for storing previously calculated data. For example, a lookup table or other storage device can be provided that indicates the active area of the display device for a particular virtual image or virtual point under a specific set of measurements and / or constraints.

[0197] Method 1300 can be run (or rerun) very quickly to display multiple different virtual images in rapid succession and / or to accurately respond to changing conditions, such as user movement. Figure 12AWhile only one eye is shown in the system, method 1300 can be configured to account for both eyes of the viewer. Furthermore, while some of the above description may refer to aperture width, it should be understood that the pupil (and most other apertures used to view an entity) is two-dimensional and can vary in size in each of these two dimensions. Method 1300 can be configured to account for two-dimensional aperture size and variations therein.

[0198] The inventors found that using reference Figure 13 The disclosed method can effectively determine the hologram of a virtual image. However, the inventors have also observed that in some cases, while not all areas of the LCOS that would normally transmit light that forms ghost images are used, only a relatively small portion of the LCOS is utilized. In another notable technical advance, the inventors have discovered a method for using additional areas of the LCOS beyond the main contributing area and calculating hologram values for these additional areas, which will enable them to contribute light to enhance the main image rather than forming unwanted ghost images.

[0199] As will be well understood, the optical path taken by a ray through a waveguide in a viewing system may increase its path length relative to the path lengths of corresponding other rays. Typically, this increase may be small compared to the virtual image distance "v" and therefore will not be visible to the eye.

[0200] Figure 14 A further improved method 1400 according to additional insights made by the inventors is shown, which may be applied to systems such as Figure 12A and 12B system 1200. Figure 14 The method 1400 includes all the steps of the method 1300 of FIG. 13 , and in addition, it includes processing of one or more ghost image points corresponding to the virtual point, which ghost image points may also exist and typically result in the perception of one or more ghost images of the virtual image.

[0201] Method 1400 may be performed by a suitable processor. The processor may include a hologram engine, or be included within or in communication with a hologram engine. The processor or hologram engine may be included within a light engine.

[0202] The processor may obtain or receive boundary information about the system before executing the method. For example, it may obtain or receive information about the size of components such as a display device, information about the absolute and / or relative positions of various components and a viewer, information about light sources, etc.

[0203] In some cases, the inventors have discovered that ghost image points are produced by light traveling from the corresponding virtual point through a portion of the display device that is different from the "primary contributing area" through which the chief ray of the primary image travels. In the preceding figures, these portions of the display device are referred to as "secondary contributing areas." The light that produces one or more ghost image points can be said to include one or more "ghost rays." The light that produces the ghost image can experience a different number of bounces within the waveguide than the number of bounces corresponding to the primary image in order to also pass through the narrow pupil of the viewer's eye and coincide with the retina. Thus, if it is determined that the chief ray corresponding to the primary image experienced "B" bounces within the waveguide, it can be determined that the light corresponding to the ghost image experienced "B+ΔB" bounces, where ΔB can be a negative or positive integer, typically a single number, such as in the range of -5 to +5.

[0204] according to Figure 14 Improved method 1400, in Figure 13 The fourth step 1308 of the method 1300 is to establish the position of the main image point on the viewing plane (e.g., its coordinates (x sensor ,y sensor ,z sensor )) after, Figure 13 The subsequent steps of method 1300 may continue to be performed, and in addition, for example, in parallel or at a later time, for at least one ΔB value, another set of steps may be performed as follows. In summary, Figure 14 The improved method 1400 is based on the coordinates of the virtual point [x virtual ,y virtual ,z virtual ] to determine how many bounces "B + ΔB" the ghost light will experience in order to form a ghost image point at the viewing plane. The improved method 1400 then determines a translated (or corrected) position of the virtual point from which the light can propagate and experience "B + ΔB" bounces within the waveguide and reach the primary image point on the viewing plane, rather than forming a separate ghost image point. The position on the LCOS through which the light travels from the translated position of the virtual point to the primary image point can then be identified, and the position can be encoded with a hologram accordingly. Thus, one or more additional regions of the LCOS (rather than the primary contributing region) can be encoded with hologram values to contribute to the primary image while still avoiding the generation of ghost images.

[0205] In more detail, the improved method 1400 is as follows:

[0206] In a first further step 1402, from the primary image point (x sensor ,y sensor ,z sensor) are traced back to the virtual image, but for rays that experience "B+ΔB" bounces / reflections within the waveguide (instead of B bounces).

[0207] In a second further step 1404, the position of the secondary virtual point of the virtual image (eg coordinates [x virtual (ΔB), y virtual (ΔB),z virtual (ΔB)])), the secondary virtual point will be imaged to the primary image point [x sensor ,y sensor ,z sensor ](that is, the light that propagates through the display device, waveguide and incident aperture to correspond to the position [x sensor ,y sensor , z sensor ] coincides with the viewing plane at ]), if the light undergoes "B+ΔB" bounces. The term "secondary virtual point" is used herein as a shorthand for the secondary (i.e., shifted or modified) position of the (primary) virtual point. That is, the inventors have recognized that if the position of the virtual point is moved to the "secondary virtual point" position (x virtual (ΔB),y virtual (ΔB), z virtual (ΔB)], then any light from the "secondary virtual point" that undergoes "B+ΔB" bounces / reflections in the waveguide will contribute to the primary image at the viewing plane.

[0208] In summary, the third further step 1406 comprises determining the coordinates [x LCOS (B+ΔB),y LCOS (B+ΔB),z LCOS ], for B+ΔB bounces in the waveguide from [x virtual (ΔB),y virtual (ΔB),z virtual ] to the viewing plane. In some cases, the z virtual To account for the different path lengths through the waveguide (ie because of the different number of bounces), this principal ray may be referred to as a "secondary principal ray".

[0209] In more detail, in a third further step 1406, a point on the display device is identified, where the distance from the secondary virtual point to the primary image point [x sensor ,y sensor ,z sensor ] will propagate through this point, experiencing B+ΔB bounces in the waveguide. This point on the display device has coordinates [x LCOS (B+ΔB), y LCOS(B+ΔB),z LCOS ].

[0210] In the fourth additional step 1408, for the point [x LCOS (B+ΔB),y LCOS (B+ΔB),z LCOS ] assigns a radius or other suitable indicator of the extent or size of the region to which it is associated. LCOS (B+ΔB), y LCOS (B+ΔB),z LCOS ] is referred to here as the "additional contribution region" because it propagates light that contributes to the principal image point at the viewing plane, but only when that light source is displaced or corrected from the (primary) virtual point (i.e., [x virtual (ΔB),y virtual (ΔB),z virtual ]), instead of [x as determined in the second further step 1404 virtual ,y virtual ,z virtual ].

[0211] The fourth further step 1408 is similar to the sixth step 1312. Specifically, the fourth further step 1408 includes identifying the intersection point [x LCOS (B+ΔB),y LCOS (B+ΔB),z LCOS (B)] The area of the display device associated with the display device. The area of the display device can be geometrically concentrated at the point [x LCOS (B+ΔB), y LCOS (B+ΔB),z LCOS (B)] on. For example, this area can be circular or elliptical, but other more complex shapes are also conceivable. If this area is a regular shape, such as a circle or an ellipse, the radius of this area can be determined, for example, based on the f-number of the lens of the viewing system. This area is referred to here as the "additional contribution area" because it will transmit light that contributes to the virtual image if an appropriate hologram is calculated based on the shifted or corrected position of the (primary) virtual point.

[0212] The fifth further step 1410 is similar to the seventh step 1314. The fifth further step 1410 is optional. In the fifth further step 1410, based on the (master) virtual point [x virtual (ΔB),y virtual (ΔB),z virtual], and determine the hologram components for the additional contribution area. Specifically, determine the optical parameters of the additional contribution area. The optical parameters can be the amplitude and / or phase of each pixel in the additional contribution area. For example, a point cloud method familiar to those skilled in the art can be used to calculate the optical parameters of the additional contribution area based on the light from different virtual points [x virtual (ΔB), y virtual (ΔB),z virtual ] to the additional contribution area, and the light amplitude and phase are determined for each pixel in the additional contribution area. virtual (ΔB),y virtual (ΔB),z virtual ] can be stored and combined with hologram components of other virtual points as part of the iteration described in the next paragraph to build a complete hologram for the entire virtual image.

[0213] The desired light modulation associated with a single virtual point output by the display device may be referred to as the “hologram component” of that virtual point. During subsequent iterations of method 1300, hologram components may be stored by the processor for one or more other virtual points within the virtual image to be created.

[0214] Figure 14 A further improvement of the method 1400 can be made with steps 1402 to 1410 Figure 13 Steps 1302 to 1314 of the method 1300 are repeated for each virtual point within the virtual image to be created. When the modulation behavior and the corresponding hologram components have been determined for each virtual point, the hologram components can be added together to produce a composite modulation behavior for each pixel of the display device. This composite modulation behavior represents the diffraction structure or hologram of the virtual image, which, if displayed and illuminated on a display device within the viewing system, results in the formation of only a primary image without any ghost images. Figure 14 The main image formed as a result of the improved method 1400 may be better than that formed by Figure 13 The corresponding primary image produced by method 1300 is brighter.

[0215] The processor may output data corresponding to the hologram in any suitable manner. The hologram may be encoded onto a display device. As a result, the display device will be tuned to modulate light so that a viewer can perceive the virtual image at a desired virtual image distance without forming any ghost images.

[0216] Method 1400 can be performed substantially simultaneously (or in very rapid succession) for each of a plurality of virtual points within a virtual image, so that for a given viewing setting and specific numerical measurements and constraints, a suitable hologram of the entire virtual image can be derived very quickly and encoded onto a display device. The method can be rerun if there are any changes that may affect the identification and / or required tuning of the display device. The processor can be configured to rerun the method on a time-controlled loop, and / or in response to a signal indicating that a change has occurred, and / or when the content or identity of the desired virtual image changes. The processor can include or can be in communication with a memory for storing previously calculated data. For example, a lookup table or other storage device can be provided that indicates the active area of the display device for a particular virtual image or virtual point under a specific set of measurements and / or constraints.

[0217] Method 1400 can be run (or rerun) very quickly to display multiple different virtual images in rapid succession and / or to accurately respond to changing conditions, such as user movement. Figure 12A While only one eye is shown in the system, method 1400 can be configured to account for both eyes of the viewer. Furthermore, while some of the above description may refer to aperture width, it should be understood that the pupil (and most other apertures used to view an entity) is two-dimensional and can vary in size in each of these two dimensions. Method 1400 can be configured to account for two-dimensional aperture sizes and variations therein.

[0218] According to a main aspect of the present disclosure, the inventors have discovered that each virtual image point corresponds to a different main contributing area on the display device. The inventors have also realized that this means that light from different parts of the virtual image (i.e., different virtual image points) follows different optical paths through the system. Figure 15A and 15B In the illustrated embodiment, the inventors have configured the system such that, in simple terms, (i) the virtual image comprises multiple discrete virtual image components or regions, and (ii) the light of each virtual image component is associated with a different number of bounces / reflections within waveguide 1508.

[0219] Figure 15A An image 1552 is shown for projection, comprising eight image regions / components V1 to V8. Figure 15A Eight image components are shown by way of example only; the image 1552 may be separated into any number of components. Figure 15AAlso shown is a coded light pattern 1554, which, when transformed by a lens of a suitable viewing system, can reconstruct the image 1552. The coded light pattern 1554 includes first through eighth sub-holograms or components H1 through H8 corresponding to first through eighth image components / regions V1 through V8. Figure 15A It further shows how a hologram calculated according to the present disclosure can effectively decompose image content by angle. Thus, a hologram can be characterized by the guidance of light it performs. This is in Figure 15B Specifically, holograms according to the present disclosure direct light into multiple discrete regions. In the example shown, the discrete regions are disks, but other shapes are also contemplated. After propagation through the waveguide, the optimal disk size and shape can be correlated with the size and shape of the viewing system's entrance pupil. This light directing occurs solely due to the specific method of determining the hologram disclosed herein.

[0220] Figure 15C Shown according to Figure 15A and 15B An improved viewing system 1500 for identification is shown. Figure 13 Method 1300 or Figure 14 The method 1400 can be applied to Figure 15A and 15B The scheme shown.

[0221] Viewing system 1500 includes a display device, which in this arrangement includes LCOS 1502. LCOS 1502 is arranged to display a modulation pattern (or "diffraction pattern") comprising a hologram and projects holographically encoded light into eye 1505, which includes a pupil acting as aperture 1504, a lens 1509, and a retina (not shown) acting as a viewing plane. There is a light source (not shown) arranged to illuminate LCOS 1502. Lens 1509 of eye 1505 performs the conversion of the hologram into an image.

[0222] Viewing system 1500 also includes a waveguide 1508 located between LCOS 1502 and eye 1505 . Figure 15C The projection distance in may be relatively large. However, as described with respect to the previous figures, the presence of waveguide 1508 enables all angular content from LCOS 1502 to be received by eye 1505 even at this relatively large projection distance. This is because waveguide 1508 acts as a pupil expander in the manner already described above.

[0223] Furthermore, in this arrangement, when LCOS 1502 is encoded according to the methods described herein, waveguide 1508 can be oriented at an angle relative to LCOS 1502 to establish a unique relationship between light from LCOS 1502 and the virtual image that a viewer will perceive. The size, orientation, and position of waveguide 1508 are configured to ensure that light from each portion of the virtual image enters waveguide 1508 and is guided along its elongated axis, bouncing between the substantially planar surfaces of waveguide 1508. Whenever light reaches the second planar surface (closest to eye 1505), some light is transmitted and some light is reflected.

[0224] Figure 15C A total of nine "bounce" points B0 to B8 are shown along the length of the waveguide 1502. The reader will note that the center of the image 1552 remains blank. Figure 15C The zeroth through ninth light "bounce" or reflection points B0 through B8 within the waveguide are shown. Although light associated with all points of the image (V1-V8) is transmitted out of the waveguide each time it "bounces" from the second planar surface of the waveguide 1508, only light from one of the angular portions of the image (e.g., light from one of V1 through V8) has a trajectory that enables it to reach the eye 1505 from each corresponding "bounce" point B0 through B8. Furthermore, light from a different angular portion of the image V1 through V8 reaches the eye 1505 from each corresponding "bounce" point. Figure 15C Light from all the different angular content is shown emitted at each "bounce" point (depicted by the multiple short arrows at each transmission point), but then only the optical path of each angular content that reaches the eye 1505 (that will actually reach the eye 1505) from that corresponding portion of the waveguide is shown, and therefore will contribute to the corresponding portion of the virtual image that the viewer will perceive. For example, for the zeroth bounce B0, the light transmitted by waveguide 1508 is simply refracted and does not experience any reflections therefrom. Light from the eighth sub-hologram H8 reaches the eye from the zeroth bounce B0. For the next bounce B1, light transmitted by waveguide 1502 experiences one bounce therefrom before being transmitted. Light from the seventh hologram H7 reaches the eye from the next bounce B1. This continues until the light transmitted by waveguide 1508 at the final bounce B8 has experienced eight bounces before being transmitted and reaching the eye 1505, and includes light encoded according to the first hologram H1.

[0225] In the example shown in FIG15 , from each bounce point, only one region of the image reaches the eye. Thus, when determining the hologram as described herein, a spatial correlation is established between regions of the virtual image and their associated bounce points on the waveguide. In some other examples, there may be relatively little overlap, such that one region of the image originates from two adjacent transmission points and is therefore included in two adjacent disks of light propagating from the waveguide to the viewing plane.

[0226] Thus, the insights made by the inventors and the methods and arrangements described herein enable the production of diffraction patterns (or light modulation patterns) including holograms that, when displayed on an LCOS or other suitable display device, enable light to be effectively emitted therefrom as multiple "disks" or beams, each "disk" or beam corresponding to (more specifically encoding) a different respective portion of a respective virtual image.

[0227] Thus, improved methods and arrangements are described herein that enable holograms to be calculated and displayed on a suitable display device in such a way that a viewer sees a clear image when the display device is illuminated by a suitable light source. The image seen by the viewer may be free of ghosting and may be brighter due to contributions from light that would traditionally contribute to the ghosting images rather than to the single primary image.

[0228] The improved methods and arrangements described herein can be implemented in a variety of different applications and viewing systems. For example, they can be implemented in a head-up display (HUD). In contrast to many conventional HUDs that create virtual images, the improved methods and arrangements described herein can be implemented to create virtual images at a limited image distance (which can be selected and adjusted by a suitable controller) while still eliminating ghost images.

[0229] Although virtual images have been discussed herein, requiring the eye to transform received modulated light to form a perceived image, the improved methods and arrangements described herein can be applied to real images.

[0230] Additional Features

[0231] The embodiments relate to electrically activated LCOS spatial light modulators by way of example only. The teachings of the present disclosure may equally be implemented on any spatial light modulator capable of displaying computer-generated holograms according to the present disclosure, such as any electrically activated SLM, optically activated SLM, digital micromirror device, or microelectromechanical device.

[0232] In some embodiments, the light source is a laser, such as a laser diode.

[0233] The disclosed system can be used to provide an improved head-up display (HUD) or head-mounted display. In some embodiments, a vehicle is provided that includes a holographic projection system installed in the vehicle to provide a HUD. The vehicle can be a motor vehicle such as a car, truck, van, delivery truck, motorcycle, train, airplane, boat, or ship.

[0234] The quality of the holographic reconstruction can be affected by the so-called zero-order problem, which is a consequence of the diffraction properties of the pixelated spatial light modulators used. This zero-order light can be considered "noise" and includes, for example, specularly reflected light and other unwanted light from the SLM.

[0235] While the examples describe illuminating the SLM with visible light, those skilled in the art will appreciate that the light source and SLM can also be used to direct infrared or ultraviolet light, such as disclosed herein. For example, those skilled in the art will be aware of techniques for converting infrared and ultraviolet light into visible light for the purpose of providing information to a user. For example, the present disclosure extends to the use of phosphors and / or quantum dot technology for this purpose.

[0236] The methods and processes described herein may be embodied on a computer-readable medium. The term "computer-readable medium" includes media arranged to store data temporarily or permanently, such as random access memory (RAM), read-only memory (ROM), buffer memory, flash memory, and cache memory. The term "computer-readable medium" should also be taken to include any medium or combination of media that can store instructions for execution by a machine, such that when the instructions are executed by one or more processors, the machine performs, in whole or in part, any one or more of the methods described herein.

[0237] The term "computer-readable medium" also encompasses cloud-based storage systems. The term "computer-readable medium" includes, but is not limited to, one or more tangible and non-transitory data storage repositories (e.g., data volumes) in the example form of solid-state memory chips, optical disks, magnetic disks, or any suitable combination thereof. In some example embodiments, instructions for execution may be conveyed by a carrier medium. Examples of such carrier media include transient media (e.g., a propagated signal conveying the instructions).

[0238] It will be apparent to those skilled in the art that various modifications and variations can be made without departing from the scope of the appended claims. This disclosure covers all modifications and variations within the scope of the appended claims and their equivalents.

Claims

1. A light engine arranged to provide spatially modulated light to a viewing system having an entrance pupil, wherein the light engine comprises: a display device arranged to display the hologram and to spatially modulate light in accordance with the hologram; as well as a hologram engine arranged to receive contribution information, the contribution information identifying contributing regions and non-contributing regions of a display device based on a location of an entrance pupil, wherein the contributing regions of the display device substantially propagate spatially modulated light that passes through the entrance pupil at the location, and the non-contributing regions of the display device substantially propagate spatially modulated light that is blocked by the entrance pupil at the location, wherein the contributing regions include at least one primary contributing region of the display device and at least one secondary contributing region of the display device, wherein the contribution information further identifies: (i) at least one primary contributing region of the display device that propagates light toward the viewing system that contributes to a primary image received through an entrance pupil, and (ii) at least one secondary contributing region of the display device that propagates light toward the viewing system that contributes to a secondary image received through the entrance pupil, wherein the secondary image is a low-intensity copy of the primary image, and Wherein the hologram engine is further arranged to determine a hologram based on at least one main contributing area of the display device, and output the hologram to the display device for display.

2. The light engine according to claim 1, wherein: The light engine further comprises a monitoring system arranged to determine a position of an entrance pupil of the viewing system.

3. The light engine of claim 1, wherein: The light engine further comprises a waveguide arranged to receive the spatially modulated light from the display device and to provide a plurality of different light propagation paths for the spatially modulated light from the display device to the entrance pupil, wherein each primary contributing area and each secondary contributing area corresponds to a different respective light propagation path provided by the waveguide.

4. The light engine of claim 1, wherein: The viewing system is arranged to form an image corresponding to the hologram.

5. The light engine of claim 1, wherein: The contribution information identifies respective contributing and non-contributing regions of the display device for each of a plurality of image points of the primary image and the secondary image.

6. The light engine of claim 5, wherein: The hologram includes a plurality of sub-holograms, wherein each sub-hologram is determined by the hologram engine based on contribution information of a corresponding image point of the image.

7. A method for determining a hologram for display on a display device; the method comprising: (i) determining the position of an entrance pupil of a viewing system arranged to view the hologram; (ii) identifying a contributing area and a non-contributing area of the display device, wherein the contributing area of the display device substantially propagates light that passes through an entrance pupil at the determined location of the viewing system, and the non-contributing area of the display device substantially propagates light that is blocked by the entrance pupil at the determined location of the viewing system; (iii) identifying at least one primary contributing region of the display device that provides light contributing to a primary image received through the entrance pupil and at least one secondary contributing region of the display device that provides light contributing to a secondary image received through the entrance pupil, wherein the secondary image is a low-intensity copy of the primary image; as well as (iv) determining a hologram based on at least one main contributing area of the display device.

8. The method for determining a hologram according to claim 7, wherein: Steps (ii) to (iv) are performed for each of a plurality of image points of the primary and secondary images, and wherein determining the location of each contributing area for each image point comprises identifying a location at which a ray travelling from said each image point to the entrance pupil intersects the display device.

9. The method for determining a hologram according to claim 7, wherein: The viewing system comprises a waveguide arranged to receive spatially modulated light from the display device and to provide a plurality of different light propagation paths for the spatially modulated light from the display device to the entrance pupil, and wherein step (iii) comprises determining, for each image point, a number of internal reflections B within the waveguide pupil expander corresponding to the primary image.

10. The method of claim 9, wherein: The step of determining, for each image point, a number B of internal reflections within the waveguide pupil expander corresponding to the primary image is based on an angle associated with the respective image point, wherein the angle is an angle relative to the optical axis of a line formed by extrapolating a line connecting the center of the display device and the determined entrance pupil position outward toward the image.

11. The method according to claim 10, wherein: Each primary contributing area and each secondary contributing area has a size based on the diameter of the entrance pupil.

12. The method of claim 9, wherein: Steps (ii) and (iii) for each image point include: For B light reflections in the waveguide, the image point [x virtual ,y virtual ,z virtual ] to the viewing plane of the viewing system to identify the position [x sensor ,y sensor ]; For the image point [x virtual ,y virtual ,z virtual ] to the position [x sensor ,y sensor ] with B reflections, determine the coordinates of the main ray at the display device [x LCOS (B),y LCOS (B)]; and Identify by [x LCOS (B),y LCOS (B)] Active pixels of a display device within a defined area.

13. The method of claim 7, wherein: Step (iv) comprises determining one or more values of the hologram only in at least one main contribution area of the display device, or wherein step (iv) comprises excluding hologram values associated with at least one secondary contribution area during determination of the hologram, or wherein step (iv) comprises limiting the hologram determination only to at least one main contribution area of the display device.

14. The method of claim 7, further comprising excluding values of the hologram in areas of the display device that are not included within the primary contribution area.

15. The method for determining a hologram according to claim 12, wherein: Step (iv) comprises determining for each image point a sub-hologram within the respective at least one main contribution area and combining the sub-holograms to form the hologram.

16. The method for determining a hologram according to claim 15, wherein: Each sub-hologram consists of a light wave that is transmitted from the image point [x virtual ,y virtual ,z virtual ]Amplitude and / or phase hologram components determined by propagating to the corresponding main contribution area.

17. The method of determining a hologram of claim 15, further comprising identifying, for each image point, an additional contributing area of the display device associated with B + ΔB bounces.

18. The method for determining a hologram according to claim 17, wherein: Identify additional contributing areas including: For B+ΔB bounces, the light is reflected from the position [x sensor ,y sensor ] Track back to the virtual image plane z virtual ; Determine the virtual point coordinates [x virtual (ΔB),y virtual (ΔB),z virtual ], which will be imaged to position [x sensor ,y sensor ]; For a virtual point with B+ΔB rebounds [x virtual (ΔB),y virtual (ΔB),z virtual ] to the viewing plane, determine the coordinates of the main ray at the display device [x LCOS (B+ΔB),y LCOS (B + ΔB)]; and Identify the display device at the coordinates [x LCOS (B+ΔB),y LCOS Additional active pixels within the second area defined by [B+ΔB]].

19. The method of determining a hologram as claimed in claim 18, further comprising determining an additional sub-hologram for each additional active pixel and combining the additional sub-hologram with the sub-hologram.

Images