Waveguide pupil expansion

Abstract

A light engine is arranged to form an image visible from an observation window, wherein the light engine includes a display device arranged to display a hologram of the image and to spatially modulate light according to the hologram. The hologram is configured to angularly distribute the spatially modulated light of the image according to the location of the image content, such that angular channels of the spatially modulated light correspond to respective consecutive regions of the image. The light engine also includes a waveguide pupil expander arranged to receive the spatially modulated light and provide a plurality of different light propagation paths for the spatially modulated light from the display device to the observation window; and a control device disposed between the waveguide and the observation window. The control device includes at least one aperture arranged such that a first observation position within the observation window receives a first light channel spatially modulated by the hologram according to a first region of the image, and a second observation position within the observation window receives a second light channel spatially modulated by the hologram according to a second region of the image.

Description

Technical Field

[0001] The present disclosure relates to image projection. More specifically, the present disclosure relates to holographic projection and methods for determining diffraction structures such as holograms or kinoform patterns. Some embodiments relate to real-time hologram computation based on eye tracking information. Some embodiments relate to virtual image projection. Other embodiments relate to real image projection. Embodiments relate to observing a projected image through a waveguide. Embodiments relate to controlling light for an image projected through a waveguide. Some embodiments relate to a light engine such as a picture generation unit. Some embodiments relate to a head-up display. Background Art

[0002] Light scattered from an object contains both amplitude and phase information. This amplitude and phase information can be captured, for example, on a photographic plate using well-known interference techniques to form a holographic recording or "hologram" consisting of interference fringes. The hologram can be reconstructed by illuminating it with appropriate light to form a two- or three-dimensional holographic reconstruction or replay image representing the original object.

[0003] Computer-generated holography can numerically simulate interference processes. Computer-generated holograms can be computed using techniques based on mathematical transformations such as the Fresnel or Fourier transforms. These types of holograms are referred to as Fresnel / Fourier transform holograms or simply Fresnel / Fourier holograms. Fourier holograms can be considered either Fourier domain / planar representations of an object or frequency domain / planar representations of an object. Computer-generated holograms can also be computed, for example, using coherent ray tracing or point cloud techniques.

[0004] The computer generated hologram may be encoded on a spatial light modulator arranged to modulate the amplitude and / or phase of the incident light. For example, light modulation may be achieved using electrically addressable liquid crystals, optically addressable liquid crystals, or micromirrors.

[0005] A spatial light modulator typically comprises a plurality of individually addressable pixels, which may also be referred to as cells or elements. The light modulation scheme may be binary, multi-level, or continuous. Alternatively, the device may be continuous (i.e., comprise no pixels), so that the light modulation may be continuous across the device. A spatial light modulator may be reflective, meaning that the light is modulated to reflect the output. A spatial light modulator may also be transmissive, meaning that the light is modulated to transmit the output.

[0006] The system described here can be used to provide holographic projectors, which have applications in head-up displays (HUDs) and light detection and ranging (LIDAR), for example. Summary of the Invention

[0007] Various aspects of the disclosure are defined in the accompanying independent claims.

[0008] The present disclosure relates to image projection. It relates to a method of image projection and an image projector including a display device. The present disclosure also relates to a projection system including an image projector and an observation system. The present disclosure is equally applicable to monocular and binocular observation systems. The observation system may include one or more eyes of an observer. The observation system includes an optical element with optical power (e.g., the lens of a human eye) and an observation plane (e.g., the retina of a human eye). The projector may be referred to as a "light engine." The display device and the image formed (or perceived) using the display device are spatially separated from each other. The observer forms or perceives the image on the display plane. In some embodiments, the image is a virtual image, and the display plane may be referred to as a virtual image plane. The image is formed by illuminating a diffraction pattern (e.g., a hologram) displayed on the display device.

[0009] A display device includes pixels. The pixels of the display device diffract light. According to well-known optical principles, the magnitude of the maximum diffraction angle depends on the size of the pixel (and other factors, such as the wavelength of the light).

[0010] In an embodiment, the display device is a spatial light modulator, such as a liquid crystal on silicon ("LCOS") spatial light modulator (SLM). Light is transmitted from the LCOS to a viewing entity / system, such as a camera or an eye, over a range of diffraction angles (e.g., from zero to a maximum diffraction angle). In some embodiments, amplification techniques can be used to increase the range of available diffraction angles beyond the conventional maximum diffraction angle of the LCOS.

[0011] In embodiments, the image is a real image. In other embodiments, the image is a virtual image perceived by a human eye (or eyes). The projection system or light engine can therefore be configured so that the observer looks directly at the display device. In such embodiments, the light encoded with the hologram is transmitted directly to the eye, and no intermediate holographic reconstruction is formed in free space or on a screen or other light receiving surface between the display device and the observer. In such embodiments, the pupil of the eye can be considered to be the entrance aperture (or "entrance pupil") of the viewing system, and the retina of the eye can be considered to be the viewing plane of the viewing system. It is sometimes said that in this configuration, the lens of the eye performs the conversion of the hologram into the image.

[0012] According to well-known optical principles, the angular range of light propagating from a display device that can be observed by an eye or other observing entity / system varies with the distance between the display device and the observing entity. For example, at a viewing distance of 1 meter, only a small range of angles of light from an LCOS can propagate through the pupil of the eye to form an image on the retina at a given eye position. The angular range of light propagating from the display device that can successfully propagate through the pupil of the eye to form an image on the retina at a given eye position determines the portion of the image that is "visible" to the observer. In other words, not all portions of the image are visible from any one point on the viewing plane (e.g., any one eye position within a viewing window such as an eye box).

[0013] In some embodiments, the image perceived by the observer is a virtual image that appears upstream from the display device. That is, the observer perceives the image as being farther away from them than the display device itself. Conceptually, consider multiple different virtual image points of the virtual image. For each virtual image point, the distance from the virtual point to the observer is referred to herein as the virtual image distance. Of course, different virtual points can have different virtual image distances. Individual light rays in the bundle associated with each virtual point can take different corresponding optical paths through the display device to reach the observer. However, only some portions of the display device, and therefore only some light rays from one or more virtual points in the virtual image, may be within the user's field of view. In other words, only some light rays from some virtual points on the virtual image will propagate through the display device to the user's eye and thus be visible to the observer. Therefore, conceptually, the observer can be thought of as viewing the virtual image through a "display-sized window," which can be very small, such as 1 cm in diameter, at a relatively large distance, such as 1 meter. Furthermore, the user will view the display-sized window through their pupil, which can also be very small. Consequently, the field of view is reduced at any given time, and the specific angular range that can be seen depends heavily on the position of the eye.

[0014] The present disclosure solves the technical problem of how to increase the field of view, that is, how to increase the angular range of light propagating from a display device and that can successfully propagate through the pupil of the eye to form an image when the display device is (relatively) small and the projection distance is (relatively) large. In some embodiments, the projection distance is at least one order of magnitude larger than the diameter or width of the aperture of the display device (i.e., the size of the pixel array), for example at least two orders of magnitude. More specifically, the present disclosure solves the technical problem of how to do this with so-called direct-view holography, in which a hologram of an image is transmitted to the human eye rather than the image itself. In other words, the light received by the observer is modulated according to the hologram of the image.

[0015] Waveguides are used to extend the field of view and therefore increase the maximum propagation distance over which the full diffraction angle of the display device can be used. The use of waveguides can also increase the user's eye box laterally, thereby allowing some eye movement while still allowing the user to see the image. Therefore, waveguides can be referred to as waveguide pupil expanders. However, the inventors have found that for non-infinite virtual image distances, i.e. near-field virtual images, so-called "ghosting" occurs due to the different possible light propagation paths through the waveguide. The ghost image is a low-brightness copy of the main image. The main, highest intensity image can be referred to as the main image. Each ghost image can be referred to as a secondary image. The presence of ghosting can significantly reduce the quality of the perceived virtual image. The ghost image can make the main image appear blurry.

[0016] This disclosure describes different approaches for solving the problems caused by ghosting. Some of the solutions disclosed herein have been shown to successfully remove ghosting. Some of the solutions disclosed herein have been shown to modify / manipulate the ghost image in order to enhance or strengthen the primary / non-ghost image.

[0017] The light engine is arranged to provide spatially modulated light to an observation system having an entrance pupil. The display system includes a display device arranged to display a hologram and spatially modulate light according to the hologram. The display system also includes a holographic engine arranged to receive contribution information, the contribution information identifying contributing and non-contributing regions of the display device based on the location of the entrance pupil. The contributing regions of the display device substantially propagate light that passes through the entrance pupil at a determined location. The non-contributing regions of the display device substantially propagate light that is blocked by the entrance pupil at the determined location. The contribution information further identifies (i) at least one primary contributing region of the display device that propagates light to the observation system, which contributes to a primary image, and (ii) at least one secondary contributing region of the display device that propagates light to the observation system, which contributes to a secondary image. The holographic engine is further arranged to determine a hologram based on the at least one primary contributing region of the display device identified by the processing engine. The holographic engine is further arranged to output the hologram to the display device for display.

[0018] For the avoidance of doubt, the image formed or perceived is a holographic reconstruction of the target image. The holographic reconstruction is formed from a hologram based on the target image. In some embodiments, the hologram is determined (e.g., calculated) from the target image.

[0019] By identifying contributing and non-contributing areas of the display device, the light engine can determine which portion or portions of the display device can be effectively encoded by a hologram, for a given position of the viewing system's entrance aperture, in order to actively contribute to the formation of the primary image. This can correspond to the position of the observer's eyes at a given time, for example. Furthermore, the light engine can determine which portions of the display device are unable to transmit light through the entrance aperture and, therefore, are not worthy of being populated with hologram values. Furthermore, the light engine can distinguish between portions of the display device that actively contribute to the "primary" target image and portions that contribute to copies / duplicates, or "ghost" versions, of the primary image. Consequently, holograms can be omitted in these so-called secondary contributing areas to eliminate ghosting.

[0020] Alternatively, in a significant further refinement, the hologram displayed in the additional contributing region can be determined based on a displaced or modified position of an image point (i.e., a point within the desired image that is to be holographically reconstructed). This modified position may be referred to as a "secondary image point," but this is shorthand for the fact that it is a secondary (i.e., altered) position of the (primary) image point. In short, the modeled / calculated position of the image point can be modified (e.g., translated in the image plane) so that light propagating from this modified position via the additional contributing region on the display device will reach the desired position on the viewing plane, effectively enhancing the primary image. Thus, in this alternative approach, the hologram for the additional contributing region is determined based on a position of the image point that is different from the position used to identify the primary contributing region on the display device. The optical path length from the primary image point is typically different from the optical path length from the secondary image point to the corresponding image formed on the viewing plane. It can therefore be said that the hologram determination process associated with the additional contributing region includes translating or moving the image point used in the hologram determination process.

[0021] Thus, an intelligent and efficient light engine is provided which can be configured and operated to provide a clear, accurate image corresponding to a hologram that has been determined in a streamlined and computationally efficient manner.

[0022] A method for determining a hologram for display on a display device is provided. The method includes determining a position of an entrance pupil of an observation system arranged to observe the hologram, and identifying a contributing area and a non-contributing area of the display device, wherein the contributing area of the display device substantially transmits light that passes through the entrance pupil of the observation system at the determined position, and the non-contributing area of the display device substantially transmits light that is blocked by the entrance pupil of the observation system at the determined position. The method also includes identifying at least one primary contributing area of the display device that provides light that contributes to a primary image and at least one secondary contributing area of the display device that provides light that contributes to a secondary image; and determining the hologram based on the at least one primary contributing area of the display device.

[0023] A diffractive structure is provided that is arranged to spatially modulate light that can be converted by an observation system into a target image, wherein the diffractive structure is configured to generate a plurality of discrete light patterns, each light pattern corresponding to a different portion of the target image, wherein the shape of each discrete light pattern substantially corresponds to the shape of an entrance aperture of the observation system.

[0024] A diffractive structure is provided that is arranged to spatially modulate light that can be converted by an observation system (including a lens) into an image, wherein the diffractive structure is arranged to guide the light into a plurality of discrete light channels, wherein each light channel has a cross-sectional shape that substantially corresponds to an entrance pupil of the observation system, and each light channel substantially corresponds to a different portion of the image.

[0025] A method for determining a hologram for display on a display device and forming a virtual image perceptible from an observation plane by observing the hologram displayed on the display device through a waveguide is provided. The method comprises, for each virtual image point of the virtual image, determining the coordinates [x virtual ,y virtual ,z virtual ], determining an observation position on an observation plane, and determining a number B of light reflections within the waveguide associated with a primary image formed by the waveguide. The method further includes tracing a ray from the virtual image point to the observation plane for the "B" light reflections within the waveguide, and .... virtual ,y virtual ,z virtual ] to the observation plane with B reflection, determine the coordinates of the main ray at the display device [x LCOS (B),y LCOS (B)]. The method further includes determining the LCOS (B),y LCOS (B)] within the area defined by the display device; and by virtual ,y virtual ,z virtual ] propagates light waves toward effective pixels to determine a sub-hologram including amplitude and / or phase hologram components of the effective pixels.

[0026] The principal ray may comprise a ray that is determined (eg, calculated or simulated) to be the main or "primary" image point that travels from the virtual image point via the display device to the virtual image point on the viewing plane.

[0027] The method may further comprise combining the sub-holograms respectively calculated for two or more corresponding virtual image points in order to form the hologram.

[0028] The method may further comprise determining the position of the primary image of the virtual image point on the observation plane [x sensor ,y sensor ].

[0029] The method may further comprise, for each value of DB allowed by the waveguide, ray from [x sensor ,y sensor ] traced back to the virtual image plane z virtual Used for B+DB rebound and to determine the virtual point coordinates [x virtual (DB),y virtual (DB),z virtual ], which will be imaged to [x sensor ,y sensor ] for B+DB reflection. The method may further include determining the coordinates of the main ray at the display device [x LCOS (B+DB),y LCOS (B+DB)], used to virtual (DB),y virtual (DB),z virtual ] to the observation plane with B+DB bounces, and identify the light propagation from [x LCOS (B+DB),y LCOS The method may further comprise adding valid pixels of the display device within a second region (i.e., an additional region) defined by [x virtual (DB),y virtual (DB),z virtual ] propagates to the additional effective pixel to determine an additional sub-hologram, which includes the amplitude and / or phase hologram components of the additional effective pixel.

[0030] Aspects of the present disclosure also relate to a hologram or kinoform characterized by the guidance or routing of holographic light. Specifically, disclosed herein is a diffractive structure arranged to spatially modulate light that can be converted into an image by an observation system, wherein the diffractive structure is configured to route the light into a plurality of hologram channels, each hologram channel corresponding to a different portion of the image.

[0031] The diffractive structure may be arranged such that the hologram channels propagate from the diffractive structure at different angles.

[0032] Each hologram channel may include spatially modulated light according to the hologram corresponding to a different portion of the image.

[0033] The diffractive structures can be arranged to spatially modulate the phase of light.

[0034] The diffractive structure may be arranged to route light through a waveguide. The waveguide may be arranged for pupil expansion.

[0035] The cross-sectional shape of the light pattern that may be formed by each hologram channel may substantially correspond to the shape of the entrance aperture of the viewing system.

[0036] The hologram channels may be spatially separated or at least partially spatially separated.

[0037] Also disclosed herein is a system comprising a diffractive structure, a waveguide arranged to receive spatially modulated light from the diffractive structure, and an observation system arranged to receive the spatially modulated light via the waveguide.

[0038] The system may be arranged such that the light of each hologram channel follows a different optical path from the diffractive structure to the observation system.

[0039] Different optical paths can include different numbers of reflections within the waveguide. Different optical paths can have different lengths. Different optical paths can pass through the observation system's entrance aperture at different angles.

[0040] The waveguides can be arranged so that all hologram channels are routed through the entrance aperture of the observation system at any observation position on the observation plane.For each allowed observation position, the waveguides route each hologram channel to the observation system through only one optical path.

[0041] At least two hologram channels of the plurality of hologram channels may partially overlap at an entrance aperture of the observation system.

[0042] The diffractive structure can be a kinoform or a hologram.

[0043] The system may include a "display system" or a "light engine."

[0044] The system can be arranged such that light from each hologram channel is emitted toward the observation system from each of a plurality of different transmission points on the waveguide. For example, each transmission point may occur after a different number of corresponding reflections (or "bounces") of light within the waveguide. The system can be arranged such that light from the same hologram channel propagates from each transmission point at the same angle or range of angles. Thus, each hologram channel can be referred to as an "angle channel." In other words, each hologram channel can include spatially modulated light according to different corresponding portions of the image. Thus, each hologram channel can correspond to different corresponding image content. Furthermore, each hologram channel can have a unique corresponding "characteristic angle" (or characteristic angular range), wherein each (i.e., "each") hologram channel propagates from each (i.e., "each") transmission point on the waveguide at its characteristic angle (or characteristic angular range). Light with the same image content can be emitted from multiple different transmission points on the waveguide, all at the same angle or within the same angular range relative to the surface of the waveguide, even if the transmission points are spatially separated from one another on the waveguide.

[0045] The system can be arranged so that only one instance of each individual hologram channel reaches a single eye or a single viewing aperture or window of the observation system. This can be achieved by computationally constraining the hologram according to the single eye or other viewing aperture / window. However, the observation system typically has multiple viewing apertures, each occupying a different position and thus defining a different respective viewing position. For example, a human observer typically has two eyes that are naturally spatially separated from one another. Therefore, it has been recognized herein that, in the absence of any control, there is a risk that multiple instances of light from the same (i.e., common) hologram channel will arrive at the respective multiple viewing apertures / windows (also referred to as "entrance pupils") of the observation system at substantially the same time. If this occurs, the observer's brain—or a processor associated with a non-human observation system—will perceive both eyes (or both or each viewing aperture) as receiving light from the same image content (i.e., light from the same portion or point of the image) at the same angle, despite the fact that the eyes or viewing apertures themselves are at different viewing positions. This is counterintuitive to the observer or the observation system, as, according to well-established mathematical principles, two different viewing positions should typically receive light from a common point at different angles.

[0046] The present disclosure therefore addresses the technical problem of avoiding confusion in an observation system when observing an image projected by a system comprising a diffractive structure (and / or a display device configured to display such a diffractive structure), a waveguide arranged to receive spatially modulated light from the diffractive structure, and an observation system having an observation window with two or more observation apertures, arranged to receive the spatially modulated light via the waveguide, wherein the diffractive structure causes the light of the image to be distributed into a plurality of different channels of the spatially modulated light depending on the image content. The light of the image may be in the holographic domain. Each of the different channels may have a different respective angular direction when emitted from the waveguide.

[0047] The present disclosure provides a control device and method for controlling light emitted by a waveguide. The present disclosure also provides a system including such a control device and a method for operating the system.

[0048] The control device can be configured to selectively block or prevent the propagation of one or more spatially modulated light channels emitted by the waveguide and allow one or more corresponding other channels to propagate forward toward the observation system. Each channel can be defined by an angle or range of angles at which the light propagates relative to a reference surface or plane, such as an emitting (or transmitting) surface of the waveguide. The control device can include one or more openings or apertures, and one or more walls or barriers, to selectively transmit and block light from the waveguide. The control device can be dynamically configurable to select and / or change which portion(s) of the control device acts as an opening and which corresponding other portion(s) of the control device acts as a barrier at a given time.

[0049] The control device can enable the spatially modulated light to be divided among a plurality of entrance pupils of the viewing system, wherein each entrance pupil has a different respective viewing position, such that no two entrance pupils receive light of exactly the same image content at the same ray angle at the same time. In an embodiment, at a given time, a maximum ray angle received by a first viewing position is substantially equal to a minimum ray angle at a second eye position.

[0050] Interleaving can be used so that spatially modulated light representing a first set of image content arrives at a first viewing location at a first time, while spatially modulated light representing a second set of image content arrives at the first viewing location at a different, second time. Interleaving can be rapid; for example, the system can switch between the corresponding light representing the first and second sets of image content received at the first viewing location within a time window shorter than the typical integration time of the human eye. The first and second sets of image content can be combined to provide all the image content required by the viewing system to form a holographic reconstructed image. In some embodiments, when the first viewing location receives light representing the first set of image content, the second viewing location receives light representing the second set of image content, and vice versa. In some embodiments, when the first viewing location receives light representing the first set of image content, the second viewing location receives light representing the third set of image content, and when the first viewing location receives light representing the second set of image content, the second viewing location receives light representing the fourth set of image content. In these embodiments, the first, second, third, and fourth sets of image content can be combined to provide all the image content of a target image. Each set of image content can correspond to a different corresponding portion or region of one or more images (e.g., one image for each viewing location in a pair of viewing locations). In embodiments, the first and second sets of image content may be combined to provide all image content for a first target image at a first viewing position, and the third and fourth sets of image content may be combined to provide all image content for a second target image at a different second viewing position. Component image portions of the same image (i.e., sets of image content) may be transmitted to respective viewing positions within an integration time of the human eye and / or within an inter-frame time of a video rate sequence of images. For the avoidance of doubt, the first image provided to the first viewing position may be different from the second image provided to the second viewing position, and each image may be delivered to each viewing position using multiple holograms displayed sequentially (i.e., time-interleaved) by dividing each image into multiple sets of image content, optionally with each hologram corresponding to a set of image content for one viewing position. In some embodiments, the three-dimensional holographic image / reconstruction may be perceivable by an observation system or observer.

[0051] According to one aspect, a light engine configured to form an image visible from an observation window is provided, wherein the light engine includes a display device configured to display a hologram of the image and to spatially modulate light according to the hologram. The hologram is configured to angularly distribute the spatially modulated light of the image according to the location of the image content, such that angular channels of the spatially modulated light correspond to respective consecutive regions of the image. The light engine also includes a waveguide pupil expander configured to receive the spatially modulated light and provide a plurality of different light propagation paths for the spatially modulated light from the display device to the observation window, and a control device disposed between the waveguide and the observation window. The control device includes at least one aperture configured such that a first observation position within the observation window receives a first light channel spatially modulated by the hologram according to a first region of the image, and a second observation position within the observation window receives a second light channel spatially modulated by the hologram according to a second region of the image.

[0052] The first and second light channels may be received at different observation positions at substantially the same time or at different times, for example sequentially one after the other. The control device is arranged to ensure that the first light channel is not simultaneously passed to the second observation position, and vice versa. For example, the control device may be configured to ensure that a copy of the first light channel formed by the waveguide pupil expander and on the route of the second observation position is blocked, and vice versa. In some embodiments, only one of the multiple observation positions receives light simultaneously. For example, the first observation position may receive one or more different light channels simultaneously, while the control device ensures (for example by aperture configuration) that the second observation position does not receive any light channels. As another example, at any one time, the first observation position may receive one light channel and the second observation position may receive multiple light channels.

[0053] The observation window may be an eye box or an eye box, and the observation system may be a human observer.

[0054] The spatially modulated light of the image can be in the holographic domain. In other words, the observation system can form a visible image only when the spatially modulated light is received at the observation window. No intermediate image may be formed in the free space between the control device and the observation window or on the light receiving surface.

[0055] The system can be arranged to display two or more holograms substantially simultaneously. For example, the system can be arranged to display a first hologram constrained according to a first observation aperture of the observation system and a second hologram constrained according to a second, different observation aperture of the observation system. For example, the two or more holograms can each correspond to two or more respective views of an image to be produced, where the first view is from a perspective of the first observation aperture and the second view is from a perspective of the second, different observation aperture of the observation system.

[0056] Two or more holograms can be combined, e.g., added, into a single hologram for display. The two or more holograms can be displayed on different respective portions of a display device. The two or more holograms can be displayed on a rapidly alternating basis, e.g., at a rate faster than the typical integration time of the human eye.

[0057] The image may be a real image or a virtual image. The image may be a virtual image perceived by a viewing system or a processor associated with the viewing system and located upstream of the display device.

[0058] The first and second regions of the image may each comprise a continuous region of the image. The first and second regions of the image may be adjacent to each other. The first and second regions of the image may be adjacent to or connected to each other. The first and second regions of the image may overlap or not overlap. The first and second regions may not be adjacent to each other. The first and second regions may comprise regions of the image observed from first and second different perspectives. For example, the first region may comprise a continuous region of the image observed from a first observation position, and the second region may comprise a continuous region of the image observed from a second observation position.

[0059] A hologram can be calculated to distribute spatially modulated light of an image angularly according to the location of the image content. For example, a hologram can be calculated using a plurality of image points within the image to be formed, wherein a primary contributing region on the display device is identified through which light propagates from each of the plurality of image points to a first viewing position or a second viewing position within the viewing window. A hologram can be calculated from a plurality of sub-holograms. For example, a hologram can include a combination of a first hologram (or sub-hologram) representing the image as viewed from a first viewing position and a second hologram (or sub-hologram) representing the image as viewed from a second viewing position.

[0060] The hologram may comprise any suitable type of hologram, for example it may comprise any of the following: a Fresnel hologram, a Fourier hologram or a point cloud hologram.

[0061] During calculations, a hologram (and / or holograms or sub-holograms contributing to a hologram) can be constrained based on the entrance pupil of the observation system. For example, the entrance pupil can be located at a first observation position or a second observation position within an observation window. The location of the entrance pupil can be used to constrain the hologram at a given time. The size of the entrance pupil (e.g., the diameter of the entrance pupil) through which light can enter the observation window at a given time can be used to constrain the hologram.

[0062] Due to the hologram, the angular distribution of the image content can result in each angular channel having a different corresponding angle or angular range at which the image content is emitted from the display device and / or from a waveguide pupil expander (which may be simply referred to as a "waveguide"). The maximum angle of a first angular channel can be equal to the minimum angle of a second angular channel. The size and / or shape of the light from the angular channels can correspond to the size and / or shape of the entrance pupil (or viewing aperture) at the first viewing position or the second viewing position.

[0063] At a given time, an observation window may include more than two observation positions. The number of observation positions within an observation window may change dynamically.

[0064] The positions of the first viewing position and / or the second viewing position can be dynamically changed. In embodiments, the hologram can be checked and / or recalculated when one or the other (or both) viewing positions are changed. For example, if the hologram is constrained based on an entrance pupil located at one of the viewing positions, and the entrance pupil is moved to change the viewing position, the hologram can be recalculated.

[0065] In an embodiment, the first and second viewing positions correspond to first and second positions of the same entrance pupil (or viewing aperture).The entrance pupil may be configured to move, eg rapidly, between the first and second viewing positions.

[0066] In an embodiment, the first and second viewing positions correspond to different first and second entrance pupils (or viewing apertures) within the viewing window. For example, they may correspond to the right and left eyes of the viewer, respectively. Thus, the control device may be configured to prevent the same angular channel (and therefore, light of the same image content at the same angle) from reaching both entrance apertures simultaneously.

[0067] Adjacent angular channels of spatially modulated light may correspond to adjacent regions of an image. The angular channels may not overlap in angular space, but may be continuous. The maximum ray angle of a first angular channel may be substantially equal to the minimum ray angle of an adjacent second angular channel.

[0068] Each of the multiple different light propagation paths of spatially modulated light from the display device to the viewing window provided by the waveguide pupil expander can include multiple angular channels, each angular channel corresponding to a different respective region of the image. In an embodiment, the waveguide pupil expander and the control device can be configured such that, for each light propagation path, only one angular channel will reach the first viewing position at a given time. In an embodiment, the waveguide pupil expander and the control device can be configured such that, for each light propagation path, only one angular channel will reach the second viewing position at a given time.

[0069] The control device can be coupled to an output face or port of the waveguide pupil expander. For example, it can be optically positioned downstream of the waveguide pupil expander, separated from it by a small distance. The waveguide pupil expander and the control device can be attached to each other in any suitable manner. The waveguide pupil expander and the control device can be arranged substantially parallel to or parallel to each other.

[0070] The control device may be configured to limit one or more regions of the waveguide output face visible from the viewing window.

[0071] The waveguide pupil expander and the viewing window may be non-parallel. For example, the viewing window may include a viewing plane including the first and / or second viewing positions, and a face of the waveguide pupil expander, such as an output face that outputs different light propagation paths, may be non-parallel to the viewing plane.

[0072] The control device may include a plurality of openings (which may alternatively be referred to as "apertures" or "windows," or as substantially "open" or "transparent" portions or sections of the control device), wherein each opening provides a respective spatially modulated light channel to the first viewing position and / or the second viewing position, such that different image content is transmitted to the first and second viewing positions, respectively, substantially simultaneously. In embodiments, all image content of the image may be fully transmitted through the openings to the viewing windows substantially simultaneously, but wherein no portion of the image content is transmitted to both viewing positions substantially simultaneously.

[0073] The control device may also comprise a plurality of barriers or closing portions.The barriers may be provided with openings alternately along the transmissive face of the control device.

[0074] The size and / or position of one or more openings (and one or more barriers) within the control device can be dynamically changed. The size and / or position of the openings can be determined based on the hologram displayed at a given time. The size and / or position of the openings can be determined based on the position of the first observation position and / or the second observation position at a given time.

[0075] While the term "opening" has been used to describe the portion of the control device through which light is emitted at a given time, it should not be understood to mean a physical gap or absence of material. Rather, an "opening" may include a portion of the control device that can be dynamically controlled to open, thereby being optically transparent, or closed, thereby being optically opaque. For example, an opening may include a portion of the control device from which a shutter or cover may be removed, and / or it may include a portion of material that can be dynamically configured to be optically transparent or opaque.

[0076] The control device can be formed at least in part from an optically variable material. The light transmission characteristics of at least a portion of the control device can be varied and controlled, for example, by applying a selected voltage, or by applying light or heat thereto. The control device can include a pixelated device, wherein each pixel can be switched between optically transparent and optically opaque. For example, the pixelated device can be a liquid crystal device. Thus, the size and position of the "opening" of the control device can be determined by the number and position of pixels switched to a transparent state at a given time.

[0077] The control device itself may be referred to as a "waveguide aperture," or simply an "aperture." The control device may be configured such that each opening is switchable between an open position and a closed position, thereby providing a plurality of different control device configurations, wherein each control device configuration includes an alternating sequence of open and closed portions of the control device.

[0078] The control device may be configured to provide a first control device configuration at a first time and a second control device configuration at a second time, wherein the first control device configuration and the second control device configuration are complementary. For example, in the first control device configuration, the control device may cause light from a first portion of an image to reach a first viewing position and cause light from a second portion of the image to reach a second viewing position, and in the second control device configuration, the control device may cause light from the second portion of the image to reach the first viewing position and cause light from the first portion of the image to reach the second viewing position.

[0079] The time interval between the first time and the second time may be less than a typical integration time of the human eye.

[0080] In a first control device configuration, the control device can transmit light modulated according to first and third image regions of a first image to a first observation position, and transmit light modulated according to second and fourth image regions of a second image to a second observation position, wherein the first to fourth regions of each image are ordered continuous regions of the image.

[0081] In a second control device configuration, the control device may transmit light modulated according to the second and fourth image regions of the image to the first viewing position and transmit light modulated according to the first and third image regions of the image to the second viewing position.

[0082] The image content of each region delivered to the first viewing position can be different from the image content of each corresponding region delivered to the second viewing position. In other words, the hologram(s) can be calculated such that the image content is divided into a first set of regions for delivery to the first viewing position, and the image content is divided into a second set of regions for delivery to the second viewing position, wherein the assignment of image content to each region takes into account the perspective of the corresponding viewing position of how the viewing system will see (or perceive) the holographic reconstructed image. For example, the first viewing position may see the image translated or shifted compared to how and where the second viewing position sees the image.

[0083] According to one aspect, a method for controlling light propagation in a light engine to form an image visible from an observation window is provided, wherein the light engine includes a display device, a waveguide pupil expander, and an observation system within the observation window. The method includes displaying a hologram of an image on the display device and illuminating the display device to spatially modulate light according to the hologram. The hologram is configured to angularly distribute spatially modulated light of the image according to the location of image content, such that angular channels of the spatially modulated light correspond to corresponding continuous regions of the image. The method also includes arranging the waveguide pupil expander to receive the spatially modulated light and providing the spatially modulated light along a plurality of different light propagation paths from the display device to the observation window, and controlling propagation of the plurality of different light propagation paths using a control device disposed between the waveguide and the observation window, wherein the control device includes at least one aperture. Controlling propagation of the plurality of different light propagation paths includes configuring the control device so that a first observation position within the observation window receives a first light channel spatially modulated by the hologram according to a first region of the image, and a second observation position within the observation window receives a second light channel spatially modulated by the hologram according to a second region of the image.

[0084] The method may further comprise calculating a hologram of the image.The hologram may comprise two or more holograms or sub-holograms that are displayed simultaneously or combined to form the hologram.

[0085] The step of configuring the control device may include allowing light to pass through a first portion of the control device and preventing light from passing through a different second portion of the control device. It may also include allowing light to pass through one or more further portions of the control device and / or preventing light from passing through one or more corresponding further portions of the control device.

[0086] The waveguide pupil expander may include a plurality of transmission points, and wherein each of the plurality of different light propagation paths is transmitted from a different respective transmission point. Each transmission point may include a region or area on an output face of the waveguide pupil expander.

[0087] The method can also include: identifying, without the control device, a first transmission point from which light from the first angular channel will propagate to the first viewing position; identifying, without the control device, a different second transmission point from which light from the first angular channel will propagate to the second viewing position; and configuring the control device to block the light path from the first angular channel to the first viewing position or the light path from the first angular channel to the second viewing position at a selected time (t). For example, the method can include alternating between blocking the light path from the first angular channel to the first viewing position and blocking the light path from the first angular channel to the second viewing position during a selected time period that includes the selected time (t). The alternation can occur very quickly, for example, faster than a typical integration time of the human eye.

[0088] The method may further include identifying a plurality of contiguous regions within the image, wherein each contiguous region corresponds to light from a different respective angular channel, and configuring a control device to allow light from a first subset of the contiguous regions within the plurality of contiguous regions to be transmitted only to a first viewing location, and to allow light from a second, different subset of the contiguous regions within the plurality of contiguous regions to be transmitted only to a second viewing location. The first and second subsets may be combined to provide all image content of the image.

[0089] While the embodiments describe one-dimensional pupil expansion, the present disclosure extends to two-dimensional pupil expansion, using, for example, a first elongated waveguide that expands in a first dimension and a second waveguide that expands in a second, perpendicular dimension. According to embodiments, a control device is located after the first waveguide pupil expander. Thus, the light engine of the present disclosure may include a second waveguide pupil expander. The control device disclosed herein can be located between the first and second waveguide pupil expanders. In some embodiments, the control device is not immediately adjacent to the exit of the first waveguide. In some embodiments, the control device is located before the entrance of the second waveguide pupil expander. In other embodiments, the control device is located downstream of the second waveguide pupil expander, i.e., between the second waveguide pupil expander and the viewing window.

[0090] The transmittance of the control device (i.e., the aperture or shutter disclosed herein) can be switched in a direction substantially parallel to a line connecting the centers of the entrance pupils of the viewing system. For example, even in the case of 2D pupil expansion, the aperture or shutter can be only one-dimensional in the horizontal direction. The open or closed aperture provided by the control device can be switched in the direction of pupil expansion of the first waveguide pupil expander. That is, in the elongated dimension of the first waveguide pupil expander.

[0091] The term "hologram" is used to refer to a record that contains amplitude information or phase information, or some combination thereof, about an object. The term "holographic reconstruction" is used to refer to the optical reconstruction of an object formed by illuminating a hologram. The system disclosed herein is described as a "holographic projector" because the holographic reconstruction can be a real image and spatially separated from the hologram. The term "replay field" is used to refer to the 2D region within which the holographic reconstruction is formed and is perfectly focused. If the hologram is displayed on a spatial light modulator comprising pixels, the replay field will be repeated in the form of multiple diffraction orders, where each diffraction order is a copy of the zeroth order replay field. The zeroth order replay field generally corresponds to the preferred or primary replay field because it is the brightest replay field. Unless otherwise explicitly stated, the term "replay field" should be taken to refer to the zeroth order replay field. The term "replay plane" is used to refer to the plane in space that contains all replay fields. The terms "image," "replay image," and "image area" refer to the area of the replay field that is illuminated by the light of the holographic reconstruction. In some embodiments, an "image" may comprise discrete points, which may be referred to as "image points," or, for convenience only, as "image pixels."

[0092] The terms "encoding," "writing," and "addressing" are used to describe the process of providing a plurality of control values, each of which determines the modulation level of each pixel, to a plurality of pixels of the SLM. It can be said that the pixels of the SLM are configured to "display" a light modulation profile in response to receiving the plurality of control values. Thus, the SLM can be said to "display" a hologram, and a hologram can be considered an array of light modulation values or levels.

[0093] It has been discovered that holographic reconstructions of acceptable quality can be formed from "holograms" that contain only phase information related to the original object (i.e., the target image for reconstruction). Such holographic recordings may be referred to as phase-only holograms. The embodiments relate to phase-only holograms, but the disclosure is equally applicable to amplitude-only holography. The disclosure is not limited to any particular method for calculating holograms. By way of example only, some embodiments relate to point cloud holograms, i.e., holograms constructed using a point cloud approach. However, the disclosure is equally applicable to Fourier or Fresnel type holograms, as well as holograms calculated according to other techniques such as coherent ray tracing.

[0094] The present disclosure is also applicable to forming a holographic reconstruction using amplitude and phase information related to the original object (i.e., the target image). In some embodiments, this is achieved by using complex modulation of a so-called fully complex hologram that contains amplitude and phase information related to the original object. Because the value (gray level) assigned to each pixel of the hologram has an amplitude and a phase component, such a hologram can be referred to as a fully complex hologram. The value (gray level) assigned to each pixel can be represented as a complex number with an amplitude and a phase component. In some embodiments, a fully complex computer-generated hologram is calculated.

[0095] Reference may be made to the phase value, phase component, phase information, or simply phase of a pixel of a computer-generated hologram or spatial light modulator as shorthand for "phase delay". That is, any phase value described is actually a number (e.g., in the range of 0 to 2π) that represents the amount of phase delay provided by that pixel. For example, a pixel of a spatial light modulator described as having a phase value of π / 2 will delay the phase of received light by π / 2 radians. In some embodiments, each pixel of the spatial light modulator can operate at one of a plurality of possible modulation values (e.g., phase delay values). The term "grayscale" may be used to refer to a plurality of available modulation levels. For example, the term "grayscale" may be used for convenience to refer to a plurality of available phase levels in a phase modulator alone, even though the different phase levels do not provide different shades of gray. For convenience, the term "grayscale" may also be used to refer to a plurality of available complex modulation levels in a complex modulator.

[0096] Thus, a hologram comprises an array of gray levels, i.e. an array of light modulation values, such as phase delay values or an array of complex modulation values. A hologram is also considered a diffraction pattern, since it is a pattern that causes diffraction when displayed on a spatial light modulator and illuminated with light of a wavelength relative to (usually less than) the pixel pitch of the spatial light modulator. Reference is made herein to combining a hologram with other diffraction patterns, such as diffraction patterns used as lenses or gratings. For example, a diffraction pattern used as a grating can be combined with a hologram to shift the replay field on the replay plane, or a diffraction pattern used as a lens can be combined with a hologram to focus the holographic reconstruction on the replay plane in the near field.

[0097] Although different embodiments and groups of embodiments may be disclosed separately in the detailed description below, any feature of any embodiment or group of embodiments may be combined with any other feature or combination of features of any embodiment or group of embodiments. That is, all possible combinations and permutations of the features disclosed in this disclosure are contemplated. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] Certain embodiments are described, by way of example only, with reference to the following drawings:

[0099] Figure 1 is a schematic diagram showing a reflective SLM producing a holographic reconstruction on a screen;

[0100] Figure 2A A first iteration of an example Gerchberg-Saxton type algorithm is shown;

[0101] Figure 2B The second and subsequent iterations of an example Gerchberg-Saxton type algorithm are shown;

[0102] Figure 2CAn alternative second and subsequent iteration of an example Gerchberg-Saxton type algorithm is shown;

[0103] Figure 3 is a schematic diagram of a reflective LCOS SLM;

[0104] Figure 4 The angular content of the virtual image effectively propagating from the display device to the aperture is shown;

[0105] Figure 5A An observation system with a relatively small propagation distance is shown;

[0106] Figure 5B An observation system with a relatively large propagation distance is shown;

[0107] Figure 6A An observation system having a relatively large propagation distance is shown, which includes a waveguide for forming a virtual image at infinity;

[0108] Figure 6B Shown Figure 6A A magnified view of the optical path;

[0109] Figure 7 shows how to form a ghost image using a finite virtual image and a waveguide pupil expander;

[0110] Figure 8 A virtual image comprising a main image and two ghost images is shown;

[0111] Figures 9A to 9C An example is shown in which the entire LCOS is used to form one main image point and two corresponding ghost image points;

[0112] Figures 10A to 10C showing first, second, and third propagation paths through the waveguide, resulting in a second ghost point, a primary image point, and a first ghost point, respectively;

[0113] Figures 11A to 11C Three propagation paths and LCOS utilization associated with three different field / image points are shown;

[0114] Figure 12A showing an observation system comprising a virtual image point and an image of the virtual image point formed by the observation system and the waveguide;

[0115] Figure 12B Shown with Figure 12A The main contributing area of the LCOS associated with the example;

[0116] Figure 13 A flow chart illustrating an improved method of deriving an improved data structure according to an embodiment;

[0117] Figure 14 A flow chart illustrating a further improved method of deriving an improved data structure according to an embodiment;

[0118] Figure 15A shows an image comprising a plurality of image regions (bottom) and a corresponding hologram comprising a plurality of hologram components (top);

[0119] Figure 15B shows a hologram according to the present disclosure characterized in that holographically encoded light is routed or directed into a plurality of discrete hologram channels;

[0120] Figure 15C An optimized system is shown, arranged to deliver the optical content of each hologram channel to the eye via a different optical path;

[0121] Figure 16 A system including a waveguide that outputs multiple instances of light angle channels is shown;

[0122] Figure 17 The waveguide and observation system are shown;

[0123] Figure 18 Includes showing the angle of light and along Figure 17 a graph showing the relationship between the position of the waveguide (PWG);

[0124] Figure 19 A control device according to an embodiment is shown, which is relative to Figure 18 Graph arrangement for an image with 4 regions;

[0125] Figure 20 The target image divided into 4 regions is shown;

[0126] Figure 21 A portion of a control device according to an embodiment is shown, which is relative to Figure 19 Graph arrangement of

[0127] Figure 22A A display system including a control device according to an embodiment is shown;

[0128] Figure 22B Shown Figure 22A The image content received by the left eye;

[0129] Figure 22C Shown Figure 22B The image content received by the right eye;

[0130] Figure 23A shows the desired image to be received by the left eye, divided into a plurality of regions;

[0131] Figure 23B shows the desired image to be received by the right eye, divided into a plurality of regions;

[0132] Figure 23C shows an image region received by an observer in a first phase of a control device according to an embodiment;

[0133] Figure 23D shows an image region received by an observer during a second phase of a control device according to an embodiment;

[0134] Figure 24A A display system including a control device in a first phase is shown;

[0135] Figure 24B Shown in Figure 24A an image area received by the right eye during the first phase;

[0136] Figure 24C Shown in Figure 24A an image area received by the left eye during the first phase;

[0137] Figure 25A Shown Figure 24A Display system, but the control device is in the second phase;

[0138] Figure 25B Shown in Figure 25A an image area received by the right eye during the second phase;

[0139] Figure 25C Shown in Figure 25A an image area received by the left eye during the second phase;

[0140] Figure 26 shows a light engine providing one-dimensional pupil expansion, and ray tracing through a finite-sized pupil of an observer, according to an embodiment;

[0141] Figure 27 Shows the Figure 26 Analysis of binocular crosstalk in an embodiment of ; and

[0142] Figure 28 、 29 30 and 30 respectively show first to third shutter phases of a control device according to an embodiment.

[0143] The same reference numbers will be used throughout the drawings to refer to the same or like parts. DETAILED DESCRIPTION

[0144] The present invention is not limited to the embodiments described below, but extends to the full scope of the appended claims. That is, the present invention can be embodied in different forms and should not be construed as limited to the described embodiments, which are set forth for illustrative purposes.

[0145] Unless otherwise stated, terms in the singular may include plural forms.

[0146] A structure described as being formed on / under or above / below another structure should be construed to include a case where the structures are in contact with each other and, further, a case where a third structure is provided therebetween.

[0147] When describing a temporal relationship, for example, when the temporal order of events is described as "after," "followed," "next," "before," etc., the present disclosure should be considered to include both consecutive and non-consecutive events unless otherwise specified. For example, unless terms such as "just," "immediately," or "directly" are used, the description should be considered to include non-consecutive situations.

[0148] Although the terms "first," "second," etc. may be used herein to describe various elements, these elements are not limited by these terms. These terms are only used to distinguish between the various elements. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element without departing from the scope of the appended claims.

[0149] The features of different embodiments may be coupled or combined with each other in part or in whole, and may interoperate with each other in different ways. Some embodiments may be performed independently of each other, or may be performed together in a mutually dependent relationship.

[0150] Optical configuration

[0151] Figure 1 An embodiment is shown in which a computer-generated hologram is encoded on a single spatial light modulator. The computer-generated hologram is a Fourier transform of the object used for reconstruction. It should be understood that this is merely an example and other methods for computer-generated holograms are also contemplated in this disclosure. Thus, it can be said that the hologram is a Fourier domain or frequency domain or spectral domain representation of the object. In this embodiment, the spatial light modulator is a reflective liquid crystal on silicon "LCOS" device. The hologram is encoded on the spatial light modulator and a holographic reconstruction is formed at a replay field, for example, a light receiving surface such as a screen or diffuser.

[0152] A light source 110, such as a laser or laser diode, is arranged to illuminate the SLM 140 via a collimating lens 111. The collimating lens causes a substantially planar wavefront of light to be incident on the SLM. Figure 1In embodiments, the direction of the wavefront is off-normal (e.g., two or three degrees from a plane that is truly normal to the transparent layer). However, in other embodiments, a substantially planar wavefront is provided at normal incidence, and a beam splitter arrangement is used to separate the input and output optical paths. Figure 1 In the embodiment shown, the arrangement is such that light from the light source reflects from the mirrored back surface of the SLM and interacts with the light modulating layer to form an exit wavefront 112. The exit wavefront 112 is applied to an optical device including a Fourier transform lens 120, the focus of which is located at a screen 125. More specifically, the Fourier transform lens 120 receives the modulated light beam from the SLM 140 and performs a frequency-to-space transform to produce a holographic reconstruction at the screen 125.

[0153] It's important to note that in this type of hologram, every pixel of the hologram contributes to the entire reconstruction. There is no one-to-one correlation between a specific point on the replay field (or image pixel) and a specific light modulation element (or hologram pixel). In other words, the modulated light leaving the light modulation layer is distributed across the entire replay field.

[0154] In these embodiments, the position of the holographic reconstruction in space is determined by the refractive (focusing) power of the Fourier transform lens. Figure 1 In the illustrated embodiment, the Fourier transform lens is a physical lens. That is, the Fourier transform lens is an optical Fourier transform lens and optically performs a Fourier transform. Any lens can function as a Fourier transform lens, but the accuracy of the Fourier transform it performs will be limited by the lens's performance. Those skilled in the art understand how to use lenses to perform an optical Fourier transform.

[0155] Hologram computing

[0156] In some embodiments, the computer-generated hologram is a Fourier transform hologram, or simply a Fourier hologram or Fourier-based hologram, in which the image is reconstructed in the far field by utilizing the Fourier transform properties of a positive lens. The Fourier hologram is calculated by Fourier transforming the desired light field in the replay plane back to the lens plane. The computer-generated Fourier hologram can be calculated using the Fourier transform.

[0157] Algorithms such as the Gerchberg-Saxton algorithm can be used to calculate Fourier transform holograms. Furthermore, the Gerchberg-Saxton algorithm can be used to calculate a hologram in the Fourier domain (i.e., a Fourier transform hologram) from amplitude-only information in the spatial domain (e.g., a photograph). Phase information about the object is effectively "retrieved" from amplitude-only information in the spatial domain. In some embodiments, computer-generated holograms are calculated from amplitude-only information using the Gerchberg-Saxton algorithm or a variation thereof.

[0158] The Gerchberg-Saxton algorithm considers that when the intensity cross section I of the beam in planes A and B is known, A (x,y) and I B (x,y) and I A (x,y) and I B (x,y) is related by a single Fourier transform. For a given intensity cross section, the phase distribution approximation Ψ in planes A and B is obtained separately. A (x,y) and Ψ B (x,y). The Gerchberg-Saxton algorithm finds the solution to this problem by following an iterative process. More specifically, the Gerchberg-Saxton algorithm iteratively applies spatial and spectral constraints while repeatedly transferring representations of I between the spatial domain and the Fourier (spectral or frequency) domain. A (x,y) and I B A data set (amplitude and phase) of (x, y) is obtained. A corresponding computer-generated hologram in the spectral domain is obtained by at least one iteration of the algorithm. The algorithm is convergent and is arranged to produce a hologram representing the input image. The hologram can be an amplitude-only hologram, a phase-only hologram, or a full complex hologram.

[0159] In some embodiments, a phase-only hologram is calculated using an algorithm based on the Gerchberg-Saxton algorithm, such as the algorithm described in UK Patent 2498170 or 2501112, the entire contents of which are incorporated herein by reference. However, the embodiments disclosed herein describe calculating a phase-only hologram by way of example only. In these embodiments, the Gerchberg-Saxton algorithm retrieves the phase information Ψ[u,v] of the Fourier transform of a data set, which produces known amplitude information T[x,y], where the amplitude information T[x,y] represents a target image (e.g., a photograph). Since amplitude and phase are inherently combined in the Fourier transform, the transformed amplitude and phase contain useful information about the accuracy of the calculated data set. Therefore, the algorithm can be used iteratively with feedback of the amplitude and phase information. However, in these embodiments, only the phase information Ψ[u,v] is used as a hologram to form a holographic representation of the target image at the image plane. A hologram is a data set (e.g., a 2D array) of phase values.

[0160] In other embodiments, an algorithm based on the Gerchberg-Saxton algorithm is used to calculate a full complex hologram. A full complex hologram is a hologram having an amplitude component and a phase component. A hologram is a data set (e.g., a 2D array) comprising an array of complex data values, where each complex data value comprises an amplitude component and a phase component.

[0161] In some embodiments, the algorithm processes complex data and the Fourier transform is a complex Fourier transform. The complex data can be viewed as comprising (i) a real component and an imaginary component, or (ii) an amplitude component and a phase component. In some embodiments, the two components of the complex data are processed differently at various stages of the algorithm.

[0162] Figure 2A A first iteration of an algorithm for calculating a phase-only hologram according to some embodiments is shown. The input to the algorithm is an input image 210 comprising a 2D array of pixels or data values, wherein each pixel or data value is an amplitude or magnitude value. That is, each pixel or data value of the input image 210 does not have a phase component. Therefore, the input image 210 can be regarded as an amplitude-only or amplitude-only or intensity-only distribution. An example of such an input image 210 is a frame of a photograph or a video comprising a time sequence of frames. The first iteration of the algorithm begins with a data formation step 202A, which comprises assigning a random phase value to each pixel of the input image using a random phase distribution (or random phase seed) 230 to form a starting complex data set, wherein each data element of the data set comprises an amplitude and a phase. It can be said that the starting complex data set represents the input image in the spatial domain.

[0163] First processing block 250 receives an initial complex data set and performs a complex Fourier transform to form a Fourier-transformed complex data set. Second processing block 253 receives the Fourier-transformed complex data set and outputs hologram 280A. In some embodiments, hologram 280A is a phase-only hologram. In these embodiments, second processing block 253 quantizes each phase value and sets each amplitude value to 1 to form hologram 280A. Each phase value is quantized according to a phase level that can be represented at a pixel of a spatial light modulator that will be used to "display" the phase-only hologram. For example, if each pixel of the spatial light modulator provides 256 different phase levels, each phase value of the hologram is quantized to one of the 256 possible phase levels. Hologram 280A is a phase-only Fourier hologram representing the input image. In other embodiments, hologram 280A is a full complex hologram, comprising an array of complex data values (each including an amplitude component and a phase component) derived from the received Fourier-transformed complex data set. In some embodiments, second processing block 253 constrains each complex data value to one of a plurality of permissible complex modulation levels to form hologram 280A. The constraining step may include setting each complex data value to the closest permissible complex modulation level in the complex plane. Hologram 280A can be said to represent the input image in the spectral, Fourier, or frequency domain. In some embodiments, the algorithm stops at this point.

[0164] However, in other embodiments, the algorithm continues as Figure 2AIn other words, following Figure 2A The steps indicated by dashed arrows are optional (ie, not essential for all embodiments).

[0165] The third processing block 256 receives the modified complex data set from the second processing block 253 and performs an inverse Fourier transform to form an inverse Fourier transformed complex data set. The inverse Fourier transformed complex data set can be said to represent the input image in the spatial domain.

[0166] Fourth processing block 259 receives the inverse Fourier transformed complex data set and extracts the distribution of amplitude values 211A and the distribution of phase values 213A. Optionally, fourth processing block 259 evaluates the distribution of amplitude values 211A. Specifically, fourth processing block 259 may compare the distribution of amplitude values 211A of the inverse Fourier transformed complex data set with input image 510, which itself is a distribution of amplitude values. If the difference between the distribution of amplitude values 211A and input image 210 is sufficiently small, fourth processing block 259 may determine that hologram 280A is acceptable. In other words, if the difference between the distribution of amplitude values 211A and input image 210 is sufficiently small, fourth processing block 259 may determine that hologram 280A is a sufficiently accurate representation of input image 210. In some embodiments, the distribution of phase values 213A of the inverse Fourier transformed complex data set is ignored for comparison purposes. It will be appreciated that any number of different methods may be employed to compare the distribution of amplitude values 211A to the input image 210, and the present disclosure is not limited to any particular method. In some embodiments, a mean square error is calculated, and if the mean square error is less than a threshold, the hologram 280A is deemed acceptable. If the fourth processing block 259 determines that the hologram 280A is unacceptable, further iterations of the algorithm may be performed. However, this comparison step is not required, and in other embodiments, the number of iterations of the algorithm performed is predetermined, preset, or user-defined.

[0167] Figure 2B 2 represents the second iteration of the algorithm and any further iterations of the algorithm. The distribution of phase values 213A from the previous iteration is fed back through the processing blocks of the algorithm. The distribution of amplitude values 211A is rejected in favor of the distribution of amplitude values of the input image 210. In the first iteration, the data forming step 202A forms a first complex data set by combining the distribution of amplitude values of the input image 210 with the random phase distribution 230. However, in the second and subsequent iterations, the data forming step 202B includes forming a complex data set by combining (i) the distribution of phase values 213A from the previous iteration of the algorithm with (ii) the distribution of amplitude values of the input image 210.

[0168] Then, with reference to Figure 2AThe same method as described is handled by Figure 2B The complex data set formed in step 202B is used to form the second iterative hologram 280B. Therefore, the description of the process is not repeated here. When the second iterative hologram 280B has been calculated, the algorithm can stop. However, any number of further iterations of the algorithm can be performed. It will be understood that the third processing block 256 is only required when the fourth processing block 259 is required or further iterations are required. The output hologram 280B generally gets better with each iteration. However, in practice, a point is often reached where no measurable improvement can be observed, or the positive benefits of performing further iterations are offset by the negative effects of the additional processing time. Therefore, the algorithm is described as iterative and convergent.

[0169] Figure 2C 213A for the previous iteration is fed back through a processing block of the algorithm. The distribution of amplitude values 211A is rejected in favor of an alternative distribution of amplitude values. In this alternative embodiment, the alternative distribution of amplitude values is derived from the distribution of amplitude values 211 for the previous iteration. Specifically, processing block 258 subtracts the distribution of amplitude values of input image 210 from the distribution of amplitude values 211 for the previous iteration, scales the difference by a gain factor α, and subtracts the scaled difference from input image 210. This is mathematically expressed by the following equation, where the subscript text and number represent the number of iterations:

[0170] R n+1 [x,y]=F'{exp(iψ n [u,v])}

[0171] ψ n [u,v]=∠F{η·exp(i∠R n [x,y])}

[0172] η=T[x,y]-α(|R n [x,y]|-T[x,y])

[0173] in:

[0174] F' is the inverse Fourier transform;

[0175] F is the forward Fourier transform;

[0176] R[x,y] is the complex data set output by the third processing block 256;

[0177] T[x,y] is the input or target image;

[0178] ∠ is the phase component;

[0179] Ψ is the phase-only hologram 280B;

[0180] η is the new distribution of amplitude values 211B; and

[0181] α is the gain factor.

[0182] The gain factor α may be fixed or variable. In some embodiments, the gain factor α is determined based on the size and rate of the input target image data. In some embodiments, the gain factor α depends on the number of iterations. In some embodiments, the gain factor α is only a function of the number of iterations.

[0183] In all other respects, Figure 2C Examples and Figure 2A and Figure 2B It can be said that only the phase hologram Ψ(u,v) comprises the phase distribution in the frequency or Fourier domain.

[0184] In some embodiments, a spatial light modulator is used to perform a Fourier transform. Specifically, the hologram data is combined with second data that provides optical power. That is, the data written to the spatial light modulation includes hologram data representing the object and lens data representing the lens. When displayed on the spatial light modulator and illuminated with light, the lens data simulates a physical lens - that is, it focuses light in the same way as a corresponding physical optical element. Therefore, the lens data provides optical power or focusing power. In these embodiments, the lens data can be omitted. Figure 1A physical Fourier transform lens 120 is shown. It is known how to calculate data representing a lens. This data representing a lens can be referred to as a software lens. For example, a phase-only lens can be formed by calculating the phase delay caused by each point of the lens due to its refractive index and spatially varying optical path length. For example, the optical path length at the center of a convex lens is greater than the optical path length at the edge of the lens. An amplitude-only lens can be formed using a Fresnel zone plate. In the field of computer-generated holography, it is also known how to combine data representing a lens with a hologram, thereby performing a Fourier transform of the hologram without requiring a physical Fourier lens. In some embodiments, the lensing data is combined with the hologram via a simple addition, such as simple vector addition. In some embodiments, a physical lens is used in conjunction with a software lens to perform the Fourier transform. Alternatively, in other embodiments, the Fourier transform lens is omitted entirely, allowing holographic reconstruction to occur in the far field. In further embodiments, the hologram can be combined in the same manner with grating data—data arranged to perform a grating function, such as image steering. Again, it is known in the art how to calculate such data. For example, a phase-only grating can be formed by modeling the phase delay caused by each point on the surface of the blazed grating. An amplitude-only grating can simply be superimposed with an amplitude-only hologram to provide angular steering for the holographic reconstruction. The second data providing lensing and / or steering can be referred to as a light processing function or light processing pattern to distinguish it from the hologram data, which can be referred to as an image forming function or image forming pattern.

[0185] In some embodiments, the Fourier transform is performed jointly by a physical Fourier transform lens and a software lens. That is, some of the optical power that contributes to the Fourier transform is provided by the software lens, while the remaining optical power that contributes to the Fourier transform is provided by one or more physical optical devices.

[0186] In some embodiments, a real-time engine is provided that uses an algorithm to receive image data and calculate holograms in real time. In some embodiments, the image data is a video comprising a sequence of image frames. In other embodiments, the holograms are pre-calculated, stored in computer memory, and recalled as needed for display on the SLM. That is, in some embodiments, a repository of pre-determined holograms is provided.

[0187] The embodiments relate to Fourier holography and Gerchberg-Saxton type algorithms by way of example only. The present disclosure is equally applicable to Fresnel holography and Fresnel holograms that can be calculated by similar methods. The present disclosure is also applicable to holograms calculated by other techniques, such as those based on point cloud methods. As will be seen, the subsequent figures herein are described as including point cloud methods for hologram calculation. However, other methods of hologram calculation may be used instead, including those described above with reference to Figures 2A to 2CDescribe the Fourier method.

[0188] Optical Modulation

[0189] A spatial light modulator can be used to display diffraction patterns, including computer-generated holograms. If the hologram is a phase-only hologram, a spatial light modulator that modulates the phase is required. If the hologram is a fully complex hologram, a spatial light modulator that modulates both phase and amplitude can be used, or a first spatial light modulator that modulates the phase and a second spatial light modulator that modulates the amplitude can be used.

[0190] In some embodiments, the light modulation elements (i.e., pixels) of the spatial light modulator are cells comprising liquid crystals. That is, in some embodiments, the spatial light modulator is a liquid crystal device in which the optically active component is a liquid crystal. Each liquid crystal cell is configured to selectively provide multiple light modulation levels. That is, each liquid crystal cell is configured to operate at a light modulation level selected from a plurality of possible light modulation levels at any time. Each liquid crystal cell can be dynamically reconfigured to a light modulation level that is different from the plurality of light modulation levels. In some embodiments, the spatial light modulator is a reflective liquid crystal on silicon (LCOS) spatial light modulator, but the present disclosure is not limited to this type of spatial light modulator.

[0191] LCOS devices provide a dense array of light modulating elements or pixels within a small aperture (e.g., a few centimeters wide). Pixels are typically around 10 microns or smaller, which results in a diffraction angle of a few degrees, meaning the optical system can be compact. It is much easier to fully illuminate the small aperture of an LCOS SLM than the large apertures of other liquid crystal devices. LCOS devices are typically reflective, which means that the circuitry that drives the LCOS SLM pixels can be buried under the reflective surface. This results in a higher aperture ratio. In other words, the pixels are densely packed, meaning there is almost no dead space between pixels. This is advantageous because it reduces optical noise in the playback field. LCOS SLMs use a silicon backplane, which has the advantage that the pixels are optically flat. This is particularly important for phase modulation devices.

[0192] The following is just an example, Figure 3 A suitable LCOS SLM is described below. An LCOS device is formed using a single crystal silicon substrate 302. It has a 2D array of square planar aluminum electrodes 301, separated by gaps 301a, arranged on the upper surface of the substrate. Each electrode 301 can be addressed by circuitry 302a embedded in the substrate 302. Each electrode forms its own plane mirror. An orientation layer 303 is disposed on the electrode array, and a liquid crystal layer 304 is disposed on the orientation layer 303. A second orientation layer 305 is disposed on a planar transparent layer 306, for example, made of glass. A single transparent electrode 307, for example, made of ITO, is disposed between the transparent layer 306 and the second orientation layer 305.

[0193] Each square electrode 301, together with the footprint of the transparent electrode 307 and the intervening liquid crystal material, defines a controllable phase-modulating element 308, often referred to as a pixel. The effective pixel area, or fill factor, is the percentage of the total pixel that is optically active, taking into account the spaces between pixels 301a. By controlling the voltage applied to each electrode 301 relative to the transparent electrode 307, the properties of the liquid crystal material of each phase-modulating element can be varied, thereby providing variable retardation to light incident thereon. The effect is to provide only phase modulation of the wavefront, i.e., no amplitude effects occur.

[0194] The described LCOS SLM outputs spatially modulated light in a reflective manner. A reflective LCOS SLM has the advantage that the signal lines, grating lines and transistors are located below the mirror surface, which results in a high fill factor (typically greater than 90%) and high resolution. Another advantage of using a reflective LCOS spatial light modulator is that the thickness of the liquid crystal layer can be half that required when using a transmissive device. This greatly increases the switching speed of the liquid crystal (a key advantage for projecting moving video images). However, the teachings of the present disclosure can also be implemented using a transmissive LCOS SLM.

[0195] Image projection using small display devices and long viewing distances

[0196] The present disclosure relates to image projection in which the separation between a display device and an observer is substantially greater than the size of the display device. The viewing distance (i.e., the distance between the observer and the display device) can be at least one order of magnitude greater than the size of the display device. The viewing distance can be at least two orders of magnitude greater than the size of the display device. For example, the pixel area of the display device can be 10 mm x 10 mm, and the viewing distance can be 1 meter. The image projected by the system is formed on a display plane that is spatially separated from the display device.

[0197] According to the present disclosure, an image is formed by holographic projection. A hologram is displayed on a display device. The hologram is illuminated by a light source (not shown), and the image is perceived on a display plane that is spatially separated from the hologram. The image can be real or virtual. For the purposes of the following explanation, it is helpful to consider a virtual image formed upstream of the display device. That is, appearing behind the display device. However, it is not important that the image is a virtual image, and the present disclosure applies equally to real images formed between the display device and the observation system.

[0198] The display device includes pixels that display the hologram. The pixel structure of the display device is diffractive. Therefore, the size of the holographic image is determined by the diffraction law. Figure 4 Interpret the results showing the diffraction properties of the device.

[0199] Figure 4A pixelated display device 402 is shown, which is arranged to display a hologram forming a virtual image 401 upstream of the display device 402. The diffraction angle q of the display device determines the size of the virtual image 401. The virtual image 401, the display device 402 and the observation system 405 are arranged on an optical axis Ax.

[0200] Observation system 405 has an entrance aperture 404 and an observation plane 406. Observation system 405 may be a human eye. Thus, entrance aperture 404 may be the pupil of the eye, and observation plane 406 may be the retina of the eye.

[0201] The light propagating between the display device 402 and the viewing system 405 is modulated by the hologram of the image (not the image itself). Figure 4 It shows how the hologram divides the virtual image content by angle. Each illustrated bundle of rays relates to a different portion of the virtual image 401. More specifically, the light in each bundle of rays is encoded by the hologram with information about a portion of the virtual image. Figure 4 Five example light beams are shown, each characterized by a respective angle relative to the optical axis Ax, and each representing a respective portion of the virtual image. In this example, one of the light beams passes through the pupil 404, while the other four light beams are blocked by the pupil 404. Likewise, the five different light beams correspond to five different portions of the virtual image 401. The complete image content of the virtual image is effectively divided by angle. The light beam propagating along the optical axis Ax carries the central portion of the image information, i.e., information relating to the center of the image. The other light beams carry other portions of the image information. The two light beams shown at the ends of the light cone carry edge portions of the image information. A consequence of the angular division of the image information is that not all of the image content can pass through the entrance aperture 404 of the observation system at a given observation position. In other words, not all of the image content is received by the eye. Figure 4 In the example of FIG4 , only one of the five beams shown passes through pupil 404 at any viewing position. The reader will understand that five beams are shown by way of example only, and the described process is not limited to dividing the image information of a virtual image into only five beams.

[0202] In this example, the center portion of the image information is received by the eye. The edges of the image information are blocked by the eye's pupil. The reader will understand that if the observer moves up or down, the eye may receive a different beam of light; for example, the center portion of the image information may be blocked. Therefore, the observer sees only a portion of the entire image. The remaining image information is blocked by the entrance pupil. The observer's field of view is severely limited because they are effectively viewing the image through the small aperture of the display device itself.

[0203] In summary, light propagates from the display device within a range of diffraction angles. At a 1m viewing distance, for a given eye position, only a small range of angles from the display device can propagate through the pupil of the eye to form an image on the retina. The visible portion of the virtual image is only those that fall within the Figure 4 The portion shown passing through the small angular range of the entrance aperture. Therefore, the field of view is very small, and the exact angular range depends heavily on eye position.

[0204] refer to Figure 4 The problems of small field of view and sensitivity to eye position explained are a consequence of the large viewing distance and small aperture of the display device. The importance of viewing distance is further explained with reference to Figures 5 to 7 .

[0205] Figure 5A A display device 502 is shown arranged to display a hologram and to propagate light modulated according to the hologram to an observation system comprising an entrance aperture 504 and an observation plane 506. The virtual image 501 is at infinity, so light rays traced between the virtual image and the display device are collimated. Figure 5A The lower part of the diagram shows an enlarged view of the viewing system. The diagram is schematic and therefore does not show the physiological details of the eye. In practice, there is of course a light source ( Figure 5A not shown).

[0206] Figure 5A Only those light rays that can propagate through the aperture 504 are shown; any other light rays that cannot pass through the aperture 504 are omitted. However, it will be understood that in practice, those other light rays will also propagate from the display device 502. Figure 5A In the optimal position, the field of view is equal to the diffraction angle of the display device. Interestingly, different image points on the retina are formed by light propagating from different areas on the display device 502, for example, the area closest to the retina. Figure 5A The top image point is formed only by light propagating from the lower portion of the display device. Light propagating from other areas of the display device does not contribute to this image point.

[0207] Figure 5B It shows what happens when the viewing distance increases.

[0208] In more detail, Figure 5BA display device 502' is shown, arranged to display a hologram and to propagate light modulated according to the hologram to an observation system comprising an entrance aperture 504' and an observation plane 506'. The virtual image 501' is at infinity, so the light between the virtual image 501' and the display device 502' is collimated. Figure 5B The lower part of the diagram shows an enlarged view of the viewing system. The diagram is schematic and therefore does not show the physiological details of the eye. In practice, there is of course a light source ( Figure 5B not shown).

[0209] Figure 5B Only those rays that can propagate through aperture 504' are shown. Figure 5B At larger viewing distances, some light beams are blocked by entrance aperture 504'. Specifically, light beams associated with the edge portions of the virtual image are blocked by entrance pupil 504'. Consequently, the entire virtual image is invisible, and the visible portion of the virtual image is heavily dependent on eye position. Therefore, due to the small size of the display device, a large distance between the display device and the viewing system is problematic.

[0210] Figure 6A An improved system is shown comprising a display device 602, with light encoded with a hologram displayed on the display device 602 being transmitted towards a viewing system comprising an entrance aperture 604 and an viewing plane 606. In practice, there is of course a light source (not shown) arranged to illuminate the display device 602. The improved system also comprises a waveguide 608 located between the display device 602 and the entrance aperture 604. Figure 6A The lower part of FIG shows an enlarged view of the entrance pupil 604 and the viewing plane 606. The figure is schematic and therefore does not show physiological details of the eye.

[0211] Figure 6A The observation distance and Figure 5B However, in Figure 5B The blocked beam is effectively restored by waveguide 608, allowing the viewing system to receive complete image information - despite the longer viewing distance.

[0212] The presence of waveguide 608 enables all angular content from display device 602 to be received by the eye, even at this relatively large projection distance. This is because waveguide 608 acts as a pupil expander in a well-known manner and will therefore only be briefly described here.

[0213] Briefly, waveguide 608 comprises a substantially elongated structure. In this example, it comprises an optical plate of refractive material, but other types of waveguides are well known and may be used. Waveguide 608 is positioned to intersect the cone of light projected from display device 602, for example, at an oblique angle. The size, position, and orientation of waveguide 608 are configured to ensure that light from each of the five light beams within the cone of light enters waveguide 608. Light from the cone of light enters waveguide 608 via a first planar surface 610 of waveguide 608 (located closest to display device 602) and is guided at least partially along the length of waveguide 608 before being emitted via a second planar surface 612 of waveguide 608, which is substantially opposite first surface 610 (located closest to the eye). As will be readily appreciated, second planar surface 612 is partially reflective and partially transmissive. In other words, as each light ray propagates within the waveguide 608 from the first planar surface 610 of the waveguide 608 to the second planar surface 612, some of the light will be transmitted out of the waveguide 608, and some of the light will be reflected by the second planar surface 612 back to the first planar surface 610. The first planar surface 610 is reflective, so that all light that strikes it from within the waveguide 608 will be reflected back to the second planar surface 612. Thus, some light may simply be refracted between the two planar surfaces 610, 612 of the waveguide 608 before being transmitted, while other light may be reflected and, therefore, may experience one or more reflections (or "bounces") between the planar surfaces 610, 612 of the waveguide 608 before being transmitted. Thus, the net effect of the waveguide 608 is that the transmission of light is effectively extended to multiple locations on the second planar surface 612 of the waveguide 608. Consequently, all angular content output by the display device 602 can appear at a greater number of locations on the display plane (and at a greater number of locations on the aperture plane) than would be the case without the waveguide 608. This means that light from every ray bundle can enter entrance aperture 604 and contribute to the image formed by viewing plane 606, despite the relatively large projection distance. In other words, the eye can receive content from all angles of display device 602. Therefore, the full diffraction angle of display device 602 is utilized, and the viewing window for the user is maximized. This, in turn, means that all light rays contribute to the perceived virtual image 601.

[0214] Figure 6B The individual optical paths of each of the five ray bundles contributing to five respective image points within the virtual image 601 are shown. Figure 6A608 , are labeled R1 to R5 from top to bottom. As can be seen therein, light from each of R1 and R2 is simply refracted and then transmitted by waveguide 608. On the other hand, light from R4 experiences a single bounce before being transmitted. Light from R3 includes some light from a corresponding first portion of display device 602 that is simply refracted by waveguide 608 before being transmitted, and some light from a different second corresponding portion of display device 602 that experiences a single bounce before being transmitted. Similarly, light from R5 includes some light from a corresponding first portion of display device 602 that experiences a single bounce before being transmitted, and some light from a different second corresponding portion of display device 602 that experiences two bounces before being transmitted. For each of R3 and R5, two different portions of the LCOS propagate light corresponding to portions of the virtual image.

[0215] The present inventors have recognized that, at least in some applications, it is preferable that the virtual image distance (i.e., the distance from the observer to the virtual image) be finite, as opposed to forming the virtual image at infinity. In certain applications, there will be a preferred virtual image distance at which it is desirable or necessary for virtual image content to appear. This may be the case, for example, in a heads-up display, such as in an automotive setting, where virtual image content is to be superimposed on real content viewed by an observer through the vehicle's windshield. For example, a desired virtual image distance may include virtual image content formed a few meters in front of the observer's vehicle or windshield, such as 3 meters or 5 meters.

[0216] Figure 7 The upper part of FIG6 shows a system comprising a display device 702 that propagates light 703, which has been encoded with (i.e., modulated according to) a hologram displayed on the display device 702, towards an eye comprising an entrance aperture 704 and an observation plane 706. There is a light source (not shown) arranged to illuminate the display device 702. The system also comprises a waveguide 708 located between the display device 702 and the entrance aperture 704, acting as a pupil expander, as described in detail above with respect to FIG6a. Figure 7 The middle portion shows a magnified view of the entrance aperture 704 and the viewing plane 706, Figure 7 The bottom portion of FIG shows a further magnified view of viewing plane 706. This figure is schematic and therefore does not illustrate the physiological details of the eye. In this arrangement, the eye perceives virtual image 701 as being located at a finite distance upstream of display device 702. The light rays between virtual image 701 and display device 702 diverge because the virtual image distance is finite.

[0217] According to the above Figure 6A , Figure 7The presence of the medium waveguide 708 effectively enables the full diffraction angle of the display device 702 to be accessed at a relatively large projection distance, so that the user at the viewing position shown can see the complete image content.

[0218] However, a further technical problem is introduced. For some light bundles, the different optical paths of light from different parts of the display device 702 may result in these light bundles each forming multiple image points on the retina 706 when forming a virtual image at a finite virtual image distance. Figure 7 ' is shown with respect to the beams labeled R3' and R5' in FIG. The additional image points formed, which are ancillary to the primary image point at a given point within the virtual image, can be referred to as "ghost image points," which together form a "ghost image" or simply "ghost." As those skilled in the art of image formation will appreciate, the formation of ghost images can lead to blurring of the virtual image and an overall decrease in perceived quality from the observer's perspective. This is particularly true if the "ghost image" partially overlaps the "primary" image.

[0219] Figure 8 An example of a virtual image of the numbers "5" and "9" is shown, which is created using a method similar to Figure 7 The viewing system shown creates, in addition to the main image, ghost images. The main image can be seen as the brightest central image of each digit, with ghost images to the left and right. Figure 8 In the example of , when the viewing distance is greater than that for '5', '9' is formed, and thus the blur of '9' is more noticeable.

[0220] The inventors have solved the problem of ghosting. The inventors have recognized that it is desirable to provide an observation system in which a virtual image can be formed at a limited virtual image distance, the virtual image including all angular image content output by a display device, and in which the formation of ghost images is reduced or eliminated. Furthermore, the inventors have found that as the size of the observation aperture in a conventional observation system increases, the risk of forming ghost image points increases because the aperture can allow additional light to enter, which may form additional image points on the display plane. Therefore, it is desirable to provide an improved observation system that can accommodate apertures of different sizes while still reducing or eliminating the formation of ghost images. The solution provided by the inventors, detailed below, is applicable to a range of different sizes and arrangements of apertures, waveguides and display devices, and can be applied to different propagation distances for which one or more ghost images may traditionally be formed.

[0221] In summary, the inventors have recognized that it is possible to provide a light engine for generating a hologram that effectively identifies one or more regions of a display device that, in conventional arrangements, would produce one or more ghost images, wherein the hologram is obtained so as to control the contribution from these one or more regions of the display device to avoid or reduce the formation of ghost image points when the hologram is displayed on the display device and illuminated. The inventors have also recognized that it is possible to provide a hologram engine for providing such a hologram even when the projection distance in the viewing system is relatively large and the display device and / or viewing aperture is relatively small, and to provide an improved viewing system for displaying and illuminating the improved hologram, for forming an improved image.

[0222] The inventors have recognized that by having a waveguide (such as the one herein Figure 6A and 7 ), which comprises a relatively small observation aperture and, optionally, a relatively small display device, it is possible to consider separately the different possible propagation paths within the waveguide. Furthermore, they have recognized that, as a result of this consideration, it is possible to identify each of: regions of the display device that are light sources contributing to the desired "main" image; regions of the display device that are light sources causing undesired "ghost" images; and regions of the display device that are light sources blocked by the aperture and therefore contribute neither to the main image nor to the ghost image. The inventors have also recognized that it is possible to restrict the hologram calculation to only the regions of the display device that contribute to the main image.

[0223] The inventors' insights and the improved systems and methods embodying these insights may be further understood with reference to the accompanying drawings, which are described in detail below.

[0224] Figure 9 shows a display device 902, which in this example is an LCOS spatial light modulator. "LCOS" referred to below is short for "display device". The teachings of the present disclosure are not limited to LCOS display devices. Figure 9 depicts light associated with a virtual image point from the LCOS 902 via a waveguide 908 toward an observing entity / system 905, which in this example comprises the eye of an observer. Figure 9 also includes a magnified view of the eye 905, showing light at the pupil 904 (i.e., the entrance aperture) and the retina 906 (i.e., the sensor or observation plane). In this example, the entire LCOS area contributes to the formation of the image point on the retina 906. In other words, the entire LCOS 902 is "visible" to the observer. This contribution of the entire LCOS 902 to the image is shown by the entire LCOS being shaded, with its entire surface area being represented as the "contributing area."

[0225] As can be seen, light from the LCOS 902 in FIG9 results in the formation of three image points on the retina 906—labeled G1, M, and G2—for that particular virtual image point. The middle image point 'M' comprises the primary image point, which contributes to the primary / primary virtual image perceived by the observer. The top image point G1 comprises a first ghost image point, and the bottom image point G2 comprises a second, different ghost image point of the same virtual image point. Notably, in yet another advancement, the inventors have recognized that it is possible to identify regions of the LCOS 902 that contribute to the primary image point M and / or the ghost image points G1, G2.

[0226] Figures 10A to 10C 9 shows a LCOS 902 and a light diagram of FIG9 , which is divided into three corresponding propagation paths—a first path including light contributing to the bottom ghost image point G2, a second path including light contributing to the main image point M, and a third path including light contributing to the top ghost image point G1. Figure 10A As shown in FIG10 , the light contributing to G2 bounces three times before being transmitted by waveguide 908. As shown in FIG10 b , the light contributing to M bounces twice before being transmitted by waveguide 908. Figure 10C As shown, the light contributing to G1 bounces once before being transmitted by waveguide 908.

[0227] Each figure (10A, 10B, 10C) also shows, by shading, the portion of the LCOS 902 that contributes to the corresponding image point. Thus, it can be seen that the bottom ghost image point G2 is contributed by the area toward the bottom of the LCOS 902, the top ghost image point G1 is contributed by the area toward the top of the LCOS 902, and the main image point is contributed by the entire LCOS 902.

[0228] 9 and 10a to 10c, the aperture 904 (ie, the observer's pupil) is relatively wide, which explains why the entire LCOS 902 contributes to the primary image point. In other words, in this example, the f-number of the viewing system is relatively low. Figures 10A to 10C As shown, although portions of the LCOS 902 also contribute to one or the other ghost image G1, G2, a region of the LCOS 902 does not contribute to either ghost image G1, G2, but rather contributes only to the main image point M. The inventors have recognized that this region can be identified as a contributing region, and more specifically, for the LCOS 902 in this example, it can be identified as a "primarily contributing region," as will be further understood from the description of the subsequent figures. Thus, it can be seen that, in this case, the primarily contributing region is not limited to a circular or elliptical shape, but can take other more complex shapes.

[0229] Figures 11A to 11C The corresponding ray diagrams for different points of the virtual image are shown when the incident aperture is relatively small (ie, the f-number is relatively high). Figure 11A The first field point of the virtual image (i.e. the first virtual image point) is involved, Figure 11B The second field point involving the virtual image, Figure 11C A third field point involving a virtual image. Figures 11A to 11C It shows that not all LCOS 902 contribute to the main image point. In fact, Figures 11A to 11C It is shown that a first region of the LCOS corresponds to a primary image point (referred to herein as the "primary contributing region"), while a second region of the LCOS corresponds to a ghost image point (referred to herein as the "secondary contributing region").

[0230] The inventors have recognized that under certain conditions, different corresponding areas of the LCOS 902 (or other display device in the viewing system) will contribute to the main image, the ghost image, or not contribute to any visible portion of the image. They further recognized that this information can be used to optimize the hologram determination process. For example, light from certain parts of the display device can be omitted, or in some cases, the way the hologram encodes them can be changed so that they contribute actively to the main image rather than to the ghost image. In addition, additional areas of the display device can be identified that can be configured to contribute actively to the main image.

[0231] As an example, the inventors' findings will be described below in conjunction with point cloud holograms. However, they can be applied to other types of holograms, such as Fourier or Fresnel holograms. That is, the LCOS information determined according to this disclosure can be used to optimize other hologram calculation methods.

[0232] As is well known, to compute a point cloud hologram of an image (e.g., a virtual image), the image is typically decomposed into (i.e., represented by) a plurality of individual points—herein referred to as "virtual points" as we describe the formation of a virtual image. A spherical wave (or "wavelet") is then propagated computationally (i.e., using models or other theoretical tools) from the expected or desired location of each virtual point within the virtual image to the plane of the display device, such as the LCOS plane in the example above. The manner in which these wavelets interfere with each other is taken into account, and the resulting amplitude and / or phase of the wavelet received at each pixel of the display device is calculated. The display device can then be tuned in a well-known manner, and therefore not described here, to demonstrate the amplitude and / or phase modulation required at each pixel location to simulate the computed wavelet and thereby create a hologram of the image.

[0233] The inventors have recognized that for an observation system having a waveguide and a large observation distance as described herein, if the entire display device has the net amplitude and phase of the corresponding wavelets for all virtual points, then the hologram that will be created, when displayed and illuminated, may produce one or more ghost images as well as the main image. This may occur, for example, when the observation system is configured to perceive the virtual image at a limited distance from the observer. Furthermore, in many cases, light emitted from pixels in some parts of the device will be wasted (i.e., they will not contribute to the image seen or perceived by the observer) because the physical limitations of the observation system (e.g., a small aperture and / or a small display device and / or a large projection distance) will dictate that light from those parts of the device will not enter the observer's eyes. Therefore, the inventors have recognized that intelligent selection can be applied as to which parts of the display device are tuned to provide a hologram. Specifically, if only those portions (or portions or regions) of the LCOS that contribute to the primary image are selected, and if the wavelet is computationally propagated only to those portions of the LCOS from the virtual point of the intended virtual image, and not to other portions of the LCOS that do not contribute to the primary image, then the composite amplitude and / or phase of the wavelet received at each pixel within the selected region of the display device can be calculated without requiring calculation for any corresponding other portions of the display device.

[0234] Then, based on the improved calculations, the display device can be tuned to display the desired amplitude and phase modulation at each pixel location within the selected portion in order to simulate the calculated wavelet and thereby create a hologram of the primary image. When doing so, no other portions of the LCOS are tuned, and therefore, when the calculated hologram is displayed on the display device and illuminated, no image information will be transmitted from those other portions to the observer's eye (or other observing entity). Consequently, no information is available to the observer, which could result in the formation of undesirable "ghost" image points. As a result, the ghost images are eliminated or "extinguished." Furthermore, no computational or image information is wasted because, for a given set of conditions (e.g., for a specific aperture width and position for the eye), only those pixels of the display device known to provide light that will be allowed to pass through the observer's pupil (or through the aperture of the corresponding other observing entity) will be tuned.

[0235] FIG12 shows a system 1200 for forming a virtual image including an example virtual point 1201. The observation system 1200 includes a display device 1202, which in this example is an LCOS SLM, including a contributing region 1203 and a non-contributing region 1207 identified in accordance with the present disclosure. The display device 1202 is arranged to display a hologram of the virtual image and project light encoded in accordance with the hologram into an eye 1205, which includes a pupil (not shown) serving as an aperture, a lens 1209, and a retina 1206 serving as an observation plane. The lens 1209 and the retina are separated by a spacing 'A'. There is a light source (not shown) arranged to illuminate the display device 1202. The observation system 1200 also includes a waveguide 1208 located between the LCOS 1202 and the eye 1205. The image is schematic and therefore does not show physiological details of the eye.

[0236] Virtual point 1201 is located upstream of display device 1202, as depicted in FIG12 by virtual point 1201 being positioned to the left of display device 1202. Virtual point 1201 has a position defined by spatial coordinates, which in this example comprise Cartesian (x, y, z) coordinates, although other coordinate systems or other means of identifying the position of a virtual point may be used. A distance 'z' is defined between virtual point 1201 and display device 1202, in a direction substantially parallel to the optical axis of display device 1202. A display-to-lens distance 'l' is also defined between display device 1201 and the eye lens 1209, in a direction substantially parallel to the optical axis of display device 1201. At a given time, the values of 'z' and 'l' will vary depending on the specific arrangement of viewing system 1200, including the position of the observer. For example, the display-to-lens distance 'l' may be on the order of approximately 1 meter, while the display-to-image distance 'z' may be greater, for example, on the order of several meters. However, these numerical examples are purely illustrative and should not be considered limiting.

[0237] The inventors have recognized that if a virtual image including virtual image point 1201 is to be perceived by an observer at the position shown in FIG12 , a corresponding image point 1211 must be formed on the retina 1206. Light rays can be traced from virtual point 1201 of the virtual image to a corresponding point 1211 on the retina 1211 by LCOS 1202.

[0238] It should be understood that due to the possible paths created / produced by the waveguide 1208, more than one possible light path can be taken through the LCOS 1202 between the virtual point 1201 and its corresponding point 1211 on the retina. According to an embodiment, a principal ray can be determined, which includes a ray path among the multiple ray paths between the virtual image point 1201 and the corresponding point 1211 on the observation plane (i.e., the retina 1206). When this principal ray path is identified, the number of bounces that the light experiences within the waveguide is determined. This number of bounces (B) can be set to the number of bounces that the light should trace between the virtual image and the observation plane. According to an embodiment, as an initial step, the principal ray and the associated number of bounces (B) can be identified.

[0239] In this example, ray tracing can determine the portion of the LCOS 1202 through which a "principal ray" passes between each virtual image point 1201 and the corresponding point 1211 on the retina in order to identify the "contributing region" 1203 of that virtual image point 1201. Thus, in FIG12 , there is a ray 'r' depicted as propagating between the virtual image point 1201 and the contributing region 1203 of the display device 1202. According to the inventors' understanding, only the wavelets contributing to the contributing region of the LCOS need to be modeled (or otherwise computationally accounted for) from the virtual image point 1201 to the display device 1202. In other words, only the identified contributing region 1203 of the display device 1202 needs to be encoded (or "tuned") in order to generate an appropriate hologram. When encoded and properly illuminated on the display device, such a hologram will enable an observer to perceive the virtual image point 1201 without any ghosting of the virtual image point 1201. This can be seen from the discussion below Figure 13 and 14 Further understanding.

[0240] Contributing area 1203 in FIG. 12—and below regarding Figure 13 and 14 The contributing area discussed - can be sized and shaped based on the size and shape of the entrance aperture of the corresponding viewing entity and associated optics (e.g., waveguide geometry, any reflections within the larger optical system, etc.). Thus, when the viewing entity is a human eye, in some cases the contributing area on the display device may comprise a substantially circular or elliptical shape, or any other suitable shape, such as a complex shape similar to the receiving pupil size. However, the present disclosure includes more complex contributing area shapes. The pupil diameter can be measured or estimated in any suitable manner. For example, measurement of the eye pupil diameter can be performed by an eye tracking system. Alternatively, it can be estimated based on a known range of eye pupil diameters (e.g., 2-6 mm) or based on another estimate of the ambient light conditions at a given time.

[0241] The contributing area can be set up so that it intentionally contributes an area slightly larger than the pupil (in the aperture plane), and / or contributes an area that is slightly different in shape from the pupil (or other aperture) (in the aperture plane). In this case, not all light from the "contributing area" may pass all the way through the pupil, but the eye will be able to move slightly while still collecting enough light to form a good image on the retina.

[0242] Figure 13 A method for determining contributing and non-contributing areas of a display device according to a main aspect of the present disclosure is shown. Optionally, these determinations can then be used to optimize the generation of one or more holograms for display and illumination by an observation system such as the system 1200 of FIG. 12 . Figure 13 In the described method, the observation system includes a lens and a camera with an "f" number (i.e., focal length and aperture). The light-sensitive component of the camera can be, for example, a CCD array and is located in the observation plane. Functionally, the lens and camera replace the eye lens and retina of the observer's human eye and are only used in the process of determining contributing and non-contributing areas of the display device. These areas of the display device can be determined for multiple observation positions (e.g., eye positions within an eye box) and / or multiple image distances (e.g., a virtual image distance in front of a vehicle). In some aspects, reference is made to Figure 13 The disclosed method can be considered a precursor to hologram computing. The method can be considered an optimization or even a calibration process.

[0243] It is easy to understand that each virtual image to be generated can be represented by one or more virtual image points, and each virtual image point has a corresponding position, for example, defined by (x, y, z) coordinates. Figure 13Steps 1302 through 1312 of method 1300 (described in detail below) can be applied separately to each virtual image point within the virtual image to be created. Furthermore, method 1300 is adapted to a specific set of conditions, i.e., specific measurements and constraints, for the observation system. Thus, any given iteration (or "run") of method 1300 is adapted to establish a specific image to be created (virtual image point by virtual image point), and is adapted to when the system has a specific display-to-image distance "z," a specific distance "d" between the display device and the retina, a specific aperture (pupil) width, and a specific virtual image distance at which the eye is focused. Iterations of method 1300 are also specific to display devices of specific sizes and types, and with permitted observation windows for specific eye positions. Additional measurements and / or constraints may exist that are unique to each iteration of the method. Depending on the embodiment, if any of these measurements or constraints change, method 1300 can be rerun to redefine the contributing area of the display device under the changed circumstances. However, it will be understood that, depending on the embodiment, certain tolerances may be applied to one or more of these measurements or constraints so that if they change by less than a predetermined amount and / or for less than a predetermined length of time, the method may not have to be rerun. The rules regarding when the method should be repeated may be determined on a per-system basis.

[0244] Method 1300 may be performed by a suitable processor. The processor may include, be included within, or communicate with a holographic engine. The processor or holographic engine may be included in a light engine.

[0245] Prior to executing method 1300, the processor may obtain or receive boundary information about the viewing system. For example, it may obtain or receive information about the size of components such as a display device, information about the absolute and / or relative positions of various components and the viewing system (e.g., a potential human observer), information about light sources, etc.

[0246] According to the method 1300, in a first step 1302, the position of a virtual image point (herein referred to as a "virtual point") is obtained according to the position where the virtual image is to be perceived, for example, the coordinates [x virtual ,y virtual ,z virtual ]. A virtual image distance between lens 1209 and the virtual point is then obtained or determined. The virtual image distance may be set or determined by the processor executing method 1300, or may be set or determined by another entity and communicated to the processor. In some arrangements, it may be preset or selected from a plurality of possible virtual image distances. In actual operation, when the observation system is an eye, eye tracking or head tracking information may be used to determine the virtual image distance.

[0247] In a second step 1304, the required distance 'A' between the lens and the sensor is determined in order to focus on the virtual image point. Each virtual image point can also be defined by an angle, see Figure 4 The “angular content” mentioned here is relative to the virtual image point of the virtual image.

[0248] In a third step 1306, the number of reflections or bounces "B" of light associated with a primary or main image formed by the viewing system within the waveguide is determined. Those skilled in the art of optics will appreciate that a waveguide generates multiple copies of light associated with a virtual image point, and each copy can be associated with a different number of light bounces / reflections within the waveguide. By way of example only, one way to determine B is to determine the intersection of the chief ray of each possible light propagation path in the waveguide with the display device and select the number of reflections / bounces that brings the chief ray closest to the center of the display device. Advantageously, this approach maximizes the area of the display device that contributes to the viewing system.

[0249] Alternatively, another way of calculating the number of bounces used in the third step 1306 includes the following sub-steps 1 to 5:

[0250] 1. Eye position is known and used as input

[0251] 2. For the first number of bounces B, trace the ray from the center of the display device to the determined eye position. The extrapolation of the ray to the virtual image defines the field of view angle (θ B ).

[0252] 3. For the second bounce number B+1, ray tracing is performed from the center of the display device to the determined eye position. The extrapolation of the ray to the virtual image defines the field of view angle (θ) for this bounce number (B+1). B +1).

[0253] 4.B is for θ B and θ B +(θ B +1-θ B ) / 2 the number of bounces of the angle content

[0254] 5.B+1 is used for θ B +(θ B +1-θ B ) / 2 and θ B Number of bounces of angular content between

[0255] The output of the first step 1302 (ie the coordinates of the virtual image point) and the output of the third step 1306 (parameter b) are used in a fourth step 1308 to determine the corresponding image position / point [x sensor ,y sensor ,zsensor ]. That is, the fourth step 1308 determines the point on the sensor that receives the light of the virtual image point. In other words, the point on the sensor where the virtual image point is imaged. Figure 14 , this point on the sensor is referred to below as the primary image point [x sensor ,y sensor ,z sensor ]. As an example only, for B bounces within the waveguide, computational ray tracing from a virtual point to the sensor may be used, but the present disclosure is not limited to this approach for the fourth step.

[0256] Those skilled in the art will appreciate that it is possible to identify the virtual point [x virtual ,y virtual ,z virtual ] to the point [x sensor ,y sensor ,z sensor ] (or simply the principal ray). Again, computational ray tracing can be used to identify or trace the principal ray, but other methods are equally applicable. In a fifth step 1310, the display device intersection [x LCOS (B),y LCOS (B),z LCOS (B)], where the display device intersection point is a location on the display device where the primary ray intersects the display device. The display device intersection point can be determined, calculated, or measured, for example, by computational ray tracing.

[0257] In a sixth step 1312, the intersection point [x LCOS (B),y LCOS (B),z LCOS (B)] The display device area related to the display device area. The display device area can be geometrically centered at the point [x LCOS (B),y LCOS (B),z LCOS (B)]. For example, the area can be circular or elliptical, but other more complex shapes can also be envisaged. If the area is a regular shape, such as a circle or an ellipse, the radius of the area can be determined, for example based on the f-number of the observation system lens. This area is referred to here as the "main contributing area", because it corresponds to the main image formed by the observation system. The word "contributing" reflects that the pixels of the display device within the recognition area of the display device are the pixels that provide the necessary information content to the sensor. Other areas of the display device (i.e. other pixels of the display device) do not contribute to the formation of the image point on the sensor. Of course, other pixels can contribute to other image points on the sensor that are related to other virtual image points.

[0258] The method according to the main aspect of the present disclosure ends with determining the main contributing area of the display device.Alternatively, the hologram may be determined based on the main contributing area instead of the entire area of the display device.

[0259] Therefore, in an optional seventh step 1314, hologram components are determined for the main contributing area based on the virtual point. Specifically, optical parameters of the main contributing area are determined. The optical parameters can be the amplitude and / or phase of each pixel in the main contributing area. For example, point cloud methods familiar to those skilled in the art can be used to determine the light amplitude and phase for each pixel in the main contributing area based on the propagation of light from the virtual point to the main contributing area. The hologram component of the virtual point can be stored and combined with hologram components of other virtual points as part of the iterative process described in the next paragraph to create a complete hologram for the entire virtual image.

[0260] In summary, in the seventh step 1314, a light modulation value (eg, amplitude and / or phase value) is assigned to each pixel value of the display device within the main contributing region. This is done by considering the light wave from [x virtual ,y virtual ,z virtual ] to the main contributing region and adds amplitude and / or phase to [x LCOS (B),y LCOS (B),z LCOS (B)] is achieved by determining the amplitude and / or phase of light emitted from the virtual image point and arriving at each point (i.e., pixel) of the primary contributing area by considering the propagation of the light wave, that is, the amplitude and / or phase of the light wave after it has traveled a certain distance from the virtual image point to each pixel. This determination can be performed using any of a variety of different techniques known to those skilled in the art of optics. This determination can also be performed through experimental measurement.

[0261] Steps 1 through 7 can be repeated for each virtual point within the virtual image to be projected using the hologram. For example, multiple hologram components can be summed together to produce a composite hologram for each pixel of a display device. For example, the complex amplitudes propagated from all virtual image points can be summed at each pixel. If the hologram is to be displayed on a purely phase modulator, the amplitude component of the resulting complex amplitude sum can be neglected, leaving only the phase. More generally, this result is a diffraction structure corresponding to the virtual image, which, when displayed and illuminated on a display device within the viewing system, forms a virtual image.

[0262] The hologram can be displayed or encoded on a display device. As a result, the display device will be tuned to modulate the light in a way that enables an observer to perceive the virtual image at the desired virtual image distance.

[0263] Method 1300 can be performed substantially simultaneously (or in very rapid succession) for each of a plurality of virtual points within a virtual image, so that for a given observation setup and particular numerical measurements and constraints, a suitable hologram of the entire virtual image can be derived very quickly and encoded onto a display device. The method can be rerun if there are any changes that may affect the identification of the contributing area and / or the desired tuning of the display device. The processor can be configured to rerun the method on a time-controlled loop, and / or in response to a signal indicating that a change has occurred, and / or when the content or identity of the desired virtual image changes. The processor can include a memory for storing previously calculated data, or can be in communication with the memory. For example, a lookup table or other storage device can be provided that indicates the valid area of the display device for a particular virtual image or virtual point under a particular set of measurements and / or constraints.

[0264] Method 1300 can be run (or rerun) very quickly to display multiple different virtual images in rapid succession and / or accurately respond to changes in conditions, such as movement of a user. Although only one eye is shown in the system of FIG12 , method 1300 can be configured to account for both eyes of the observer. Furthermore, although some of the above description may refer to aperture width, it should be understood that the pupil (and most other apertures used to view an entity) is two-dimensional and can change size in each of these two dimensions. Method 1300 can be configured to account for two-dimensional aperture sizes and variations therein.

[0265] The inventor discovered that using reference Figure 13 The disclosed method can effectively determine the hologram of a virtual image. However, the inventors have also observed that in some cases, when all areas of the LCOS that would traditionally transmit light that would form ghost images are not being used, only a relatively small portion of the LCOS is being utilized. In another notable technical advance, the inventors have found a way to use additional areas of the LCOS in addition to the main contributing area, and calculate hologram values for these additional areas that will enable them to contribute light to enhance the main image rather than forming unwanted ghost images.

[0266] It is well known that the optical path taken by a ray through a waveguide in the viewing system may increase its path length relative to the path lengths of other rays. Typically, this increase may be small compared to the virtual image distance 'v' and therefore not visible to the eye.

[0267] Figure 14 Yet another improved method 1400 according to additional insights of the inventors is shown, which may be applied to a system such as the system 1200 of FIG. 12 . Figure 14 The method 1400 includes Figure 13 All steps of method 1300 are included, and in addition, it also includes the processing of one or more ghost image points corresponding to the virtual point, which virtual point may also exist and generally result in the perception of one or more ghost images of the virtual image.

[0268] Method 1400 may be performed by a suitable processor. The processor may include a holographic engine, or be included within or in communication with a holographic engine. The processor or holographic engine may be included within a light engine.

[0269] Before executing the method, the processor may obtain or receive boundary information about the system. For example, it may obtain or receive information about the size of components such as a display device, information about the absolute and / or relative positions of various components and an observer, information about light sources, etc.

[0270] In some cases, the inventors have discovered that ghost image points appear because light from the corresponding virtual point passes through a portion of the display device's "primary contributing area" that is different from the portion through which the primary image's chief ray passes. In the preceding figures, these portions of the display device are referred to as "secondary contributing areas." The light that produces one or more ghost image points can be said to include one or more "ghost rays." The light that produces the ghost image may experience a different number of bounces within the waveguide than the light corresponding to the primary image in order to also pass through the narrow pupil of the observer's eye and coincide with the retina. Therefore, if it is determined that the chief ray corresponding to the primary image experienced "B" bounces within the waveguide, it can be determined that the light corresponding to the ghost image experienced "B+ΔB" bounces, where ΔB can be a negative or positive integer, typically a single digit, such as in the range of -5 to +5.

[0271] according to Figure 14 Improved method 1400, in Figure 13 After the fourth step 1308 of the method 1300, the position of the main image point on the observation plane is established, for example its coordinates (x sensor ,y sensor ,z sensor ), Figure 13 The subsequent steps of method 1300 may continue, and in addition, for example, in parallel or at a later time, another set of steps may be performed for at least one DB value as follows. In summary, Figure 14 The improved method 1400 from the coordinates of the virtual point [x virtual ,y virtual ,z virtual] determines how many bounces 'B+ΔB' the ghost ray will undergo in order to form a ghost image point at the observation plane. The improved method 1400 then determines a translated (or corrected) position of the virtual point from which the light can travel and undergo 'B+ΔB' bounces within the waveguide and reach the primary image point on the observation plane, rather than forming a separate ghost image point. The locations on the LCOS through which the light travels from the translated position of the virtual point to the primary image point can then be identified and can be encoded with a hologram accordingly. Thus, one or more additional regions of the LCOS (in addition to the primary contributing region) can be encoded with hologram values to contribute to the primary image while still avoiding the generation of ghost images.

[0272] In more detail, the improved method 1400 is as follows:

[0273] In a first further step 1402, from the primary image point (x sensor ,y sensor ,z sensor ) are traced back to the virtual image, but for rays that experience 'B+ΔB' bounces / reflections (instead of B bounces) within the waveguide.

[0274] In a second further step 1404, the position of the secondary virtual point of the virtual image is determined (eg coordinates [x virtual (ΔB),y virtual (ΔB),z virtual (ΔB)] (eg as a result of the ray tracing performed in the first further step 1402), the secondary virtual point will be imaged to the primary image point [x sensor ,y sensor ,z sensor ], that is, the secondary virtual point will propagate the light that will pass through the display device, waveguide and incident aperture to correspond to the position [x sensor ,y sensor ,z sensor ] if the light undergoes a 'B+ΔB' bounce. The term 'secondary virtual point' is used here as a shorthand for the secondary (i.e., shifted or modified) position of the (primary) virtual point. That is, the inventors have recognized that if the position of the virtual point is moved to the 'secondary virtual point' position [x virtual (ΔB),y virtual (ΔB),z virtual (ΔB)], any light from the 'secondary virtual point' that undergoes 'B+ΔB' bounces / reflections in the waveguide will contribute to the primary image at the observation plane.

[0275] In summary, the third further step 1406 comprises determining the coordinates [x LCOS (B+ΔB),y LCOS (B+ΔB),zLCOS ], used to change the direction of the waveguide from [x virtual (ΔB),y virtual (ΔB),z virtual ] to the observation plane for light propagation after B+ΔB bounces. In some cases, z virtual can be adjusted to account for different path lengths through the waveguide (ie due to different numbers of bounces). This principal ray may be referred to as the "secondary principal ray".

[0276] In more detail, in a third further step 1406, a point on the display device is identified, where the distance from the secondary virtual point to the primary image point [x sensor ,y sensor ,z sensor ] will pass through this point and undergo B+ΔB bounces in the waveguide. This point has coordinates [x LCOS (B+ΔB),y LCOS (B+ΔB),z LCOS ].

[0277] In the fourth additional step 1408, the point [x LCOS (B+ΔB),y LCOS (B+ΔB),z LCOS ] is assigned a radius or other suitable indicator of the extent or size of the region associated with it. The region associated with the point [xlcos(b+δb], ylcos(b+δb], zLCOS] is referred to herein as the "additional contributing region" because it propagates light that contributes to the primary image point at the viewing plane, but only when that light originates from a displaced or modified position of the (primary) virtual point, i.e., [x virtual (ΔB),y virtual (ΔB),z virtual ] instead of [x virtual ,y virtual ,z virtual ], as determined in a second further step 1404.

[0278] The fourth further step 1408 is similar to the sixth step 1312. Specifically, the fourth further step 1408 includes identifying the intersection point [x LCOS (B+DB),y LCOS (B+DB),z LCOS (B)] The area of the display device related to the point [x LCOS (B+DB),y LCOS (B+DB),z LCOS(B)] is centered. For example, this area can be circular or elliptical, but other more complex shapes are also conceivable. If the area has a regular shape, such as a circle or an ellipse, the radius of the area can be determined, for example, based on the f-number of the observation system lens. This area is referred to here as the "additional contributing area" because it will transmit light that contributes to the virtual image if an appropriate hologram is calculated based on the displaced or modified position of the (primary) virtual point.

[0279] The fifth further step 1410 is similar to the seventh step 1314. The fifth further step 1410 is optional. In the fifth further step 1410, based on the modified position [x virtual (ΔB),y virtual (ΔB),z virtual ], determine the hologram components for the additional contributing area. Specifically, determine the light parameters of the additional contributing area. The light parameters can be the amplitude and / or phase of each pixel in the additional contributing area. For example, a point cloud method familiar to those skilled in the art can be used based on the light from different virtual points [x virtual (ΔB),y virtual (ΔB),z virtual ] to the additional contributing area, and the light amplitude and phase are determined for each pixel in the additional contributing area. Different virtual points [x virtual (ΔB),y virtual (ΔB),z virtual ] and combines it with the hologram components of other virtual points as part of the iteration described in the following paragraphs to build a complete hologram for the entire virtual image.

[0280] This desired light modulation by the display device (which is output relative to a single virtual point) can be referred to as the "hologram component" of that virtual point. During subsequent iterations of method 1300 for one or more other virtual points, the hologram component can be stored by the processor within the virtual image to be created.

[0281] Figure 14 Further improvements to method 1400 can be made with steps 1402 to 1410 of Figure 13 Steps one 1302 to seven 1314 of method 1300 are repeated for each virtual point within the virtual image to be created. When the modulation behavior and the corresponding hologram components have been determined for each virtual point, the hologram components can be added together to produce a composite modulation behavior for each pixel of the display device. This composite modulation behavior represents the diffraction structure or hologram of the virtual image, which, if displayed and illuminated on a display device within the observation system, will only result in the formation of a main image without any ghost images. As performed Figure 14The main image formed as a result of the improved method 1400 may be better than that formed by Figure 13 The corresponding main image produced by method 1300 is brighter.

[0282] The processor may output data corresponding to the hologram in any suitable manner. The hologram may be encoded onto a display device. As a result, the display device will be tuned to modulate light so that an observer can perceive the virtual image at a desired virtual image distance without forming any ghost images.

[0283] Method 1400 can be performed substantially simultaneously (or in very rapid succession) for each of a plurality of virtual points within a virtual image, so that for a given observation setting and particular numerical measurements and constraints, a suitable hologram of the entire virtual image can be derived very quickly and encoded onto a display device. The method can be rerun if there are any changes that may affect the identification and / or required tuning of the display device. The processor can be configured to rerun the method on a time-controlled loop, and / or in response to a signal indicating that a change has occurred, and / or when the content or identity of the desired virtual image changes. The processor can include a memory for storing previously calculated data, or can be in communication with such a memory. For example, a lookup table or other storage device can be provided that indicates the valid area of the display device for a particular virtual image or virtual point under a particular set of measurements and / or constraints.

[0284] Method 1400 can be run (or rerun) very quickly to display multiple different virtual images in rapid succession and / or accurately respond to changes in conditions, such as movement of a user. Although only one eye is shown in the system of FIG12 , method 1400 can be configured to account for both eyes of the observer. Furthermore, although some of the above description may refer to aperture width, it should be understood that the pupil (and most other apertures used to view an entity) is two-dimensional and can change size in each of these two dimensions. Method 1400 can be configured to account for two-dimensional aperture sizes and variations therein.

[0285] According to the main aspects of the present disclosure, the inventors have discovered that each virtual image point corresponds to a different main contributing area on the display device. The inventors have further realized that this means that light from different parts of the virtual image (i.e., different virtual image points) follows different light paths through the system. Figure 15A and 15B In the illustrated embodiment, the inventors have configured the system such that, in simple terms, (i) the virtual image comprises multiple discrete virtual image components or regions, and (ii) the light of each virtual image component is associated with a different number of bounces / reflections within the waveguide 1508.

[0286] Figure 15AAn image 1552 is shown for projection, comprising eight image regions / components V1 to V8. Figure 15A Eight image components are shown by way of example only, and the image 1552 may be divided into any number of components. Figure 15A Also shown is a coded light pattern 1554 that can reconstruct image 1552—eg, when transformed by a lens of a suitable viewing system. The coded light pattern 1554 includes first through eighth sub-holograms or components H1 through H8, corresponding to first through eighth image components / regions V1 through V8. Figure 15A It further shows how the hologram calculated according to the present disclosure can effectively decompose the image content by angle. Therefore, the hologram is characterized by the light guidance it performs. This is in Figure 15B Specifically, holograms according to the present disclosure direct light into multiple discrete regions. In the example shown, the discrete regions are disks, but other shapes are also contemplated. After propagation through the waveguide, the optimal disk size and shape can be correlated to the size and shape of the observation system's entrance pupil. This light directing occurs only due to the specific method of determining the hologram disclosed herein.

[0287] Figure 15C Shown according to Figure 15A and 15B An improved viewing system 1500 is shown. Figure 13 Method 1300 or Figure 14 The method 1400 can be applied to Figure 15A and 15B The scheme shown.

[0288] Observation system 1500 includes a display device, which in this arrangement includes LCOS 1502. LCOS 1502 is arranged to display a modulation pattern (or "diffraction pattern") comprising a hologram and projects holographically encoded light into eye 1505, which includes a pupil serving as aperture 1504, a lens 1509, and a retina (not shown) serving as an observation plane. There is a light source (not shown) arranged to illuminate LCOS 1502. The lens 1509 of eye 1505 performs the conversion of the hologram into an image.

[0289] The observation system 1500 also includes a waveguide 1508 located between the LCOS 1502 and the eye 1505 . Figure 15C The projection distance in FIG. 1504 may be relatively large. However, as described with respect to the previous figures, the presence of waveguide 1508 enables all angular content from LCOS 1502 to be received by eye 1505 even at this relatively large projection distance. This is because waveguide 1508 acts as a pupil expander in the manner already described above.

[0290] Furthermore, in this arrangement, when LCOS 1502 has been encoded according to the methods described herein, waveguide 1508 can be oriented at an angle relative to LCOS 1502 to establish a unique relationship between light from LCOS 1502 and the virtual image that an observer will perceive. The size, position, and positioning of waveguide 1508 are configured to ensure that light from each portion of the virtual image enters waveguide 1508 and is guided along its elongated axis, bouncing between the substantially planar surfaces of waveguide 1508. Whenever light reaches the second planar surface (closest to eye 1505), some light is transmitted and some light is reflected.

[0291] Figure 15C A total of nine "bounce" points B0 to B8 are shown along the length of waveguide 1502. The reader will note that the center of image 1552 remains blank. Figure 15C The 0th through 9th light "bounce" or reflection points B0 through B8 within the waveguide are shown. Although light associated with all points of the image (V1-V8) is transmitted out of the waveguide on each "bounce" from the second planar surface of the waveguide 1508, only light from one angular portion of the image (e.g., light from one of V1 through V8) has a trajectory that enables it to travel from each respective "bounce" point B0 through B8 to the eye 1505. Furthermore, light from a different angular portion of the image V1 through V8 reaches the eye 1505 from each respective "bounce" point. Figure 15C Light from all the different angular contents is shown emitted at each "bounce" point (indicated by the multiple short arrows at each transmission point), but then only the light path of each angular content to the eye 1505 is shown, which will actually reach the eye 1505 and thus contribute to the corresponding portion of the virtual image that the observer will perceive from that portion of the waveguide. For example, for the zeroth bounce B0, the light transmitted by waveguide 1508 is simply refracted and does not experience any reflections therefrom. Light from the eighth sub-hologram H8 reaches the eye from the zeroth bounce B0. For the next bounce B1, light transmitted by waveguide 1502 experiences one bounce therefrom before being transmitted. Light from the seventh hologram H7 reaches the eye from the next bounce B1. This continues in sequence until the light transmitted by waveguide 1508 at the final bounce B8 has experienced eight bounces before being transmitted and reaching the eye 1505, and includes light encoded according to the first hologram H1.

[0292] In the example shown in Figure 15, only one image region's light reaches the eye from each bounce point. Thus, when determining the hologram as described herein, a spatial correlation is established between the regions of the virtual image and their associated bounce points on the waveguide. In some other examples, there may be relatively little overlap, such that one region of the image originates from two adjacent transmission points and is therefore contained within two adjacent disks propagating from the waveguide to the viewing plane.

[0293] Thus, the inventors' recognition and the methods and apparatus described herein enable the production of diffraction patterns (or light modulation patterns) including holograms that, when displayed on an LCOS or other suitable display device, enable light to be effectively emitted therefrom as multiple "disks" or beams, each disk or beam corresponding to (more specifically, encoding) a different respective portion of a respective virtual image.

[0294] Thus, improved methods and apparatus are described herein that enable holograms to be calculated and displayed on a suitable display device in such a manner that, when the display device is illuminated by a suitable light source, an observer sees a clear image. The image seen by the observer can be free of ghosting and can be made brighter by contributions of light that would normally produce ghosting images, rather than a single main image.

[0295] The improved methods and apparatus described herein can be implemented in a variety of different applications and viewing systems. For example, they can be implemented in a head-up display (HUD). In an improvement to many conventional HUDs in which virtual images are formed, the improved methods and apparatus described herein can be implemented to create virtual images at a limited image distance that can be selected and adjusted by a suitable controller while still eliminating ghost images.

[0296] Although virtual images have been discussed herein, which require the eye to convert received modulated light to form a perceived image, the improved methods and apparatus described herein can be applied to real images.

[0297] Waveguide aperture

[0298] Although the above figures have shown a monocular or single "opening" or "entrance pupil", all of the arrangements and methods described herein are applicable to viewing systems having multiple entrance pupils, such as, most commonly, human observers having two eyes.

[0299] The inventors have recognized that when computing a hologram of a target image as described above, including constraining the hologram according to an entrance pupil of an observation system, the possible effects of having multiple entrance pupils in the observation system should be considered—for example, when an observer observes the holographically reconstructed image with both eyes. In embodiments, the hologram can be constrained according to one of the multiple entrance pupils, but some or all of the image may also be visible to one or more other entrance pupils of the multiple entrance pupils. In embodiments, the hologram can be constrained according to two or more of the multiple entrance pupils. For example, two sub-holograms can be computed and combined (e.g., added) into a single hologram for display on a display device, where each sub-hologram is constrained according to a different corresponding entrance pupil. For example, a left-eye hologram and a right-eye hologram can be computed according to corresponding left-eye and right-eye images, where the left-eye hologram is constrained according to the entrance pupil of the observer's left eye during computation, and the right-eye hologram is constrained according to the entrance pupil of the observer's right eye during computation. In an embodiment, two or more holograms constrained according to different corresponding entrance pupils in a plurality of entrance pupils can be interleaved with each other. In other words, two holograms can be alternately displayed in rapid succession so that an observer perceives that the two corresponding images are formed substantially simultaneously.

[0300] The inventors have recognised that additional guidance or control may be applied to the light propagating from a displayed hologram towards a viewing system in order to make it more suitable for reception by the multiple entrance pupils of the viewing system.

[0301] As described in detail above, when a hologram is calculated according to the method and displayed and transmitted using a system such as that shown in Figure 15c, at each transmission point - or "bounce point", such as B0 to B8 shown in Figure 15c - light associated with all points of the image (V1-V8) is transmitted out of the waveguide. Figure 15A The middle area of the image 1552 is blank, so Figure 15B The hologram channels associated with the middle of the image are not shown or labeled. Figure 15C, the optical path associated with transmission point B4 may not deliver any image content to the observation system. However, for a single entrance pupil observation system, only light from one angular portion of the image (e.g., light from each of V1 to V8) has a trajectory that enables it to reach the eye 1505 from each corresponding "bounce" point B0 to B8. In addition, light from a different angular portion of the image (V1 to V8) reaches the eye 1505 from each corresponding "bounce" point. Therefore, in Figure 15c, a single entrance pupil 1504 is shown as receiving all angular content of the image, but each angular portion of the image comes from a different corresponding bounce point on the waveguide 1508. However, the inventors have recognized that if the observation system has two or more entrance pupils, there is a risk that light from the same angular portion of the image will reach both entrance pupils at the same time because all angular portions of the image are emitted at each bounce point on the waveguide. For example, when the observation system is a person and the multiple entrance pupils include both eyes, the right eye may receive light including a particular angular content V1 via the optical path. x The light is in a first quantity B y After bouncing, it is emitted from the waveguide, and the left eye can receive the same angular content V at the same time (or substantially at the same time, given the speed of light) via the optical path. x , which bounces at a different second quantity B z It is then emitted from the waveguide. This can lead to confusion because the human brain—indeed, the processor associated with any non-human viewing system—does not expect to receive the same portion of the image (i.e., the same image content) simultaneously and at the same angle at two spatially separated entrance pupils. Instead, the brain (or other processor) expects light from a single point or single portion of the image to be received by two (or more) entrance pupils at different respective angles.

[0302] Figure 16 It is shown that the above-mentioned problem has been solved by the present inventors. Figure 16 A display device 1602 is shown displaying a hologram calculated according to the methods described herein.

[0303] Display device 1602 is illuminated by a light source (not shown). The light is spatially modulated by the displayed hologram and propagates toward waveguide 1604. As described in detail with respect to the previous figures, the light is refracted within waveguide 1604. After refraction, some of the light is emitted toward the viewer at a first transmission point (or "bounce point"), while other portions of the light are internally reflected (or "bounced") within waveguide 1604 before being transmitted, with corresponding portions of the light being emitted toward the viewer at different corresponding transmission points (or "bounce points") along the length of waveguide 1604.

[0304] Figure 16 Only a portion of the image represented by the displayed hologram is shown (V x) of the light path 1610. In reality, light from other parts of the image will also be transmitted from the waveguide to the observer, but for ease of understanding, Figure 16 It will be understood that light from other angular portions of the image will follow different respective optical paths between the bounce points on the waveguide 1604 and the observer, and that light from each (i.e., every) angular portion of the image will be emitted toward the observer at each (i.e., every) bounce point on the waveguide.

[0305] Figure 16 An entrance pupil plane 1612 and an image display plane 1614 are shown. The entrance pupil plane 1612 represents the plane where the pupil of the observer's eye is located, and the image display plane 1614 represents the plane where the image is formed on the retina of the observer's eye. Figure 16 Also shown are a first marker 1608 indicating which light path 1610 is traveling toward and will enter through the entrance pupil of the observer's right eye, and a second marker 1606 indicating which light path 1610 is traveling toward and will enter through the entrance pupil of the observer's left eye. It can be seen that each eye will only receive light from one of the multiple light paths 1610, and each eye will receive light through a different light path 1610 relative to the respective other because the observer's eyes are naturally spatially separated from each other. However, as described above, Figure 16 All of the light paths shown include light for the same (i.e., common) portion of the image. Consequently, both eyes will receive the same image content at the same angle of incidence, albeit through different respective light paths. The inventors have recognized that this can lead to confusion and can impair the clarity of the image perceived by the observer. The inventors have addressed this potential problem, as will be understood from the following description. Furthermore, although Figure 16 Only the optical path of light of one angular portion of the image is shown in FIG, but the left eye and the right eye of the observer can also simultaneously receive light of one or more other angular portions of the image in a similar manner.

[0306] Figure 17 A display system or light engine is shown that includes a waveguide 1704 and an observation system 1710 that includes two entrance pupils. In this example, the observation system 1710 is a human observer, and the two entrance pupils are part of the observer's left eye 1706 and right eye 1708, respectively, however this should be considered non-limiting and the present disclosure is applicable to other types of observation systems as well. Figure 17 The rest of the display system is omitted. The waveguide 1704 has a (substantially) center point 1702 defined on its larger face closer to the viewing system 1710, wherein a "position along the waveguide" (P WG) is equal to zero (P WG =0). The optical axis is represented by a substantially horizontal dashed line extending from the center point 1702 (in the "z" direction in this example) to the viewing system 1710. The "x" axis is defined by a substantially vertical line extending from the center point 1702 and substantially perpendicular to the optical axis. The waveguide 1704 is arranged at an angle "α" to the x-axis. In this example, the entrance pupil plane, in which the pupils of the observer's eyes (which are spatially separated from each other in the entrance pupil plane) are located, is substantially parallel to the x-axis. However, the present disclosure is not limited to Figure 17 The arrangement of the display system is not limited to the relative positions of any features therein. For example, in practice, the observer's eyes may not be precisely located in a common entrance pupil plane that is perpendicular to the optical axis extending from the waveguide to the observer. The position of the observer's eyes at a given time can be calculated or otherwise obtained by a processor included in or in communication with the display system. For example, any suitable eye tracking method can be used.

[0307] The “position along the waveguide” (P WG ) can be defined based on their position relative to the center point 1702. As a non-limiting example, Figure 17 From the perspective of observer 1710 in FIG, any point to the right of center point 1702 can be assigned a positive (+) P WG Value, P WG The size of is defined by the distance between the center point 1702 and another point along the waveguide surface. Figure 17 From the perspective of observer 1710 in FIG, any point to the left of center point 1702 can be assigned a negative (-) P WG Value, P WG The size of the waveguide is also defined by the distance between the center point 1702 and another point along the waveguide surface. A distance "D" substantially parallel to the optical axis is defined between the center point 1702 and the entrance pupil plane. Distance "D" can be relatively large compared to the size of the entrance pupil of the observer's eye and / or compared to the size of a display device (not shown) that displays the hologram of the image to be viewed by the observation system 1710. For example, distance "D" can be approximately one thousand millimeters (1000 mm).

[0308] It can be seen that the viewer's eyes 1706, 1708 are naturally spatially separated from each other. The separation between the entrance pupils of the viewer's eyes may be referred to as the "inter-pupillary distance" (IPD). Figure 17In the example shown, the entrance pupil of the right eye 1708 is offset from the intersection of the optical axis and the entrance pupil plane in the positive (+) direction along the x-axis, while the entrance pupil of the left eye 1706 is offset from the intersection of the optical axis and the entrance pupil plane in the negative (-) direction along the x-axis. Again, this is merely an example and not limiting. The inventors have recognized that controls can be provided so that the different relative positions of the observer's two eyes (and, accordingly, the different relative positions of two or more entrance pupils in any multi-entry pupil viewing system) can be taken into account to ensure that no portion of the image is received by both eyes at substantially the same time. As a reminder, special types of holograms according to the present disclosure effectively divide or separate image content by angle (albeit in the hologram domain).

[0309] Figure 18 Included is a graph illustrating insights made by the inventors as applied to a display system or light engine, such as the display system or light engine shown in Figures 15c, 16, or 17 herein. The graph shows the position of a point along the waveguide facing the viewer or viewing system (P WG ) and the angle from each eye to a point on the waveguide. In an embodiment, the waveguide is tilted relative to the viewing or entrance pupil plane of the display device (i.e., hologram) and / or viewing system.

[0310] Figure 18 The graph in FIG shows two lines—one 1806 for a first entrance pupil of the viewing system, such as the left eye of the viewer, and another 1808 for a different second entrance pupil of the viewing system, such as the right eye of the viewer. It can be seen that for any given light angle θ, there exists a first position P along the waveguide. WG1 , from which light is emitted to reach the left eye. For this same light angle θ, there is a second position P along the waveguide that is different WG2 , from which light is emitted to reach the right eye. Therefore, both eyes will receive the same image content (i.e., the angle of light) at essentially the same time, but from different parts of the waveguide. This is the reason for the problem that the inventors solved.

[0311] All light rays (or bundles of light rays) corresponding to different parts of the image are emitted from multiple locations on the waveguide (i.e., from multiple "bounce points"). Therefore, if two or more spatially modulated light rays emitted from different locations on the waveguide but at the same angle enter two entrance pupils at substantially the same time, the image content received by the two entrance pupils (e.g., by both eyes of an observer) will be identical. This may cause observer confusion and may inhibit the quality of the image seen or perceived by the observer.

[0312] Thus, the present inventors have recognized that, in accordance with embodiments, the light received by each entrance pupil of a multiple entrance pupil observation system should be controlled. For example, simultaneous reception of light of the same image content at the same angle by two or more entrance pupils of a multiple entrance pupil observation system should be reduced, and at least in some cases should be eliminated. Furthermore, the present inventors have recognized that, at least in some cases, simultaneous reception of light of the same image content (i.e., light of the same specific angular range) by two or more entrance pupils of a multiple entrance pupil observation system should be prevented. Figure 19 It will be appreciated that the inventor's solution Figure 19 Also shown are first and second lines 1906 and 1908, which represent light received by the left and right eyes of a viewer, respectively, according to an example.

[0313] Due to the special type of hologram employed in accordance with the present disclosure, Figure 19 Each angle in corresponds to a different part of the image. In summary, the inventors discovered from their analysis of this optical geometry that multiple angular ranges or "regions" (i.e., sub-regions) of the waveguide can be identified where only one eye will receive the corresponding image content. For example, in region 2 of the waveguide, the angular range of light receivable by the left eye does not overlap with the angular range of light receivable by the right eye. In any region, the maximum angle receivable by one eye is less than the minimum angle receivable by the other eye, so there is no overlap. But the regions are seamlessly connected so that all image content (i.e., all angles) is transmitted to the viewing system. From another perspective, Figure 19 The angle range associated with region 2 in the waveguide is from the first sub-region (i.e., P WG The first sub-area of the waveguide delivers the same angular content to the first eye, and the second sub-area of the waveguide delivers the same angular content to the second eye (i.e., P WG The end of the first sub-area (e.g., the P of the first eye) does not overlap (but is seamlessly connected). WG The upper limit of the second sub-region is adjacent to the beginning of the second sub-region (for example, the P WG In short, due to the geometry of the system and the nature of the hologram, the inventors have discovered that there are multiple regions (i.e., sub-regions) of the waveguide from which the two eyes will receive light encoded with different (and crucially, non-overlapping) parts of the image. Those skilled in the art will appreciate that with reference to Figure 19 The described approach approximates a complex optical system, and the limited size of the entrance pupil, for example, may still allow some crosstalk of angular content between the eyes. However, the inventors have found that this approach is very effective in identifying waveguide aperture configurations that significantly reduce crosstalk between the eyes while still delivering full image content to both eyes.

[0314] In more detail, Figure 19 By way of example, four distinct regions are shown, each of which (Region 1, Region 2, Region 3, Region 4) defines a different, respective angular range of light emitted from the waveguide toward an observer. Each region (Region 1, Region 2, Region 3, Region 4) corresponds to a different, respective angular channel, and therefore, to different, respective image content. In other words, due to the unique nature of computational holograms, each region represents a different, respective portion of the image and is therefore considered to correspond to different image content. These regions are adjacent to and abut one another so that, in combination, they cover the entire angular range that constitutes the image and, therefore, in combination, provide all the image content required by the observer's brain to holographically reconstruct the image when receiving the spatially modulated light.

[0315] The inventors have recognized that for each portion of the image, and therefore for each region, light should preferably be allowed to be received by only one entrance pupil (i.e. by one eye of the observer rather than both eyes) to avoid observer confusion. Notably, the inventors have designed a control device at one or more selected locations between the waveguide and the observer to ensure that at any given time, light from each portion of the image (i.e. each angle) is received by only one eye of the observer. The control device may include one or more openings or holes, and one or more barriers or baffles. The control device may be referred to as an "aperture" or "waveguide aperture" having a closed portion and an open portion. The waveguide aperture may be configured so that a first eye of an observer receives angular content that a second eye cannot receive, and vice versa.

[0316] Figure 19 An exemplary waveguide aperture 1900 is shown in a first phase 1900A and a second phase 1900B. These phases may be referred to as "configurations." The present disclosure is not limited to the specific phases shown in the figures. These are merely illustrative examples.

[0317] Waveguide aperture 1900 is shown in Figure 19 Near the curve in Figure 19 The graph in is similar to Figure 18 , to show how the waveguide aperture 1900 will affect light that would otherwise propagate from regions 1 to 4 of the hologram via the waveguide to the observer's eye. Each phase 1900A, 1900B represents a possible waveguide aperture configuration that can be applied to ensure that only one eye receives light from each region at any given time. For each phase 1900A, 1900B, the waveguide aperture 1900 includes an "open" portion and a "closed" portion, each portion corresponding to a portion along the waveguide (P WG) is a position range (in millimeters) from which light can be emitted. In practice, the waveguide aperture 1900 can be physically located near the waveguide, extending through the light path between the waveguide and the observer to selectively allow and block certain light paths, as described in detail below. For example, as shown here Figure 22A 、 24A As shown in FIG25A , the waveguide aperture can be located directly in front of the waveguide. In this arrangement, the waveguide aperture can be tilted relative to the entrance pupil plane, for example, it can be substantially parallel to the elongated face of the waveguide. However, other locations and orientations of the waveguide aperture are contemplated that would provide similar functionality.

[0318] A fixed waveguide aperture corresponding to either phase 1900A, 1900B may be provided, wherein the fixed waveguide aperture is located between the waveguide and the observer to allow observation from certain positions along the waveguide (P WG ) reaches the observer and blocks light from some other location along the waveguide (P WG ) light, such as Figure 19 1900B.

[0319] For the first phase 1900A, the waveguide aperture is spatially divided into five sections, each section defining a path along the waveguide (P WG ) position range, wherein these parts are alternately open and closed. In more detail, a first open portion 1921 is defined on the left hand side of the figure. It should be understood that positional terms such as "left", "right", etc. are only used to help understand the examples shown in the figures and should not be considered as limiting. The first open portion 1921 defines the position of the waveguide (P WG ) of the right eye receiving area 4. The left eye does not receive any light from this position range. Moving from left to right, adjacent to the first open portion 1921 is the first closed portion 1922, which defines the position of the light along the waveguide (P WG ) can receive light from region 3. However, because it is a closed portion 1922, it ensures that the right eye does not receive any light from region 3. In addition, the first closed portion 1922 also covers the range of positions where the left eye can receive light from region 4. However, again, because it is a closed portion 1922, it ensures that the left eye does not receive any light from region 4. Moving further to the right, adjacent to the first closed portion 1922 is a second open portion 1923, which defines a region along the waveguide (PWG ) of the left eye receiving area 3. Furthermore, the same position range along the waveguide is the range of light receiving area 2 for the right eye. Moving further to the right, adjacent to the second open portion 1923 is a second closed portion 1924, which defines the position range along the waveguide (P WG ) will receive light from region 2. However, because it is a closed portion 1924, it ensures that the left eye will not receive any light from region 2. In addition, the second closed portion 1924 also covers the range of positions where the right eye will receive light from region 1. However, again, because it is a closed portion 1924, it ensures that the right eye will not receive any light from region 1. Finally, adjacent to the second closed portion 1924 is a third open portion 1925. It defines the range of positions along the waveguide (P WG ) indicates a range of positions where the left eye receives light from region 1. The right eye receives no light from this range of positions. Thus, when the waveguide aperture is in the configuration represented by the first phase 1900A, light from each region is allowed to enter one eye while being blocked from the corresponding other eye. Thus, duplication of received image content is avoided.

[0320] For the second phase 1900B, the waveguide aperture is again spatially divided into five sections, defining the same along-waveguide (P WG ), but in the second phase 1900B, when viewed from left to right, these parts are alternately closed and open, as shown in FIG. Figure 19 . More specifically: First closed portion 1921' prevents light from region 4 from entering the right eye. It does not affect the left eye because the left eye does not receive any light from this position range. Moving to the right, adjacent to first closed portion 1921' is first open portion 1922'. It allows light from region 4 to enter the left eye and allows light from region 3 to enter the right eye. Moving to the right again, adjacent to first open portion 1922' is second closed portion 1923'. It prevents light from region 3 from entering the left eye and prevents light from region 2 from entering the right eye. Moving to the right again, adjacent to second closed portion 1923' is second open portion 1924'. It allows light from region 2 to enter the left eye and allows light from region 1 to enter the right eye. Moving to the right again, adjacent to second open portion 1924' is third closed portion 1925'. It blocks light from region 1 from entering the left eye. Therefore, the functions of the two phases of the waveguide aperture are summarized in Table 1 below.

[0321] By way of example only, the embodiments describe an arrangement in which the first and second aperture configurations are opposite (i.e., fully complementary). It is not essential that the phases of the control devices are fully complementary. In other embodiments described herein, more complex configurations and phases are used, particularly when the limited size of each entrance pupil is taken into account.

[0322] Table 1

[0323]

[0324] Figure 20 is an example of a target image 2000 for which a hologram is computed according to the methods described herein to form a virtual image including a holographic reconstruction. The target image 2000 is divided into four regions, each with a distinct image content. Thus, when the hologram is properly displayed and illuminated, the spatially modulated light emitted by the display device and waveguide comprises four distinct rays (or bundles of rays, or angular channels), each ray having a unique corresponding angular range, and each ray comprising a distinct corresponding portion of the image.

[0325] A hologram is calculated by dividing the image content by angle. Various methods can be used to calculate such a hologram. Generally speaking, a hologram can be constrained according to at least one entrance pupil of an observation system. A characteristic feature of a hologram is the angular division of the image content into channels. According to an embodiment, a point cloud method is used to calculate the hologram. As described above, this method uses individual virtual image points of a virtual image to be generated to calculate the hologram. However, this method is described by way of example only, and other methods for calculating holograms characterized by angular channels of image content can be used. In an embodiment utilizing the non-limiting example point cloud method, each virtual image point can correspond to a ray angle of light between the display device and the entrance pupil of the observation system. Each virtual image point can be considered a separate image component. In embodiments, a region of the virtual image can include a single virtual image point or multiple virtual image points. In embodiments, two or more adjacent virtual image points can be included within a region of the virtual image, where the angular range of the region includes all angles corresponding to the individual virtual image points included in the region.

[0326] To successfully control the content reaching each eye (or entrance pupil) of an observer (or viewing system), where each eye (or entrance pupil) occupies a different respective viewing position, each portion (i.e., each zone) of the waveguide aperture disclosed herein defines a maximum ray angle and a minimum ray angle that can be received from each viewing position—e.g., from each eye when the viewing system is a two-eyed human observer. To avoid overlap of received holographic content between viewing positions (e.g., between two eyes), for each zone, the maximum ray angle for the first eye position is less than the minimum ray angle for the second eye position. If two viewing positions are configured to receive content from two adjacent zones, the maximum ray angle for the first eye position is substantially equal to the minimum ray angle for the second eye position. For example, the waveguide aperture can be configured such that the first viewing position receives ray content corresponding to the ray angles up to and including its maximum ray angle (θ max1 ), and the second observation position receives the spatially modulated light corresponding to an angle greater than but not equal to θ max1 Maximum ray angle (θ) up to the second eye position max2 )'s spatially modulated light content within an angular range.

[0327] Figure 21 The above is shown, wherein an open portion 2100 of the waveguide aperture is shown, and the rest of the waveguide aperture is not shown. The open portion 2100 should be positioned relative to the waveguide (not shown) to allow the P WG1 and P WG2 The protrusion position range along the waveguide (P WG ) any light emitted from the waveguide passes toward the viewer. Thus, it allows light from region 2 to enter the left eye, as shown by the first line 2106, and allows light from region 1 to enter the right eye, as shown by the second line 2108. The angular extent of region 2 is shown as the minimum angle 2106. MIN Extended to the maximum angle 2106 MAX Similarly, the angular range of region 1 is shown as 2108 from the minimum angle MIN Extended to the maximum angle 2108 MAX .like Figure 21 As shown by the horizontal dashed line in FIG, region 2 and therefore the maximum angle 2106 of the left eye MAX Equal to area 1 and therefore the minimum angle 2108 for the right eye MIN Therefore, from the lower limit of area 2, 2106 MIN To the upper limit of zone 1 2108 MAX The entire angular range is covered, with each portion of the spatially modulated light of the image in regions 1 and 2 being received by one eye or the other through the open portion 2100.

[0328] Figure 22AA waveguide aperture 2250 is shown in operation. Waveguide aperture 2250 is positioned in front of, i.e., optically downstream from, waveguide 2204, which is provided as part of a display system or light engine. Other elements of the display system are not shown. For example, the display system also includes a display device configured to display a computed hologram as described herein and illuminated by a suitable light source. An observer is positioned optically downstream from waveguide aperture 2250, with the observer's left eye 2206 and right eye 2208 defining an entrance pupil plane in the "x" direction. The separation between the observer's eyes 2206, 2208 is defined as the "inter-pupillary distance" (IPD) in the x-direction. Waveguide aperture 2250 is positioned just in front of waveguide 2250 and is substantially parallel to waveguide 2250, such that both are tilted relative to the entrance pupil plane.

[0329] The open portion 2251 of the waveguide 2250 is shown to allow light to pass through the waveguide (P WG ) is emitted toward the observer from a predetermined position range. In the embodiment, there will be more than one open portion in the waveguide hole, but for ease of understanding, Figure 22A Only one is shown in FIG. Figure 21 As shown, the waveguide aperture 2250 allows light within a first angular range to be emitted from the open portion 2251 and reach the right eye 2208, and allows light within a second, different angular range to be emitted from the open portion 2251 and reach the left eye 2206. A skilled reader will understand that, in practice, light may also be emitted from the waveguide 2204 and pass through the open portion 2251 of the waveguide aperture 2250 at other angles, but only the two angular ranges shown include light from the open portion 2251 that will reach one or the other eye of the observer.

[0330] Figure 22B The image content of region 1 of the target image 2000 is shown using Figure 22A system, whose light will be delivered to the observer's left eye. Figure 22C The image content of region 2 of target image 2000 is shown, the light of which will be delivered to the observer's right eye. As the skilled reader will appreciate, the observer's brain (or alternatively, a processor in a non-human viewing system) can combine the image content received from each eye (or other entrance pupil) so as to "see" both regions simultaneously. However, due to the different perspectives of the observer's two eyes, at least in some cases, the image content contained in each region will actually be slightly different for each eye.

[0331] According to an embodiment, according to the present disclosure, the "target image" for which a hologram is calculated and holographically reconstructed is different for each eye. Therefore, the hologram corresponding to the image calculated for each eye can actually be a hologram of a different image. In other words, the hologram calculated for the left eye is a hologram corresponding to the image when viewed from the left eye's perspective, and conversely, the hologram for the right eye is a hologram corresponding to the image when viewed from the right eye's perspective.

[0332] Thus, the inventors have recognized that the image content contained within each region can be different for each eye (or for each aperture of any other multi-aperture viewing system). Thus, according to embodiments, the waveguide apertures disclosed herein can be configured to provide light from all regions to each eye within a narrow time window, such that the brain (or a processor associated with a non-human viewing system) perceives each eye as receiving all holographic content for its corresponding image substantially simultaneously.

[0333] Depending on the embodiment, the waveguide aperture may be dynamically configurable, and / or more than one waveguide aperture or other control device may be provided, wherein a suitable controller may dynamically control switching between different corresponding configurations of the waveguide aperture, and / or switching between different control devices. Preferably, this should be done very quickly, for example, faster than the typical integration time of the human eye. Such switching enables all spatially modulated light emitted by the control device in two phases to be received by an observer within a very short time window, such that the observer perceives it as being received simultaneously.

[0334] Figure 23A The image required by the observer's left eye is shown, which is divided into four areas, labeled L1, L2, L3 and L4. Figure 23B The image desired for the observer's right eye is shown, divided into four regions, labeled R1, R2, R3, and R4. To ensure that both eyes receive all desired image content within a short time window, but that no image content is received by both eyes at exactly the same time and angle, a control device, such as a waveguide aperture, is provided, which is configured to dynamically switch between a first phase and a second phase that complement each other. Figure 23C shows the target image for hologram calculation for phase 1, Figure 23D The target image for hologram calculation of phase 2 is shown. That is, Figure 23C The target image in is used to calculate the first hologram reconstructed during phase 1, while Figure 23D The target image in is used to calculate the second hologram reconstructed during phase 2. The target image used ensures that each eye receives content appropriate for its image position / viewing angle.

[0335] from Figures 24A to 24CThe first phase can be more fully understood. The waveguide aperture 2450 is arranged in a first configuration 2400A in the first phase, having three open portions 2451, 2453, 2455 and two closed portions 2452, 2554. The waveguide aperture 2450 is provided in a display system or light engine, which is similar to Figure 22A A system is shown in which the waveguide aperture 2450 is tilted relative to the plane of the entrance pupil defined by the observer's left eye 2406 and right eye 2408 so that the waveguide aperture 2450 is substantially parallel to the waveguide 2404, and the waveguide 2450 receives spatially modulated light from the waveguide 2404. Other aspects of the display system are not shown, such as a display device displaying one or more holograms calculated as described herein. In a first phase, the first open portion 2451 allows light from area 1 of the right eye view image to reach the right eye, thereby receiving image content "R1". The second open portion 2453 allows light from area 3 of the right eye view image to reach the right eye, thereby receiving image content "R3". The second open portion 2453 also allows light from area 2 of the left eye view image to reach the left eye - thereby receiving image content "L2". Finally, the third open portion 2455 allows light from area 4 of the left eye view image to be received by the left eye, thereby receiving image content "L4". Therefore, in the first phase, Figure 24B shows the image content received by the right eye 2408, while Figure 24C The image content received by the left eye 2406 is shown.

[0336] from Figures 25A to 25C The second phase can be more fully understood. The waveguide aperture 2450 is in the second phase 2400B, having two open portions 2452', 2454' and three closed portions 2451', 2453', 2455'. In addition, other aspects of the display system will appear in practice but are not described in detail. Figure 25A . In the second phase, the first open portion 2452' allows light from area 2 of the right-eye view image to reach the right eye, thereby receiving image content "R2". The first open portion 2452' also allows light from area 1 of the left-eye view image to reach the left eye, thereby receiving image content "L1". The second open portion 2454' allows light from area 4 of the right-eye view image to reach the right eye, thereby receiving image content "R4". Finally, the second open portion 2454' also allows light from area 3 of the left-eye observation image to be received by the left eye, thereby receiving image content "L3". Therefore, in the second phase, Figure 25B shows the image content received by the right eye 2408, while Figure 25C The image content received by the left eye 2406 is shown.

[0337] The waveguide aperture 2450 can be controlled to switch rapidly between the first and second phases, for example, faster than the typical integration time of the human eye, so that the corresponding image content (e.g., Figure 23C and 23D The images are interlaced with each other (as shown in the combination of the two). The observer thus perceives that they are seeing a complete image with both eyes, each from its own unique perspective depending on its position. However, the observer does not perceive image degradation or experience any confusion that would result from actually receiving common or overlapping image content with both eyes at the same time and angle. Thus, as described herein, the control device works in conjunction with the computed hologram to create a clear and accurate holographically constructed image for the observer. It does this in a simple and effective manner.

[0338] The inventors have found that the above working embodiment can be adequately determined based on the geometry of each entrance pupil center. Figures 26 to 30 In further improvements described herein, the inventors take into account the finite size of the entrance pupil. According to these embodiments, image quality is further improved, and image crosstalk between multiple observation systems is further reduced or even eliminated. Those skilled in the art will appreciate how the following methods can be used to dynamically reconfigure the control device and aperture arrangement in real time in response to, for example, changes in eye or head position and / or pupil size. The aperture configuration provided by the control device is software reconfigurable, so the system disclosed herein can also adjust itself in response to parameters of the observation systems, such as the separation between two observation systems (e.g., the interpupillary distance of a human observer).

[0339] In some embodiments, pupil diameter is measured (e.g., by an eye tracking system) and this is used as part of the shutter control system. For example, the pupils of a driver can vary greatly. When the ambient light is brighter, the pupil diameter will be smaller. An advantage of the system disclosed herein is that in bright conditions, when the pupil diameter is relatively small (e.g., 2 mm), less "closed" aperture area is required to eliminate eye crosstalk. This is advantageous because it means that the overall efficiency (the amount of light transmitted to the observer) is relatively high. Therefore, there is an excellent synergy between the shutter system of the present disclosure and image displays, especially heads-up displays in vehicles.

[0340] Figures 26 to 30A more complex shutter scheme according to another embodiment is shown. These figures show how the size and position of the shutter area changes during operation of the dynamic shutter. Any number of different shutter schemes can be implemented to ensure that no part of the image is received by multiple viewing positions (e.g., eye positions) at the same time. Again, this is achieved by ensuring that each light angle from the waveguide pupil expander reaches only one viewing position (e.g., one eye) at any one time. Figures 28 to 30 By way of example only, an embodiment is shown in which three different shutter configurations are implemented sequentially (eg, cycled) during operation.

[0341] Figure 26 Shown is the corresponding Figures 28 to 30 The geometric configuration of the three-phase dynamic shutter scheme is shown. Figure 26 A spatial light modulator 2600 is shown that displays a hologram of one or more images (e.g., according to Figure 23C or two different partition images of 23D). Figure 26 Also shown is a waveguide pupil expander 2604, which in this example is in the form of a rod. Waveguide pupil expander 2604 is arranged to receive spatially modulated light from spatial light modulator 2600. The received light is spatially modulated according to the displayed hologram. A control device 2650, in the form of a dynamic shutter, receives light from the output face of waveguide pupil expander 2604. As described above, waveguide pupil expander 2604 effectively replicates the received light, causing it to be emitted at multiple points along its length.

[0342] Some of the light that passes through control device 2650 reaches one of the multiple viewpoints. In this example, there are two viewpoints. A first observation system includes a first entrance pupil 2609A and a first light sensor 2611A located at the first viewpoint. A second observation system includes a second entrance pupil 2605B and a second light sensor 2607B located at the second viewpoint.

[0343] Figure 26 An example shutter configuration is shown, wherein five discrete shutter regions are provided by way of example only. Each shutter region can be independently controlled to transmit or not transmit (e.g., absorb or reflect) light received from a corresponding sub-region of the rod. As will be understood below, the control device is reconfigurable such that the number, size, and position of the shutter regions can be varied among a set of shutter configurations designed to transmit the entire image content to the corresponding viewing position. As will be understood from the previous embodiments, the image content transmitted to a first viewing position by a control region is not necessarily the same as the image content transmitted to a second viewing system by the control device.

[0344] In this embodiment, control device 2650 includes, in sequence, a first shutter area 2651, a second shutter area 2652, a third shutter area 2653, a fourth shutter area 2654, and a fifth shutter area 2655. In this embodiment, first light 2662A passing through second shutter area 2652 of control device 2650 and second light 2664A passing through fourth shutter area 2654 of control device 2650 reach the first observation system. Third light 2664B passing through fourth shutter area 2654 of control device 2650 reaches the second observation system. As will be understood from the foregoing description, the system is configured so that light at a specific angle is transmitted to the observation position at a specific time. Each light angle corresponds to a different portion of the image, even though it is encoded in the holographic domain. No light angle is transmitted to both or all observation systems (e.g., the eyes) simultaneously.

[0345] In Figure 26 In this embodiment, the spatial light modulator 2600, waveguide pupil expander 2604, control device 2650, and first and second observation systems are substantially coplanar. The optical axis of the system lies in a single plane. A straight line connects the center of the control device 2650 to the midpoint between the first and second observation systems. In this embodiment, the waveguide pupil expander and control device are substantially elongated, and an angle is defined between the long axis of the rod and the long axis of the control device. This angle lies in a plane containing the optical axis of the system.

[0346] exist Figure 26 In the configuration shown, the second shutter region 2652 and the fourth shutter region 2654 of the control device are operated to be transmissive or at least substantially transmissive, while the first, third and fifth shutter regions 2651, 2653 and 2655 are operated to be non-transmissive or at least substantially non-transmissive.

[0347] In this embodiment, due account has been taken of the finite size of the entrance pupil associated with each observation system (eg, each eye) of the pair of observation systems. Figure 27 is with Figure 18 Similar style diagram, but corresponding to Figure 26 The geometric configuration and entrance pupils 2609A and 2605B represent a pair of human eyes with an interpupillary distance and a finite pupil size. Figure 27 The angle to each viewing system / eye center is shown as a function of position along the waveguide. Figure 27 Three lines are drawn for each eye. Line 2703 corresponds to the center of entrance pupil 2609A, lines 2701 and 2705 correspond to two opposite edges of entrance pupil 2609A on the system plane. Line 2709 corresponds to the center of entrance pupil 2605B, and lines 2707 and 2711 correspond to two opposite sides of entrance pupil 2605B on the system plane.

[0348] Based on this geometry, the inventors have identified a dynamic shutter scheme that eliminates crosstalk between two eyes with limited pupil sizes. In this embodiment, the dynamic shutter scheme includes three phases, configurations, or stages. That is, three different shutter arrangements / patterns are used to reconstruct each hologram. The three shutter arrangements are formed in a temporal sequence. Therefore, it can be said that the different arrangements are temporally interleaved. However, the present disclosure is not limited to three different shutter arrangements, and any number of different shutter arrangements are contemplated within the scope of the present disclosure.

[0349] Figure 28 A first phase is shown, in which only a first region 2850 of the control device is configured to be non-transmissive. Figure 29 A second phase is shown, in which only the second region 2950 of the control device is configured to be non-transmissive. The boundary of the first region 2850 is at Figure 29 The first region 2850 and the second region 2950 are formed at different times, but partially overlap (in space). Figure 30 A third phase is shown, wherein the third region 3051 and the fourth region 3052 of the control device are configured to be non-transmissive. The boundary of the second region 2950 is at Figure 30 The illustration is for comprehension purposes only. The reader should understand that the total area of the control device that blocks / does not transmit or does not block / transmit is not constant during the shutter phase cycle.

[0350] The control device is described herein as a "waveguide aperture", but any suitable control device may be used to provide the functionality described herein. A control device such as a waveguide aperture may be formed from any suitable material. For example, it may comprise a liquid crystal device or a plurality of liquid crystal devices, such as an array, each liquid crystal device being switchable between opaque and transmissive. For example, the control device may comprise "smart glass" or "switchable glass" whose light transmission properties may be changed when voltage, light or heat is applied. The control device may be controlled by any suitable processor or controller. Its configuration may be rapidly changed in order to coordinate or synchronize with the dynamic display of multiple different holograms on a display device, for example to reconstruct different corresponding target images and / or to accommodate movement of an observer or viewing system.

[0351] The control devices disclosed herein can take many different forms. In some embodiments, the control device comprises a plurality of individually controllable light receiving / processing elements, such as pixels, in a 2D array, for example. In some embodiments, the control device comprises a pixelated liquid crystal device or display. In some embodiments, the elements or pixels can be operated in sequential groups to form transmissive and non-transmissive shutter regions as disclosed herein. Each group of pixels can be switched between a first mode, such as transmissive, and a second mode, such as reflective. Those skilled in the art are familiar with how to control pixelated display devices so that, during operation, the size and position of groups or regions of pixels can be varied, for example in real time, with each region having a different response to light. Each region is larger than the device's pixel size. Thus, each region can include multiple pixels. Those skilled in the art are also familiar with how to implement optical components, such as polarizers and wave plates, in conjunction with pixelated liquid crystal devices to provide a reconfigurable light shutter. By way of example only, the control device can utilize polarization selection, but other approaches based on other characteristic properties of light are equally applicable. In some embodiments, the control device comprises a pixelated liquid crystal display and, optionally, other optical elements, collectively configured to transmit light having a first polarization and absorb or reflect light having a second polarization, optionally where the first and second polarizations are opposite or complementary. For the avoidance of doubt, any number of different optical systems may be used to form the control device, depending on the characteristics of the light forming the image, such as polarization and wavelength, and thus the present disclosure is not limited by the construction of the control device. Therefore, it should be understood that the control device disclosed herein is defined by its function rather than its structure.

[0352] The control device is dynamically reconfigurable. In some embodiments, the control device is pixelated. That is, the control device includes an array of individually controllable pixels. Each pixel may include, for example, a liquid crystal that is configurable between a transmissive state and a non-transmissive state. Any alignment defects between the edge of a pixel identified in accordance with the present disclosure and the edge of an ideal aperture area can be handled by letting too much or too little light through. That is, by "turning on" another row of pixels or "turning off" another row of pixels in the area configuration. In embodiments including three or more aperture phases (for example, when the finite size of each entrance pupil is fully accounted for), the control device has sufficient resolution or number of pixels so that the pixels at the interface are always able to block light.

[0353] The examples described above should not be considered limiting. For example, the observation system may have more than two observation apertures or entrance pupils. For example, the image (and accordingly, the calculated hologram and the resulting holographic reconstructed image) may be divided into any number of regions, thereby generating any number of corresponding angular channels of spatially modulated light. For example, the control device has been described as switching between a first and a second phase, but it may be configured to switch between more than two phases. For example, the control may have four phases. According to one embodiment, one phase (e.g., each phase) of the control device may cause light to be transmitted to only one eye (or observation aperture). According to one embodiment, one phase (e.g., each phase) of the control device may cause light to be transmitted to only more than one eye (alternatively, more than one observation aperture).

[0354] The number and / or size of regions receiving image content need not be the same for each phase. In other words, some phases of the control device may deliver more image content than some other corresponding phases. Similarly, when multiple phases of the control device are interleaved, the two eyes do not need to receive the same amount of image content in each phase or overall. For example, depending on their relative positions and / or other factors, one eye may see more image content than the other eye.

[0355] A display system including a control device such as a waveguide aperture as described herein can be configured to display multiple different images one after another and / or at different corresponding times. Thus, the display device in such a system can be configured to display different corresponding holograms, sometimes in rapid succession. Different images may have different numbers of regions. Furthermore, the size of the regions may be different between different corresponding images. Similarly, the rays of light of a region of one image may be defined by different angles relative to rays of light of a region of a different second image. The control device can be configured to be dynamically adjustable to accommodate changes in the number and / or size of the regions. In other words, the precise location along the waveguide (P WG ) may not be fixed at the precise position at which the control device changes from open to closed and vice versa. Instead, the control device may be configured to dynamically change the position along the waveguide (P) at which the control device changes from open to closed. WG ). Thus, the total number of open and closed sections provided by the control device can vary. The individual dimensions of one or more of these sections can also vary.

[0356] The system can be configured to display a sequence of images, such as a video-rate sequence of images. Each image can correspond to a frame in a sequence of frames having a frame rate, such as 50 or 60 Hz. Each frame can include multiple subframes. For example, the subframe rate can be 4 or 8 times the frame rate. The displayed hologram can change with each successive subframe. Each subframe can be considered a separate display event. Each subframe can correspond to an image or at least a portion of an image. While embodiments have been described as delivering light to both eyes during each display event, the present disclosure is not limited thereto. For example, the light engine can be configured to deliver light to only one eye / entrance pupil during each display event. The configuration of the waveguide aperture (i.e., the size and / or distribution of open and closed apertures / openings) can change during each display event or every n display events, where n is an integer. Similarly, while the described embodiments have shown adjacent image content being delivered through each aperture / opening during a display event, the present disclosure is not limited to this scenario, and the image content delivered through each aperture may not be adjacent image content. In some embodiments, only one angular range of light is delivered to one eye per display event / aperture configuration. In some embodiments, the control system is configured to deliver light to each eye / entrance pupil in turn.

[0357] As previously described in this disclosure, a hologram of a target image can be calculated for a specific size and position of a viewing aperture, such as the size and position of the entrance pupil of an observer's eye. If constraints such as the entrance pupil diameter or position change, the hologram can be recalculated, even if the target image to be reconstructed (and therefore the image content that the observer will see or perceive) remains unchanged. Each hologram does not need to have the same number or size of regions, even if two holograms represent the same target image.

[0358] Additional Features

[0359] The embodiments relate to electrically activated LCOS spatial light modulators by way of example only. The teachings of the present disclosure may equally be implemented on any spatial light modulator capable of displaying computer-generated holograms according to the present disclosure, such as any electrically activated SLM, optically activated SLM, digital micromirror device, or microelectromechanical device.

[0360] In some embodiments, the light source is a laser, such as a laser diode.

[0361] The system of the present disclosure can be used to provide an improved head-up display (HUD) or head-mounted display. In some embodiments, a vehicle is provided that includes a holographic projection system installed in the vehicle to provide a HUD. The vehicle can be a motor vehicle, such as a car, truck, van, delivery truck, motorcycle, train, airplane, boat, or ship.

[0362] While the examples describe illuminating the SLM with visible light, those skilled in the art will appreciate that the light source and SLM can also be used to direct infrared or ultraviolet light, such as disclosed herein. For example, those skilled in the art will be aware of techniques for converting infrared and ultraviolet light into visible light for the purpose of providing information to a user. For example, the present disclosure extends to the use of phosphors and / or quantum dot technology for this purpose.

[0363] The methods and processes described herein may be embodied on a computer-readable medium. The term "computer-readable medium" includes media arranged to store data temporarily or permanently, such as random access memory (RAM), read-only memory (ROM), buffer memory, flash memory, and cache memory. The term "computer-readable medium" should also be taken to include any medium or combination of media that can store instructions for execution by a machine, such that when the instructions are executed by one or more processors, the machine performs, in whole or in part, any one or more of the methods described herein.

[0364] The term "computer-readable medium" also encompasses cloud-based storage systems. The term "computer-readable medium" includes, but is not limited to, one or more tangible and non-transitory data storage repositories (e.g., data volumes) in the example form of solid-state memory chips, optical disks, magnetic disks, or any suitable combination thereof. In some example embodiments, instructions for execution may be conveyed by a carrier medium. Examples of such carrier media include transient media (e.g., a propagated signal conveying the instructions).

[0365] It will be apparent to those skilled in the art that various modifications and variations can be made without departing from the scope of the appended claims. This disclosure covers all modifications and variations within the scope of the appended claims and their equivalents.

Claims

1. A light engine arranged to form an image visible from a viewing window, wherein The light engine includes: a display device arranged to display a hologram of an image and to spatially modulate light according to the hologram, wherein the hologram is configured to angularly distribute the spatially modulated light of the image according to the location of the image content such that angular channels of the spatially modulated light correspond to respective consecutive regions of the image; a waveguide pupil expander arranged to receive the spatially modulated light and to provide a plurality of different light propagation paths for the spatially modulated light from the display device to the viewing window; and A control device is arranged between the waveguide and the observation window, wherein the control device includes at least one aperture, and the aperture is arranged so that a first observation position in the observation window receives a first light channel spatially modulated by the hologram according to the first area of the image, and a second observation position in the observation window receives a second light channel spatially modulated by the hologram according to the second area of the image.

2. The light engine according to claim 1, wherein: The first region and the second region are adjacent regions of the image.

3. The light engine according to claim 1 or 2, wherein: Adjacent angular channels of the spatially modulated light correspond to adjacent areas of the image.

4. The light engine according to claim 1 or 2, wherein: The first region and the second region of the image do not substantially overlap.

5. The light engine according to claim 1 or 2, wherein: The image is a virtual image perceived upstream of the display device.

6. The light engine according to claim 1 or 2, wherein: The control device is coupled to the output face of the waveguide pupil expander.

7. The light engine according to claim 1 or 2, wherein: The control device limits the area of the waveguide output face visible from the viewing window.

8. The light engine according to claim 1 or 2, wherein: The waveguide pupil expander and the viewing window are not parallel.

9. The light engine according to claim 1 or 2, wherein: The control device comprises at least one opening, wherein each opening provides a respective spatially modulated light channel to the first viewing position and / or the second viewing position such that different image contents are delivered to the first and second viewing positions, respectively, substantially simultaneously.

10. The light engine of claim 9, wherein: The corresponding channels do not overlap but are continuous.

11. The light engine of claim 9, wherein: The control device is configured such that each opening is switchable between an open position and a closed position, thereby providing a plurality of different control device configurations, wherein each control device configuration includes an alternating sequence of open and closed openings.

12. The light engine of claim 11, wherein: The control device is configured to provide a first control device configuration at a first time and to provide a second control device configuration at a second time, wherein the first control device configuration and the second control device configuration are complementary.

13. The light engine of claim 12, wherein: The time interval between the first time and the second time is shorter than the integration time of the human eye.

14. The light engine of claim 1, wherein: A first control device configuration provided by the control device transmits light modulated according to first and third image regions of the image to the first observation position, and transmits light modulated according to second and fourth image regions of the image to the second observation position, wherein the first to fourth regions are ordered continuous regions of the image.

15. The light engine of claim 1, wherein: A second control device configuration provided by the control device transmits light modulated according to the second and fourth image regions of the image to the first viewing position and transmits light modulated according to the first and third image regions of the image to the second viewing position.

16. The light engine of claim 14 or 15, wherein: The image content of any one of the first to fourth image areas when transferred to the first observation position is different from the image content of the corresponding one of the first to fourth image areas when transferred to the second observation position.

17. The light engine of claim 1 or 2, wherein: The first and second viewing positions are first and second eye positions of an observer, and the viewing window is an eye box.

18. The light engine of claim 9, wherein: In the control device, the size and / or position of at least one opening is dynamically variable.

19. The light engine of claim 9, wherein the control device comprises a plurality of openings.

20. The light engine of claim 1 or 2, wherein the waveguide pupil expander comprises a plurality of transmission points, and wherein, Each of the plurality of different light propagation paths is transmitted from a different respective transmission point.

21. A method of controlling light propagation in a light engine to form an image visible through a viewing window, wherein the light engine comprises a display device, a waveguide pupil expander, and a viewing system including the viewing window; the method comprising: Displaying a hologram of an image on a display device; illuminating a display device to spatially modulate light according to a hologram, wherein the hologram is configured to angularly distribute the spatially modulated light of the image according to the location of the image content such that angular channels of the spatially modulated light correspond to respective consecutive regions of the image; a waveguide pupil expander arranged to receive the spatially modulated light and to provide a corresponding plurality of different light propagation paths for the spatially modulated light from the display device to the viewing window; as well as controlling propagation of the plurality of different light propagation paths using a control device disposed between the waveguide and the viewing window, wherein the control device includes at least one aperture; Among them, the step of controlling the propagation of multiple different light propagation paths includes configuring a control device so that a first observation position within the observation window receives a first light channel spatially modulated by the hologram according to a first area of the image, and a second observation position within the observation window receives a second light channel spatially modulated by the hologram according to a second area of the image.

22. The method of claim 21 further comprising computing a hologram of the image.

23. The method according to claim 21 or 22, wherein The step of configuring the control device includes allowing light to pass through a first portion of the control device and preventing light from passing through a second, different portion of the control device.

24. The method of claim 21 or 22, wherein: The waveguide pupil expander comprises a plurality of transmission points, and wherein each of the plurality of different light propagation paths is transmitted from a different respective transmission point.

25. The method of claim 24, further comprising: i. identifying a first transmission point from which, in the absence of the control device, the light of the first angular channel will propagate to the first observation position; ii. identifying a different second transmission point from which the light of the first angular channel would propagate to the second observation position in the absence of a control device; as well as iii. Configuring the control device to block the light path from the first angular channel to the first observation position or the light path from the first angular channel to the second observation position at a selected time (t).

26. The method of claim 25, wherein: Step (iii) includes alternately blocking the light path from the first angular channel to the first observation position and blocking the light path from the first angular channel to the second observation position during selected time periods.

27. The method of claim 21 or 22 further includes identifying multiple continuous regions within the image, wherein each continuous region corresponds to light from a different corresponding angular channel, and configuring the control device to allow light from a first subset of continuous regions within the multiple continuous regions to be transmitted only to the first observation position within a selected time period, and to allow light from a different second subset of continuous regions within the multiple continuous regions to be transmitted only to the second observation position.

Images