Image projection

Abstract

A diffractive structure arranged to spatially modulate light that can be converted by an observation system into an object image. The diffractive structure is configured to produce a plurality of discrete light patterns. Each light pattern corresponds to a different portion of the object image. The shape of each discrete light pattern substantially corresponds to the shape of an entrance aperture of the observation system.

Description

Technical Field

[0001] This disclosure relates to image projection. More specifically, this disclosure relates to diffraction structures and methods for determining diffraction structures such as holograms or phase holograms. Some embodiments relate to real-time hologram computation based on eye-tracking information. Some embodiments relate to virtual image projection. Other embodiments relate to real image projection. Embodiments relate to observing projected images via waveguides. Some embodiments relate to optical engines such as image generation units. Some embodiments relate to head-up displays. Background Technology

[0002] Light scattered from an object contains amplitude and phase information. This amplitude and phase information can be captured on, for example, a photosensitive plate using well-known interferometry techniques to form a holographic record, or "hologram," including interference fringes. The hologram can be reconstructed by illuminating it with appropriate light to form a two-dimensional or three-dimensional holographic reconstruction or replay image representing the original object.

[0003] Computer-generated holography can numerically simulate interference processes. Computer-generated holograms can be calculated using techniques based on mathematical transformations such as Fresnel or Fourier transforms. These types of holograms are called Fresnel / Fourier transform holograms or simply Fresnel / Fourier holograms. A Fourier hologram can be considered a Fourier domain / plane representation of an object or a frequency domain / plane representation of an object. For example, computer-generated holograms can also be calculated using coherent ray tracing or point cloud techniques.

[0004] Computer-generated holograms can be encoded on spatial light modulators arranged to modulate the amplitude and / or phase of incident light. For example, optical modulation can be achieved using electrically addressable liquid crystals, optically addressable liquid crystals, or micromirrors.

[0005] Spatial light modulators typically comprise multiple individually addressable pixels, which may also be referred to as cells or elements. The light modulation scheme can be binary, multilevel, or sequential. Alternatively, the device can be sequential (i.e., excluding pixels), so the light modulation can be continuous on the device. Spatial light modulators can be reflective, meaning the modulated light is output as reflected light. Spatial light modulators can also be transmissive, meaning the modulated light is output as transmitted light.

[0006] The system described herein can be used to provide holographic projectors. For example, such projectors have already been used in head-up displays (HUDs) and light detection and ranging (LiDAR). Summary of the Invention

[0007] The aspects of this disclosure are defined in the appended independent claims.

[0008] For ease of explanation and illustration, this disclosure and the accompanying drawings generally show a one-dimensional case. However, those skilled in the art of optics will understand that the concepts described and illustrated can be extended to two dimensions to provide a two-dimensional image from a two-dimensional hologram. For example, although only one-dimensional pupil extension is described and illustrated, the reader should understand that this disclosure extends to two-dimensional pupil extension, for example, using two one-dimensional pupil extenders in series.

[0009] This disclosure relates to image projection. It relates to methods of image projection and image projectors including display devices. This disclosure also relates to projection systems including image projectors and observation systems. This disclosure is equally applicable to monocular and binocular observation systems. An observation system may include one or more eyes of an observer. An observation system includes optical elements (e.g., the lens of the human eye) with light power and an observation plane (e.g., the retina of the human eye). The projector may be referred to as a "light engine." The display device and the image formed (or perceived) using the display device are spatially separated from each other. An observer forms or perceives the image on the display plane. In some embodiments, the image is a virtual image, and the display plane may be referred to as a virtual image plane. The image is formed by illuminating a diffraction pattern (e.g., a hologram) displayed on the display device. This disclosure also relates to providing (e.g., calculating) a diffraction pattern for image projection, and relates to the diffraction pattern.

[0010] Display devices consist of pixels. The pixels of a display device diffract light. According to well-known optical principles, the magnitude of the maximum diffraction angle depends on the pixel size (and other factors, such as the wavelength of light).

[0011] In this embodiment, the display device is a spatial light modulator, such as a liquid crystal on silicon (“LCOS”) spatial light modulator (SLM). Light travels from the LCOS to the observing entity / system, such as a camera or eye, within a diffraction angle range (e.g., from zero to the maximum diffraction angle). In some embodiments, amplification techniques may be used to increase the range of available diffraction angles beyond the conventional maximum diffraction angle of the LCOS.

[0012] In this embodiment, the image is a real image. In other embodiments, the image is a virtual image perceived by the human eye (or multiple eyes). The projection system or light engine can therefore be configured so that the observer looks directly at the display device. In such an embodiment, holographically encoded light propagates directly to the eye. This light may be referred to as “spatially modulated light” or “holographic light.” In such an embodiment, no intermediate holographic reconstruction is formed in the free space between the display device and the observer, or on the screen or other light-receiving surface. In such an embodiment, the pupil of the eye can be considered as the entrance aperture of the observation system, and the retina of the eye can be considered as the observation plane of the observation system. It is sometimes said that in this configuration, the lens of the eye performs the holographic-to-image conversion.

[0013] According to well-known optical principles, the range of angles at which light propagating from a display device or viewing window can be observed by the eye or other observation entity / system varies with the distance between the display device and the observation entity. For example, at an observation distance of 1 meter, only a small range of angles from the LCOS can propagate through the pupil of the eye to form an image on the retina at a given eye location. The range of angles at which light propagates from the display device determines the portion of the image that the observer can successfully propagate through the pupil of the eye to form an image on the retina at a given eye location. In other words, not all parts of the image are visible from any point on the observation plane (e.g., any eye position within the viewing window of an eye-tracking box).

[0014] In some embodiments, the image perceived by the observer is a virtual image appearing upstream of the display device; that is, the image perceived by the observer is farther away than the display device. Conceptually, multiple different virtual image points can be considered. The distance from this virtual image point to the observer is referred to here as the virtual image distance. Of course, different virtual points can have different virtual image distances. Individual rays in the ray beam associated with each virtual point can reach the observer via different corresponding optical paths through the display device. However, only some portions of the display device, and therefore only some rays from one or more virtual points of the virtual image, may be within the user's field of view. In other words, only some rays from some virtual points on the virtual image will propagate through the display device to the user's eyes and thus be seen by the observer. Therefore, conceptually, it can be considered that the observer is viewing the virtual image through a "window the size of the display device," which can be very small, for example, 1 cm in diameter, at a relatively large distance, such as 1 meter. And the user will view the window the size of the display device through the pupils of their eyes, which can also be very small. Therefore, at any given time, the field of view shrinks, and the specific range of angles that can be seen heavily depends on the eye position.

[0015] It is generally desirable for optical systems to be small in physical size—for example, for use in locations with limited space and / or high real estate value. However, physical constraints are often associated with functional limitations. For example, in conventional optical systems, the use of small display devices is often associated with a limited field of view (FOV), thus limiting the visibility of the image. This disclosure addresses the technical problem of how to increase the field of view—that is, how to increase the angular range of light propagating from the display device when the display device is (relatively) small and the projection distance is (relatively) large, and that the light can successfully propagate through the pupil of the eye to form an image. In some embodiments, the projection distance is at least one (e.g., at least two) orders of magnitude larger than the diameter or width of the aperture of the display device (i.e., the size of the pixel array). More specifically, this disclosure addresses the technical problem of how to do this using so-called direct-view holography, in which a hologram of an image, rather than the image itself, is propagated to the human eye. In other words, the light received by the observer is “holographic light” that has been spatially modulated according to a hologram of the image.

[0016] A pupil expander is used to extend the field of view, thus increasing the maximum propagation distance usable across the full diffraction angle of the display device. The use of a waveguide also laterally enlarges the user's eyebox, allowing for some eye movement while still enabling the user to see the image. In this embodiment, the pupil expander is a waveguide pupil expander. This disclosure generally (but not exclusively) relates to non-infinite virtual image distances, i.e., near-field virtual images.

[0017] The inventors have discovered that in conventional holography for non-infinite virtual image distances (i.e., near-field virtual images), a so-called "ghosting image" occurs. This is likely caused by different possible light propagation paths through the waveguide. The ghosting image is a low-brightness copy of the main image. The main image with the highest intensity can be referred to as the primary image. Each ghosting image can be referred to as a secondary image. The presence of ghosting images significantly reduces the quality of the perceived virtual image. Ghosting images may make the primary image appear blurry.

[0018] Various aspects of this disclosure relate to different methods for resolving problems caused by ghosting images. Some solutions disclosed herein have demonstrated the successful elimination or prevention of ghosting image formation. Some solutions disclosed herein have demonstrated the modification / manipulation of ghosting images to enhance or strengthen the primary / non-ghosting image.

[0019] According to various aspects of this disclosure, a light engine is arranged to provide spatially modulated light to an observation system having an entrance pupil. The display system includes a display device arranged to display a hologram and spatially modulate light based on the hologram. The display system also includes a holographic engine arranged to output a hologram for display on the display device, enabling the observation system to see or perceive an image of a target image or object at a specific location. A pupil expander is provided and can be arranged relative to the display device to ensure that the observation system receives light corresponding to the entire image (i.e., provides a maximum field of view) without changing the position of the entrance pupil.

[0020] This document discloses a method for determining the diffraction structure of an image for a system including a display device and an observation system. The diffraction structure may be a hologram, and the term "hologram" is used hereafter only as an example of a diffraction structure according to this disclosure. The diffraction structure may be a composite hologram, a pure phase hologram, or a phase hologram. The display device is arranged to display the hologram. The observation system is arranged to observe the hologram through a pupil expander. The pupil expander provides multiple light propagation paths from the display device to the observation system.

[0021] According to various aspects of this disclosure, a hologram is provided (e.g., calculated) such that a display device displaying the hologram outputs channels of spatially modulated light when properly illuminated. In at least some arrangements, each channel corresponds to a corresponding consecutive region of an image represented by the hologram. Each channel may correspond to different consecutive portions or regions of the image, such that these channels, when combined, provide holographic light for the entire image. A pupil expander may be provided between the display device and the observation system, arranged to guide each channel to the entrance aperture of the observation system. Each channel may be considered to have a unique, respective central axis that defines the principal (or “core”) direction of travel of the channel relative to the display device, for example, relative to the center point of the emitting surface of the display device or another reference point. Thus, the reader will understand that each channel may be characterized by an axis, which may be considered the optical axis or propagation axis of the channel. Each axis may be characterized by a unique angle relative to the normal of the display device. Each axis may be a straight line connecting the center of the display device (or hologram) to the center of the illuminated area or light pattern of the channel (i.e., the center of the cross-sectional area of ​​the light channel).

[0022] The size and / or shape of the cross-sectional area of ​​one or more channels may correspond to the size and / or shape of the incident aperture of the observation system. For example, if the incident aperture is the human eye, the cross-section of the channel may be substantially elliptical or oval. In embodiments that include hologram calculations, the calculation process may include limiting or cropping the hologram according to the size and / or shape of the incident aperture and / or according to the size and / or shape of the display device.

[0023] In some embodiments, there is no overlap between channels in terms of image content. In other embodiments, there is some overlap between channels in terms of image content. This overlap is partial and relatively small. For example, two adjacent channels may both contain some information about the same part of the image. Therefore, in terms of image content / spatial angle in the field of view, it can be said that the channels may partially overlap.

[0024] Channels can be characterized by angles within the field of view. These angles can be measured starting from the normal of the display device / hologram. Each channel can be characterized by two angles—for example, a first angle in the xz plane and a second angle in the yz plane, where the z-direction is perpendicular to the display device / hologram and represents the general direction of light propagation from the hologram. The x-direction can be horizontal, while the y-direction can be vertical. For example, in the x-direction (horizontal field of view), the first channel could correspond to an angle range of 0 to +4 degrees, and the second angle could involve +3 to +7 degrees. In this example, there is a 1-degree overlap. In this example, both the first and second channels contain information related to the angular content of the image or the angle of the horizontal field of view within the +3 to +4 degree range. Of course, both channels contain additional information. The overlap is relatively small—for example, no more than 25% of the angle range associated with each channel, e.g., no more than 10%.

[0025] In some embodiments, the total angular range (not just the overlapping angular range) associated with each channel is the same. In other embodiments, the angular range associated with one angular channel is different from the angular range of another angular channel.

[0026] This paper discloses a method for computing holograms of an image, comprising at least one step of cropping according to the entrance pupil of an observation system to form a hologram, which, when illuminated, forms spatially modulated light, wherein continuous optical channels of the spatially modulated light correspond to continuous regions of the image. The continuous optical channels can be defined by a continuous range of ray angles of the spatially modulated light. All pixels of the hologram contribute to each channel, such that rays contributing to each channel can be emitted from multiple different pixels of a display device displaying and illuminating the hologram thereon. These rays combine to form continuous channels, wherein each channel has a unique corresponding principal propagation direction relative to the display device. A core or axis can be defined for each channel in the propagation direction. For each individual hologram pixel, light from different corresponding channels will be output from that pixel at different corresponding angles.

[0027] Each consecutive optical channel of spatially modulated light corresponds to a corresponding consecutive region of the image. Spatially modulated light can be divided into any number of consecutive optical channels. In some embodiments, the optical channels do not overlap. In other embodiments, such as those further including an optical combiner with optical power (e.g., a vehicle windshield) between the waveguide and the observer, some optical channels may at least partially overlap. The method disclosed herein determines a diffraction structure arranged to spatially modulate light, which can be converted by an observation system, into an image, wherein the diffraction structure is configured to route light into multiple holographic channels, each holographic channel corresponding to a different portion of the image.

[0028] To avoid confusion, the image formed or perceived by the observer is a holographic reconstruction of the target image. Holographic reconstruction is formed from a hologram based on the target image. In some embodiments, a hologram is determined (e.g., calculated) from the target image.

[0029] Holograms can be computed using any suitable technique. This document discloses various possible hologram computation techniques; however, this disclosure is not limited to the examples provided. According to some embodiments, modeling techniques, such as ray tracing techniques, or so-called “point cloud” hologram computation techniques, can be used to compute holograms. In such embodiments, a holographic engine can be arranged to receive contribution information that identifies contributing and non-contributing regions of a display device based on the position of an entrance pupil. The contributing regions of the display device substantially propagate light through the entrance pupil at a defined location. The non-contributing regions of the display device substantially propagate light blocked by the entrance pupil at the defined location. The contribution information also identifies (i) at least one primary contributing region of the display device that propagates light to the observation system and contributes to the primary image, and (ii) at least one secondary contributing region of the display device that propagates light to the observation system and contributes to the secondary image. The holographic engine is also arranged to determine a hologram based on the at least one primary contributing region of the display device identified by the processing engine. The holographic engine is also arranged to output the hologram to a display device for display.

[0030] By identifying contributing and non-contributing regions of the display device, for a given location of the incident aperture of the observation system, the optical engine can determine which or more parts of the display device can be effectively encoded by a hologram to actively contribute to the formation of the main image. For example, this could correspond to the position of the observer's eye at a given time. Furthermore, the optical engine can determine which parts of the display device cannot propagate light through the incident aperture and are therefore not worth filling with hologram values. Additionally, the optical engine can distinguish between parts of the display device that actively contribute to the "master" target image and parts that contribute to copies / replicas or "ghosting" versions of the main image. Therefore, holograms can be omitted in so-called secondary contributing regions to eliminate ghosting.

[0031] Alternatively, in a significant further improvement, the hologram displayed in the additional contributing region can be determined based on the displacement or modified position of image points (i.e., points within the desired image that will be holographically reconstructed). This modified position may be referred to as a "minor image point," but this is shorthand for its minor (i.e., altered) position as a (primary) image point. In short, the modeled / calculated position of an image point can be modified (e.g., translated on the image plane) such that light propagating from said modified position via the additional contributing region on the display device will reach the desired position on the observation plane to effectively enhance the primary image. Therefore, in this alternative approach, the hologram of the additional contributing region is determined based on the position of the image point that differs from the position used to identify the primary contributing region on the display device. The optical path length from the primary image point to the corresponding image formed on the observation plane is typically different from the optical path length from the minor image point to the corresponding image formed on the observation plane. Thus, it can be said that the hologram determination process associated with the additional contributing region involves translating or moving the image points used in the hologram determination process.

[0032] Therefore, an intelligent and efficient light engine is provided, which can be configured and operated to provide clear and accurate images corresponding to holograms that have been determined in a streamlined and computationally efficient manner.

[0033] A method is provided for determining a hologram for display on a display device. The method includes determining the position of an entrance pupil of an observation system arranged to observe the hologram, and identifying contributing and non-contributing regions of the display device, wherein the contributing regions of the display device substantially propagate light through the entrance pupil of the observation system at the determined position, and the non-contributing regions of the display device substantially propagate light blocked by the entrance pupil of the observation system at the determined position. The method further includes identifying at least one primary contributing region of the display device that provides light contributing to a primary image and at least one secondary contributing region of the display device that provides light contributing to a secondary image; and determining the hologram based on the at least one primary contributing region of the display device.

[0034] A diffraction structure is provided, which is arranged to modulate the light space that can be converted by the observation system into a target image, wherein the diffraction structure is configured to generate multiple discrete light patterns, each light pattern corresponding to a different part of the target image, wherein the shape of each discrete light pattern substantially corresponds to the shape of the incident aperture of the observation system.

[0035] A diffraction structure is provided, which is arranged to modulate the light space into an image that can be converted by an observation system (including lenses), wherein the diffraction structure is arranged to guide light into a plurality of discrete optical channels, wherein each optical channel has a cross-sectional shape substantially corresponding to the entrance pupil of the observation system, and each optical channel substantially corresponds to a different portion of the image.

[0036] A method is provided for determining a hologram for display on a display device and for forming a virtual image perceptible from an observation plane via a waveguide by observing the hologram displayed on the display device. The method includes determining the coordinates [x] of each virtual image point in the virtual image. virtual ,y virtual ,z virtual The method determines the observation position on the observation plane and the number of light reflections B within the waveguide associated with the main image formed by the waveguide. The method also includes tracing the light rays from the virtual image point to the observation plane for the "B"th light reflections within the waveguide, and for the light rays from [x...] virtual ,y virtual ,z virtual The propagation of light with B-order reflection to the observation plane determines the coordinates of the principal ray at the display device. LCOS (B),y LCOS (B)]. The method also includes determining [x] LCOS (B),y LCOS (B)] defines the effective pixels of the display device within the area; and through [x virtual ,y virtual ,z virtual Propagate light waves to the effective pixels to determine a sub-hologram that includes the amplitude and / or phase hologram components of the effective pixels.

[0037] The master ray may include a ray that is determined (e.g., calculated or simulated) as the main or “master” image point that travels from a virtual image point through the display device to a virtual image point on the viewing plane.

[0038] The method may also include combining sub-holograms calculated for two or more corresponding virtual image points to form a hologram.

[0039] The method may also include determining the position of the virtual image point's main image on the viewing plane [x] sensor ,y sensor ].

[0040] The method may also include, for each allowable ΔB value of the waveguide, performing a calculation from [x] for B+ΔB bounces. sensor ,y sensor ] to the virtual image plane z virtual Ray tracing and determining the coordinates of a virtual point [x] virtual (ΔB),y virtual (ΔB),z virtual ], which will image B+ΔB sub-reflections to [x sensor ,y sensor The method may also include targeting [x] virtual (ΔB),yvirtual (ΔB),z virtual The propagation of light with B+ΔB bounces to the observation plane determines the coordinates of the principal ray at the display device. LCOS (B+ΔB),y LCOS (B+ΔB)], and identify [x LCOS (B+ΔB),y LCOS The method may also include additional valid pixels of the display device within the second region (i.e., the additional region) defined by [B+ΔB]. virtual (ΔB),y virtual (ΔB),z virtual The light waves propagate to additional effective pixels to determine an additional sub-hologram, which includes the amplitude and / or phase hologram components of the additional effective pixels.

[0041] According to some embodiments, a hologram can be calculated by determining a composite light field, wherein the method can be iterative. According to one embodiment, the method includes first to fifth stages. The first stage includes determining a first composite light field at the entrance pupil of the observation system. The first composite light field is generated by the propagation of light from the display plane of the display device along at least one light propagation path of the pupil expander. The first stage also includes cropping according to the entrance pupil of the observation system. The second stage includes determining a second composite light field at the sensor plane of the sensor of the observation system. The second composite light field is generated by the propagation of light from the first composite light field from the entrance pupil through a lens of the observation system. The second stage also includes modifying the amplitude component according to the image. The third stage includes determining a third composite light field at the entrance pupil. The third composite light field is generated by the backpropagation of light from the second composite light field from the sensor plane through a lens. The third stage also includes cropping according to the entrance pupil. The fourth stage includes determining a fourth composite light field at the display plane. The fourth composite light field is generated by the propagation of light from the third composite light field along at least one light propagation path of the pupil expander. The fourth stage also includes cropping according to the display device. The hologram is extracted from a fourth dataset. Steps one through four can be repeated iteratively. With each iteration, the hologram converges, may improve, but will tend to plateau. For example, the method can stop when the hologram that can be extracted from the fourth stage is considered to have acceptable quality, or when the rate of change in each iteration is below a threshold, or when the allocated time has expired. To avoid ambiguity, the extracted hologram is the one intended for display on a display device.

[0042] The term "backward propagation" is used only to reflect that the direction of light propagation in the third and fourth stages is different from or substantially opposite to the direction of light propagation in the first and second stages. In this respect, the light propagation in the first and second stages can be referred to as "forward propagation." In some embodiments, "forward propagation" and "backward propagation" are the mathematical inverses of each other.

[0043] As used herein, the term "clipping" refers to the process of selectively discarding information (e.g., light field information) outside the region or range of interest (e.g., outside the optical aperture). In some embodiments, "clipping" is a data processing step that includes discarding data points outside the aperture, zeroing out data points, or simply ignoring data points.

[0044] The term "composite light field" is mentioned here. The term "light field" simply refers to a light pattern of finite size in at least two orthogonal spatial directions (x and y). The term "composite" as used here simply means that the light at each point in the light field can be defined by amplitude and phase values, and therefore can be represented by complex numbers or a pair of values. For holographic computation purposes, the composite light field can be a complex two-dimensional array, where complex numbers define the light intensity and phase at multiple discrete locations within the light field. According to the method disclosed herein, the composite light field propagates forward and backward along the +z and -z directions between the holographic plane and the image plane. Light propagation can be simulated or modeled using any of a variety of different methods or mathematical transformations familiar to those skilled in the art of wave optics.

[0045] The inventors have devised a method for determining holograms for relatively small display devices and projection over relatively long distances, wherein the holograms are projected directly onto an observation system, and the method can be implemented in real time. The relatively small size of the display device and the relatively long projection distance may require a pupil expander. The inventors' method also addresses the optical complexity introduced by using a pupil expander. At least in some embodiments, the method also allows image content to appear at different and / or multiple distances from the observation system, optionally simultaneously, for example, using a single hologram. Furthermore, the method allows image content to appear downstream and upstream of the display device, optionally simultaneously, for example, using a single hologram.

[0046] Importantly, the light from the hologram itself (i.e., holographic light) propagates to the observation system, not the holographic reconstruction (i.e., the image) formed by the hologram. It can be said that the spatially modulated light received by the observation system is in the holographic domain, not in the spatial or image domain. It can also be said that the observation system performs the holographic-to-image conversion. More specifically, optical elements such as lenses of each observation system perform the conversion. In embodiments, no holographic reconstruction or image is formed between the display device and the observation system. In some embodiments, optionally, an interleaving scheme is used to compute different holograms and propagate them to each eye of the observer.

[0047] The display device has an effective or display area with a first dimension less than 10 cm, for example, less than 5 cm or less than 2 cm. The propagation distance between the display device and the observation system can be greater than 1 m, for example, greater than 1.5 m or greater than 2 m. The optical propagation distance within the waveguide can be up to 2 m, for example, up to 1.5 m or up to 1 m. This method is capable of receiving images and determining corresponding holograms of sufficient quality within less than 20 ms, such as less than 15 ms or less than 10 ms.

[0048] The method disclosed herein forms a hologram configured to route light into multiple channels, each corresponding to a different portion (i.e., a sub-region) of an image. The hologram can be displayed (e.g., shown) on a display device such as a spatial light modulator. When displayed on a suitable display device, the hologram spatially modulates the light that can be converted into an image by an observation system. The channels formed by the diffraction structure are referred to herein as “hologram channels” simply to reflect that they are light channels encoded by a hologram containing image information. It can be said that the light in each channel is in the holographic domain, rather than in the image or spatial domain. In some embodiments, the hologram is a Fourier or Fourier transform hologram, and therefore the hologram domain is Fourier or the frequency domain. The hologram can also be a Fresnel or Fresnel transform hologram.

[0049] Holograms are described herein as routing light into multiple holographic channels, merely to reflect that the image reconstructable from the hologram has a finite size and can be arbitrarily divided into multiple image sub-regions, where each holographic channel corresponds to a corresponding image sub-region. Importantly, the holograms of this disclosure are characterized by how they distribute image content when illuminated. Specifically, the hologram divides image content by angles. That is, each point on the image is associated with a unique ray angle or range of angles (at least a unique pair of angles or ranges, since the hologram is two-dimensional) in the spatially modulated light formed by the hologram when illuminated. In other words, each point in the target / desired image can be associated with an optical channel formed by the hologram of the reconstructed image. More specifically, the image content or information associated with each image point is encoded within the corresponding channel. Unless there is some overlap as described above, each image point (or a continuous range of image points forming an image sub-region) is uniquely associated with the corresponding optical channel. Therefore, each part of the image can be associated with an angle (or a pair of angles) defining the axis of the discrete optical channel formed by the hologram. To avoid doubt, this holographic behavior is not conventional. When illuminated, the spatially modulated light formed by this particular type of hologram can be arbitrarily divided into multiple holographic channels. As understood above, any hologram that can be considered in the spatially modulated light will be associated with a corresponding part or sub-region of the image. That is, all the information needed to reconstruct that part or sub-region of the image is contained within the channels of the spatially modulated light formed by the hologram of the image. When the spatially modulated light is viewed as a whole, there is not necessarily any evidence of multiple discrete optical channels. However, in some embodiments, multiple spatially separated holographic channels are formed by intentionally leaving the region of the target image for hologram computation blank or empty (i.e., without image content).

[0050] However, holograms can still be identified. For example, if only one holographic channel (i.e., only one continuous portion or sub-region of spatially modulated light formed by the hologram) is reconstructed, then only one corresponding sub-region of the image should be visible. If different holographic channels (i.e., different continuous portions or sub-regions of spatially modulated light) are reconstructed, then different corresponding sub-regions of the image should be visible. Another identifying feature of this type of hologram is that the shape of the cross-sectional region of any holographic channel substantially corresponds to the shape of the incident pupil (i.e., substantially the same), although the dimensions may differ—at least in the correct plane in which the hologram is computed. Each optical holographic channel propagates from the hologram along the core (or axial) direction. While these are example ways of characterizing or identifying this type of hologram, other methods may also be used. In summary, the holograms disclosed herein are characterized and identified by how the image content is distributed within the holographically encoded light, and the appended claims are stated accordingly.

[0051] This method may include segmenting the image during hologram computation such that the number of optical channels formed by the hologram and therefore the spacing between the optical channels are coordinated with the replication process described herein. In some embodiments, each channel uniquely corresponds to multiple reflections and bounces within the waveguide. For example, a first channel may correspond to zero "bouncings," a second channel may correspond to one "bouncing," and so on.

[0052] The method disclosed herein offers several technological advancements. First, it avoids creating ghosting images that might be produced by other methods. This is because the method inherently ensures that the correct image content arrives at the correct position by fully considering all possible light propagation paths in the waveguide. Second, at least in some embodiments, the method can render image content in any depth plane, unlike other methods (which may be poor at very small image point distances). This is a significant problem in optical systems that utilize optical combiners with optical power (e.g., car windshields) to form virtual images. Third, the method inherently considers the effects of wavelength, eliminating the need for image size correction by wavelength, as disclosed in U.S. Patent 10,514,658, in color projectors that include multiple monochrome holographic channels.

[0053] Different propagation paths of the hologram channels can pass through the entrance aperture of the observation system at different corresponding angles. The pupil expander can be arranged such that all hologram channels are routed through the entrance aperture of the observation system at any observation position on the observation plane. For each permitted observation position, the pupil expander routes each hologram channel to the observation system via only one propagation path. At least two of the multiple hologram channels can partially overlap at the entrance aperture of the observation system.

[0054] In embodiments of the method that include first to fifth stages, the first to fourth stages may be ordered stages. The disclosed method operates by projecting back and forth between an image plane and a hologram, and the method may begin from either the image plane or the hologram plane. The amplitude component of the light field is modified or constrained after each propagation to the image plane or the hologram plane, but the phase component is preserved. In some embodiments, the method begins with a first stage, which is equivalent to starting from the hologram plane. However, in other embodiments, the method begins with a third stage, which is equivalent to starting from the image plane. In these other embodiments, the third stage is followed by a fourth stage. The fourth stage is followed by the first stage, and the first stage is followed by a second stage. Each stage may be performed once before hologram extraction, or at least some stages may be performed multiple times before hologram extraction.

[0055] At least one optical propagation path may include multiple optical propagation paths provided by a pupil expander. The structure of the pupil expander facilitates or allows multiple different possible optical paths to pass through. The different possible optical paths may partially overlap. In some embodiments, a series of different optical paths are created by the pupil expander, wherein each optical path in the series is longer than the previous one. Each optical path in the series exits the pupil expander at a different point on its exit surface to create a corresponding series of light exit points or sub-regions. This series of light exit points or sub-regions may be substantially uniformly spaced along the exit surface of the pupil expander.

[0056] A pupil expander can be a waveguide pupil expander. Each ray entering the pupil expander can be replicated multiple times. The pupil expander can be arranged to propagate light through a series of internal reflections and output light at multiple points along its principal surface. Each light propagation path can be defined by the number of internal reflections within the waveguide associated with that light propagation path. For example, a first light propagation path may include zero internal reflections, thus corresponding to light passing directly through the waveguide. A second light propagation path may include two internal reflections before leaving the waveguide, namely a first reflection at a first principal / reflecting surface of the waveguide and a second reflection at a second principal / reflecting surface of the waveguide, wherein the second principal / reflecting surface is opposite to or complementary to the first principal / reflecting surface. To avoid ambiguity, the light propagation paths therefore have some overlap. In other examples, the first light propagation path includes one reflection, and the second light propagation path includes three reflections. The first light propagation path can be the shortest light propagation path, and the nth light propagation path can be the longest light propagation path. Different propagation paths can pass through the entrance aperture of the observation system at different angles.

[0057] At least one optical propagation path may be just one of multiple optical propagation paths provided by the pupil expander. First through fourth stages can be performed on each of the multiple optical propagation paths to extract a hologram for each path. First through fourth stages can be performed independently for each optical propagation path. Multiple holograms corresponding to the multiple optical propagation paths can be combined to form a hologram for display on a display device.

[0058] It is worth noting that this method considers multiple optical propagation paths through the waveguide by performing steps one through four (regardless of the starting point) for each optical propagation path. Steps one through four can be performed sequentially for each propagation path. Alternatively, step one can be performed for each propagation path, followed by step two, then step three, and so on. It should be understood that, due to the partial overlap of different propagation paths, the steps performed for the nth propagation path can reuse the calculations for the (n-1)th propagation path, where the nth propagation path is the next longest propagation path after the (n-1)th propagation path. Multiple holograms determined for multiple different optical propagation paths can be combined by addition, especially if the hologram is a phase or pure phase hologram.

[0059] The light propagating from the display plane in the first stage may include a zeroth composite light field with random phase components, a quadratic function, or a sampled quadratic function.

[0060] The amplitude component of the zeroth composite optical field can be equal to the amplitude component of the illuminating beam. In some embodiments, the amplitude of the zeroth composite optical field is 1. If the method starts from the first stage, the phase component of the zeroth composite optical field may be random. The random phase distribution is sometimes referred to as a random phase seed, and when the method starts at the holographic plane (i.e., the first stage), it can be used only as the starting point of the method.

[0061] The first through fourth stages can be repeated iteratively before the step of extracting the hologram from the final iteration. For the second and subsequent iterations, the light propagating from the display device may include the phase distribution of the fourth composite light field from the previous iteration.

[0062] If further iterations of the first stage are performed before the method stops (i.e., holograms are acceptable), the phase components from the fourth step are saved, retained, or carried over. That is, the phase component of the composite light field propagating to the display plane according to the first stage is equal to the phase component of the fourth composite light field.

[0063] The hologram can be the phase component of the fourth dataset. The hologram can also be the phase component of the fourth dataset in the final iteration or stage of the method. In some embodiments, the hologram is a phase hologram, a phase hologram, or a pure phase hologram. The amplitude component of the fourth composite optical field can be discarded.

[0064] A hologram can be a hologram of multiple images. Each image can have a different image distance. The second stage of the method can be performed independently for each image. Importantly, the method disclosed herein forms a hologram that can simultaneously form image content on multiple planes. This is achieved by performing the second stage on each different plane and combining the results, for example, by adding composite light fields together. Each image can be a real image or a virtual image. The image content may be visible in front of the display device (i.e., downstream of the display device) and / or behind the display device (i.e., upstream of the display device).

[0065] Each composite optical field is determined by wave propagation optics, such as Fresnel propagation, shifted Fresnel propagation, fractional Fresnel propagation, fractional Fourier transform, or scaled fast Fourier transform.

[0066] The modification of the amplitude components in the second stage may include replacing the amplitude components of the second composite light field with the amplitude components of the image, or weighting the amplitude components of the second composite light field based on the amplitude components of the image.

[0067] Each step of the trimming process may include trimming the composite light field based on at least one of the size and position of the corresponding pupil. At least one of the size and position of the incident pupil may be determined by tracking or monitoring an observation system or by receiving information about the observation system. In embodiments where the observation system is an eye, the method may include eye tracking or head tracking. If at least one attribute of the incident pupil (e.g., position or size) changes, the first through fourth stages disclosed herein may be repeated.

[0068] The image, or each image, may be a virtual image. The image, or each image, may appear to the observation system to be behind or on a distant side of the display device. That is, the image distance from the observation system to the perceived image may be greater than the distance from the observation system to the display device. However, in other embodiments, the image content may additionally or alternatively be formed downstream of the display device, i.e., between the display device and the observation system.

[0069] The observation system can be the observer's eye. The method may also include tracking the observer's eye or head to determine at least one of the size and position of the observation system's entrance pupil. In some embodiments, the size and / or position of the observation system's entrance pupil are used as part of the method for determining the hologram. In some embodiments, if observer movement or, for example, a change in ambient light conditions affects the size of the observer's entrance pupil, the method is performed in real time (e.g., at video rate), and the hologram is re-determined, for example, recalculated.

[0070] Propagation along each light propagation path provided by the pupil expander can include a separate composite light field combining the individual light propagation paths. These individual composite light fields can be combined by addition. Each of the multiple different light propagation paths provided by the pupil expander is considered individually. The composite light field formed by each light propagation path is determined individually.

[0071] A pupil expander can be a waveguide pupil expander. Each light propagation path corresponds to a different number of internal reflections within the waveguide. In some embodiments, the pupil expander is a waveguide pupil expander having a substantially one-dimensional (i.e., elongated) or two-dimensional shape (e.g., substantially planar, such as a plate). In embodiments, the exit pupil expands along the longitudinal direction or dimension of the component. The pupil expander may include a pair of opposing or complementary reflective surfaces. One of these surfaces may be only partially reflective to allow light to exit at a series of light exit points or sub-regions.

[0072] Combining individual composite optical fields can include determining the lateral position of each individual composite optical field on the plane containing the incident pupil. The number of internal reflections within the waveguide determines the lateral position.

[0073] Combining individual composite light fields can also include determining the total phase shift associated with internal reflections for each light propagation path. This can include summing multiple phase shifts associated with each light propagation path, where each phase shift is generated by reflections within the pupil expander.

[0074] A holographic engine is also disclosed herein, arranged to determine a hologram of an image for observation using a head-up display (HUD). The HUD includes a display device and a pupil expander. The HUD is configured to operate with at least one observation system. Each observation system includes an entrance pupil on an entrance pupil plane, a lens on a lens plane, and a sensor on a sensor plane. The HUD may be configured to operate with a pair of observation systems, such as a pair of eyes. The display device (e.g., a spatial light modulator) is arranged to display the hologram. The pupil expander is arranged to receive light spatially modulated according to the hologram. For example, the displayed hologram may be illuminated with at least partially coherent light from a light source. The display device spatially modulates the received light according to the displayed hologram. The holographic engine is arranged to determine a first composite light field at the entrance pupil of the observation system. The first composite light field is generated by the propagation of light from the display plane of the display device along each light propagation path of the pupil expander. The first composite light field is also generated by clipping according to the entrance pupil of the observation system. The holographic engine is also arranged to determine a second composite light field at the sensor plane of the sensor of the observation system. The second composite light field is generated by the propagation of light from the first composite light field through the lens of the observation system from the input pupil. The second composite light field is also generated by modifications to the amplitude components based on the image. The holographic engine is further arranged to determine a third composite light field at the entrance pupil. The third composite light field is generated by the backpropagation of light from the second composite light field from the sensor plane through the lens. The third composite light field is also generated based on cropping of the entrance pupil. The holographic engine is further arranged to determine a fourth composite light field on the display plane. The fourth composite light field is generated by the backpropagation of light from the third composite light field along each light propagation of the pupil expander. The fourth composite light field is also generated based on cropping of the display device. The holographic engine is arranged to extract a hologram from the fourth dataset. The holographic engine can be embedded in a display driver, such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). The display driver can be part of an image generation unit (PGU) for a head-up display (HUD).

[0075] Various aspects of this disclosure relate to a hologram or phase hologram characterized by the guidance or routing of holographic light. Specifically, this document discloses a diffraction structure arranged to spatially modulate light that can be converted into an image by an observation system, wherein the diffraction structure is configured to route light into a plurality of hologram channels, each hologram channel corresponding to a different portion of the image.

[0076] The diffraction structure can be displayed at a display device, such as a spatial light modulator, including, but not limited to, a liquid crystal on silicon (LCOS) spatial light modulator (SLM). When the display device displaying the diffraction structure is properly illuminated, the diffraction structure is configured to spatially modulate the light, resulting in the light emitted by the display device being routed into multiple hologram channels. A single (i.e., common) light source can be used to illuminate the entire diffraction structure. The diffraction structure may include multiple pixels, with each pixel contributing light to each hologram channel.

[0077] Holographic channels can be called "holographic channels" because they include light channels that have been spatially modulated by the diffraction structure.

[0078] The diffraction structure can be arranged such that holographic channels propagate from the diffraction structure at different angles. That is, relative to the display device described above, each holographic channel is characterized by a unique axial angle or a pair of angles. Each such angle can be defined between the principal or core direction of travel of the corresponding channel and a point on the display device (e.g., the center point of the display device showing the diffraction structure). Each pixel of the hologram or diffraction structure can contribute to each channel.

[0079] Depending on the corresponding portion of the image, each holographic channel may primarily consist of spatially modulated light. The word "primarily" is used to reflect that some overlap may occur between channels, but the overlap is relatively small. For example, the first channel may primarily (i.e., roughly or essentially) correspond to the first angular portion of the field of view (i.e., the image), and the second channel may roughly or essentially correspond to the second angular portion of the field of view (i.e., the image), but there may be some partial overlap between the first and second angular portions. That is, both the first and second channels can encode information about the overlapping areas of the field of view.

[0080] Each channel corresponds to a sub-region of the image (i.e., a angular region of the field of view). While there may be some partial overlap between sub-regions (as described above), each channel is uniquely associated with a point in the field of view that forms the center of the corresponding sub-region. In some embodiments, the sub-regions have the same size and / or shape. In other embodiments, the sub-regions have different sizes and / or shapes. In some embodiments, the shape of each sub-region is determined by the shape of the display device, and more specifically, by the shape of the region defining the pixel array.

[0081] Diffraction structures can be arranged to spatially modulate the phase of light.

[0082] Diffraction structures can be arranged to route light via waveguides. Waveguides can be arranged for pupil expansion or pupil replication.

[0083] The cross-sectional shape of the light pattern formed by each holographic channel can substantially correspond to the shape of the incident aperture of the observation system. When the cross-section is observed in the plane of the incident aperture, the cross-sectional size of each channel can also substantially correspond to the size of the incident aperture. When the cross-section is observed in the plane of the incident aperture, the size of the cross-section can be similar to but larger than the size of the incident aperture.

[0084] Holographic channels can be spatially separated or at least partially spatially separated. As the channels propagate from the diffraction structure toward the observer or observation system, they can spread out or diverge to separate themselves spatially. In some embodiments, the channels spread out in only one direction / dimension, such as the horizontal direction.

[0085] A system is also disclosed herein, comprising a diffraction structure, a waveguide arranged to receive spatially modulated light from the diffraction structure, and an observation system arranged to receive the spatially modulated light via the waveguide.

[0086] The system can be arranged such that light from each hologram channel arrives at the observation system from the diffraction structure along a different optical path.

[0087] Different optical paths can include different numbers of reflections within the waveguide. Different optical paths can have different lengths. Different optical paths can pass through the entrance aperture of the observation system at different corresponding angles.

[0088] Waveguides can be arranged such that all hologram channels are routed through the entrance aperture of the observation system at any observation location on the observation plane. For each permitted observation location, the waveguides can route each hologram channel to the observation system via only one optical path.

[0089] At least two of the multiple holographic channels can partially overlap at the entrance aperture of the observation system.

[0090] The diffraction structure can be a phase hologram or a hologram. It can include computer-generated holograms. A hologram engine or other controller or processor can be provided to output signals to control a display device to display the diffraction structure.

[0091] The term "hologram" is used to refer to a record containing amplitude or phase information about an object, or some combination thereof. The term "holographic reconstruction" is used to refer to the optical reconstruction of an object formed by illuminating a hologram. The system disclosed herein is described as a "holographic projector" because the holographic reconstruction can be a real image and is spatially separate from the hologram. The term "reproduced field" is used to refer to a 2D region within which the holographic reconstruction is formed and fully focused. If the hologram is displayed on a spatial light modulator including pixels, the reproduced field will be repeated in the form of multiple diffraction orders, where each diffraction order is a copy of the zero-order reproduced field. The zero-order reproduced field typically corresponds to the preferred or master reproduced field because it is the brightest reproduced field. Unless otherwise explicitly stated, the term "reproduced field" should be considered to refer to the zero-order reproduced field. The term "reproduced plane" is used to refer to a plane in space containing all the reproduced fields. The terms "image," "reproduced image," and "image region" refer to the region of the reproduced field illuminated by the light reconstructed by the hologram. In some embodiments, an “image” may include discrete points, which may be referred to as “image points” or simply as “image pixels” for convenience.

[0092] The terms “encoding,” “writing,” and “addressing” are used to describe the process of providing multiple corresponding control values ​​to multiple pixels of an SLM, each determining the modulation level of the pixel. In other words, the pixels of an SLM are configured to “display” an optical modulation distribution in response to receiving multiple control values. Therefore, it can be said that an SLM “displays” a hologram, and a hologram can be considered an array of optical modulation values ​​or levels.

[0093] It has been found that acceptable-quality holographic reconstructions can be formed from "holograms" containing only phase information related to the original object (i.e., the target image used for reconstruction). Such holographic records can be referred to as phase-only holograms. Embodiments relate to phase-only holograms, but this disclosure is equally applicable to amplitude-only holography. This disclosure is not limited to any particular method of hologram computation. By way of example only, some embodiments relate to point cloud holograms, i.e., holograms constructed using point cloud methods. However, this disclosure is equally applicable to Fourier or Fresnel-type holograms and holograms computed according to other techniques such as coherent ray tracing.

[0094] This disclosure is equally applicable to forming holographic reconstructions using amplitude and phase information associated with the original object (i.e., the target image). In some embodiments, this is achieved by using complex modulation of a so-called fully complex hologram that incorporates amplitude and phase information associated with the original object. Because the value (gray level) assigned to each pixel of the hologram has amplitude and phase components, such a hologram may be called a fully complex hologram. The value (gray level) assigned to each pixel can be represented as a complex number with amplitude and phase components. In some embodiments, a fully complex computer-generated hologram is computed.

[0095] The term "phase delay" can be a shorthand for the phase value, phase component, phase information, or simply phase of a pixel in a computer-generated hologram or spatial light modulator. That is, any phase value described is actually a number representing the amount of phase delay provided by that pixel (e.g., in the range of 0 to 2π). For example, a spatial light modulator is described as a pixel having a π / 2 phase value causing a π / 2 radian phase delay in the received light. In some embodiments, each pixel of a spatial light modulator can operate on one of a plurality of possible modulation values ​​(e.g., phase delay values). The term "gray level" can be used to refer to a plurality of available modulation levels. For example, the term "gray level" can be used for convenience to refer to a plurality of available phase levels in a phase modulator, even if different phase levels do not provide different shades of gray. For convenience, the term "gray level" can also be used to refer to a plurality of available complex modulation levels in a complex modulator.

[0096] Therefore, a hologram comprises an array of gray levels, i.e., an array of optical modulation values, such as phase delay values ​​or complex modulation values. A hologram is also considered a diffraction pattern because it is a pattern that causes diffraction when displayed on a spatial light modulator and illuminated with light whose wavelength is relative to (typically less than) the pixel spacing of the spatial light modulator. Reference is made herein to combining holograms with other diffraction patterns, such as those used as lenses or gratings. For example, a diffraction pattern used as a grating can be combined with a hologram to translate the playback field on the playback plane, or a diffraction pattern used as a lens can be combined with a hologram to focus the holographic reconstruction onto the playback plane in the near field.

[0097] Although different embodiments and groups of embodiments may be disclosed separately in the detailed description below, any feature of any embodiment or group of embodiments may be combined with any other feature or combination of features of any embodiment or group of embodiments. That is, all possible combinations and substitutions of the features disclosed in this disclosure are contemplated.

[0098] The following items with the following numbers were also disclosed:

[0099] 1. A light engine arranged to provide spatially modulated light to an observation system having an entrance pupil, wherein the display system comprises:

[0100] A display device arranged to display a hologram and spatially modulate light according to the hologram; and

[0101] A holographic engine is configured to receive contribution information from a contributing and non-contributing region of a display device based on the position of an entrance pupil. The contributing region of the display device substantially propagates spatially modulated light through the entrance pupil at that position, while the non-contributing region of the display device substantially propagates spatially modulated light blocked by the entrance pupil at that position.

[0102] The contribution information further identifies (i) at least one primary contributing region of the display device that propagates light to the observation system, contributing to the primary image, and (ii) at least one secondary contributing region of the display device that propagates light to the observation system, contributing to the secondary image.

[0103] The holographic engine is also configured to determine the hologram based on at least one major contributing region of the display device and output the hologram to the display device for display.

[0104] 2. The light engine as described in Project 1, wherein the light engine further includes a monitoring system arranged to determine the position of the entrance pupil of the observation system.

[0105] 3. The optical engine as described in Item 1 or 2, wherein the optical engine further includes a waveguide arranged to receive spatially modulated light from a display device and to provide multiple different optical propagation paths for the spatially modulated light from the display device to the entrance pupil, wherein each contributing region corresponds to a different corresponding optical propagation path provided by the waveguide.

[0106] 4. The light engine as described in any one of items 1 to 3, wherein the observation system is arranged to form an image corresponding to the hologram.

[0107] 5. The light engine as described in Project 4, wherein the primary image includes a first version of the image and the secondary image includes a second version of the image.

[0108] 6. The light engine as described in any of the preceding items, wherein the contribution information is the identification of the corresponding contributing and non-contributing regions of the display device for each of a plurality of image points of the image.

[0109] 7. The optical engine as described in Project 6, wherein the hologram comprises multiple sub-holograms, wherein each sub-hologram is determined by the hologram engine based on contribution information of corresponding image points of the image.

[0110] 8. A method for determining a hologram for display on a display device; the method comprising:

[0111] (i) Determine the position of the entrance pupil of the observation system arranged to observe the hologram;

[0112] (ii) Identify the contributing and non-contributing regions of the display device, wherein the contributing region of the display device substantially propagates light through the entrance pupil of the observation system at a defined location, and the non-contributing region of the display device substantially propagates light blocked by the entrance pupil of the observation system at a defined location.

[0113] (iii) Identifying at least one primary contributing region of a display device that provides light contributing to the primary image and at least one secondary contributing region of a display device that provides light contributing to the secondary image; and

[0114] (iv) Determine the hologram based on at least one major contributing region of the display device.

[0115] 9. The method for determining a hologram as described in Item 8, wherein steps (ii) to (iv) are performed for each of a plurality of image points of an image, and wherein determining the location of each contributing region of each image point includes identifying the location where light rays propagating from the image point to the incident pupil intersect with the display device.

[0116] 10. A method for determining a hologram as described in item 8 or 9, wherein the observation system includes a waveguide arranged to receive spatially modulated light from a display device and to provide multiple different light propagation paths for the spatially modulated light from the display device to the entrance pupil, and wherein step (iii) includes, for each image point, determining the number B of internal reflections within the waveguide pupil expander corresponding to the main image.

[0117] 11. The method of Item 10, wherein the step of determining the number of internal reflections (B) is based on an angle associated with the corresponding image point, wherein the angle is relative to the optical axis of a line formed by extrapolating a line connecting the center of the display device and the determined location of the incident pupil to the image.

[0118] 12. The method as described in Item 11, wherein each contributing region has a size based on the diameter of the incident pupil.

[0119] 13. The method of any one of items 8 to 12, wherein steps (ii) and (iii) comprise, for each image point:

[0120] For B-order light reflection within the waveguide, from image point [x] virtual ,y virtual ,z virtual Ray tracing from the observation plane of the observation system to identify the position [x] on the observation plane. sensor ,y sensor ];

[0121] For image point [x] virtual ,y virtual ,z virtual [x] to the position on the observation plane sensor ,y sensor The propagation of light with B-fold reflection is used to determine the coordinates [x] of the principal ray at the display device. LCOS (B),y LCOS (B)]; and

[0122] Identify by [x] LCOS (B),y LCOS(B)] refers to the effective pixels of the display device within the defined area.

[0123] 14. The method of any one of items 8 to 13, wherein step (iv) comprises determining one or more values ​​of the hologram only in at least one major contributing region of the display device.

[0124] 15. The method of any one of items 8 to 13 further includes excluding the values ​​of the hologram in the region of the display device that are not included in the main contribution area.

[0125] 16. The method for determining a hologram as described in any one of items 8 to 13, wherein step (iv) includes excluding hologram values ​​associated with at least one minor contributing region during the determination of the hologram.

[0126] 17. The method for determining a hologram as described in any one of items 8 to 13, wherein step (iv) includes determining the hologram to be limited to at least one major contributing region of the display device.

[0127] 18. The method for determining a hologram as described in any one of items 8 to 17, wherein step (iv) comprises determining a sub-hologram for each image point in a corresponding at least one major contributing region and combining the sub-holograms to form a hologram.

[0128] 19. The method for determining a hologram as described in item 18, wherein each sub-hologram comprises determining a hologram by moving light waves from [x] virtual ,y virtual ,z virtual The amplitude and / or phase hologram components determined by propagation to the corresponding major contribution region.

[0129] 20. The method for determining a hologram as described in any one of items 8 to 13, further comprising, for each image point, identifying an additional contributing region of the display device associated with the B+ΔB bounce.

[0130] 21. The method for determining a hologram as described in Item 20, wherein identifying additional contributing regions includes:

[0131] For the B+ΔB bounces, proceed from [x] sensor ,y sensor Inverse ray tracing to the virtual image plane zvirtual;

[0132] Determine the image to be formed for B+ΔB reflections [x] sensor ,y sensor The virtual point coordinates [x] virtual (ΔB),y virtual (ΔB),z virtual ];

[0133] For the B+ΔB bounces from [x] virtual (ΔB),y virtual (ΔB),z virtual The propagation of light to the observation plane determines the coordinates of the principal ray at the display device. LCOS (B+ΔB),y LCOS (B+ΔB)]; and

[0134] Identify by [x] LCOS (B+ΔB),y LCOS The additional valid pixels of the display device within the second region defined by (B+ΔB)].

[0135] 22. The method for determining a hologram as described in item 21 further includes determining an additional sub-hologram for each additional valid pixel and combining the additional sub-hologram with the sub-hologram.

[0136] 23. The method for determining a hologram as described in item 22, wherein each additional sub-hologram includes amplitude and / or phase hologram components, which are determined by moving light waves from [x... virtual (ΔB),y virtual (ΔB),z virtual It is determined by the propagation to the additional contribution region.

[0137] 24. The light engine or method for determining a hologram as described in any of the preceding items, wherein the hologram is configured such that the primary image and at least one secondary image are virtual images, each virtual image having a non-infinite virtual image distance.

[0138] 25. The light engine or method for determining a hologram as described in the preceding items, wherein the display area of ​​the display device has a first dimension of less than 5 cm, for example less than 2 cm or less than 1 cm.

[0139] 26. The light engine or method for determining a hologram as described in the preceding items, wherein the distance from the display device to the entrance pupil of the observation system is equal to or greater than 20 cm or 50 cm, for example, greater than 75 cm or greater than 100 cm.

[0140] 27. The light engine or method for determining a hologram as described in any of the preceding items, wherein the waveguide is arranged to receive spatially modulated light from a display device, and wherein the primary image is the image with the maximum brightness selected from a plurality of images formed by the waveguide. Attached Figure Description

[0141] Specific embodiments are described by way of example only with reference to the following figures:

[0142] Figure 1 This is a schematic diagram illustrating a reflective SLM that generates holographic reconstruction on a screen;

[0143] Figure 2A The first iteration of the example Gerchberg-Saxton type algorithm is shown;

[0144] Figure 2B The second and subsequent iterations of the example Gerchberg-Saxton type algorithm are shown;

[0145] Figure 2C Alternative second and subsequent iterations of the example Gerchberg-Saxton type algorithm are shown;

[0146] Figure 3 This is a schematic diagram of a reflective LCOS SLM;

[0147] Figure 4 The image is shown at an angle through a small observation window toward the aperture, indicating the effective propagation of the image.

[0148] Figure 5A An observation system with a relatively short propagation distance is shown;

[0149] Figure 5B An observation system with a relatively large propagation distance is shown;

[0150] Figure 6A An observation system with a relatively large propagation distance is shown, which includes waveguides for forming virtual images at infinity;

[0151] Figure 6B It shows Figure 6A A magnified view of the optical path;

[0152] Figure 7 This demonstrates how to use a finite virtual image and a waveguide pupil expander to form a ghost image;

[0153] Figure 8 A virtual image is shown, comprising a main image and two ghost images;

[0154] Figures 9A to 9C An example is shown where the entire LCOS is used to form a primary image point and two corresponding ghost image points;

[0155] Figures 10A to 10C The first, second, and third propagation paths through the waveguide are shown, which generate the second ghost point, the main image point, and the first ghost point, respectively.

[0156] Figures 11A to 11C Three propagation paths and LCOS utilization are shown in relation to three different field / image points;

[0157] Figure 12AAn observation system is shown, comprising virtual image points and an image of those virtual image points formed by the observation system and waveguides;

[0158] Figure 12B It shows the relationship with Figure 12A The main contribution areas of LCOS related to the example;

[0159] Figure 13 A flowchart illustrating an improved method for deriving an improved data structure according to an embodiment is shown;

[0160] Figure 14 A flowchart illustrating a further improved method for deriving an improved data structure according to an embodiment is shown;

[0161] Figure 15 An optical system according to an embodiment is shown;

[0162] Figure 16 This is a flowchart illustrating the steps of a method according to an embodiment;

[0163] Figure 17A An image comprising multiple image regions (bottom) and a corresponding hologram comprising multiple holographic components (top) are shown;

[0164] Figure 17B The illustration shows a hologram according to the present disclosure, characterized in that holographically encoded light is routed or directed into a plurality of discrete hologram channels; and

[0165] Figure 17C An optimized system is shown, which is arranged to route the light content of each hologram channel to the eye via different optical paths.

[0166] In all the accompanying drawings, the same reference numerals will be used to refer to the same or similar parts. Detailed Implementation

[0167] This invention is not limited to the embodiments described below, but extends to the full scope of the appended claims. That is, the invention may be implemented in different forms and should not be construed as limited to the described embodiments, which are illustrated for illustrative purposes.

[0168] Unless otherwise stated, singular terms may include plural forms.

[0169] A structure described as being formed above or below another structure should be interpreted as including situations where the structures are in contact with each other, and also including situations where a third structure is placed between them.

[0170] When describing temporal relationships, such as when the chronological order of events is described as “after,” “following,” “next,” “before,” etc., this disclosure should be considered to include both consecutive and discontinuous events, unless otherwise stated. For example, unless terms such as “exactly,” “immediately,” or “directly” are used, the description should be considered to include discontinuous cases.

[0171] Although the terms “first,” “second,” etc., may be used herein to describe various elements, these elements are not limited by these terms. These terms are used only to distinguish individual elements. For example, without departing from the scope of the appended claims, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element.

[0172] Features of different embodiments may be coupled or combined with each other in part or in whole, and may interoperate differently with each other. Some embodiments may be implemented independently of each other, or may be implemented together in a mutually dependent relationship.

[0173] Optical configuration

[0174] Figure 1 An embodiment is shown in which a computer-generated hologram is encoded on a single spatial light modulator. The computer-generated hologram is a Fourier transform of the object used for reconstruction. It should be understood that this is merely an example, and other methods for computer-generated holograms are also contemplated in this disclosure. Thus, a hologram can be described as a Fourier domain, frequency domain, or spectral domain representation of the object. In this embodiment, the spatial light modulator is a reflective liquid crystal on silicon (LCOS) device. The hologram is encoded on the spatial light modulator, and a holographic reconstruction is formed at the reproduction field, such as a light-receiving surface (e.g., a screen or diffuser).

[0175] A light source 110 (e.g., a laser or laser diode) is configured to illuminate the SLM 140 via a collimating lens 111. The collimating lens causes a generally plane wavefront of the light to be incident on the SLM. Figure 1 In this embodiment, the wavefront is oriented off-normal (e.g., two or three degrees away from a plane that is truly orthogonal to the transparent layer). However, in other embodiments, a generally planar wavefront is provided with normal incidence, and beam splitters are arranged to separate the input and output optical paths. Figure 1 In the illustrated embodiment, the arrangement is such that light from the light source is reflected from the mirrored rear surface of the SLM and interacts with the light modulation layer to form an outgoing wavefront 112. The outgoing wavefront 112 is applied to an optics device including a Fourier transform lens 120, the focal point of which is located at screen 125. More specifically, the Fourier transform lens 120 receives the modulated beam from the SLM 140 and performs a frequency-space transformation to produce a holographic reconstruction at screen 125.

[0176] It is worth noting that in this type of hologram, each pixel of the hologram contributes to the overall reconstruction. There is no one-to-one correlation between a specific point (or image pixel) on the playback field and a specific optical modulation element (or hologram pixel). In other words, the modulated light leaving the optical modulation layer is distributed across the entire playback field.

[0177] In these embodiments, the spatial position of the holographic reconstruction is determined by the diopter (focusing) of the Fourier transform lens. Figure 1 In the illustrated embodiment, the Fourier transform lens is a physical lens. That is, the Fourier transform lens is an optical Fourier transform lens and performs a Fourier transform optically. Any lens can act as a Fourier transform lens, but the lens's performance will limit the accuracy of the Fourier transform it performs. Those skilled in the art will understand how to use lenses to perform optical Fourier transforms.

[0178] Gerchberg-Saxton method

[0179] In some embodiments, the computer-generated hologram is a Fourier transform hologram, or simply a Fourier hologram or a Fourier-based hologram, wherein the image is reconstructed in the far field by utilizing the Fourier transform properties of a positive lens. The Fourier hologram is computed by Fourier transforming the desired light field in the reproduction plane back to the lens plane. The Fourier transform can be used to compute computer-generated Fourier holograms.

[0180] Algorithms such as the Gerchberg-Saxton algorithm can be used to compute Fourier transform holograms. Furthermore, the Gerchberg-Saxton algorithm can be used to compute holograms in the Fourier domain (i.e., Fourier transform holograms) based solely on amplitude-only information in the spatial domain (e.g., a photograph). This effectively "retrieves" phase information related to the object from the amplitude-only information in the spatial domain. In some embodiments, the Gerchberg-Saxton algorithm or its variants are used to compute computer-generated holograms from amplitude-only information.

[0181] The Gerchberg-Saxton algorithm takes into account the fact that the intensity cross section I of the beams in planes A and B is known. A (x,y) and I B (x,y) and I A (x,y) and I B The case where (x,y) is correlated via a single Fourier transform. For a given intensity cross section, the approximate phase distribution Ψ in planes A and B is obtained. A (x,y) and Ψ B(x,y). The Gerchberg-Saxton algorithm finds a solution to the problem by following an iterative process. More specifically, the Gerchberg-Saxton algorithm iteratively applies spatial and spectral constraints while repeatedly transferring the representation I between the spatial and Fourier (spectral or frequency) domains. A (x,y) and I B A dataset (x, y) containing amplitude and phase. A corresponding computer-generated hologram in the spectral domain is obtained through at least one iteration of the algorithm. The algorithm is convergent and arranged to produce a hologram representing the input image. The hologram can be an amplitude-only hologram, a phase-only hologram, or a fully complex hologram.

[0182] In some embodiments, the phase-only hologram is computed using an algorithm based on the Gerchberg-Saxton algorithm, such as the algorithm described in British Patents 2498170 or 2501112, the entire contents of which are incorporated herein by reference. However, the embodiments disclosed herein are described by way of example only when calculating the phase-only hologram. In these embodiments, the Gerchberg-Saxton algorithm retrieves the phase information Ψ[u,v] of the Fourier transform of a dataset, which produces known amplitude information T[x,y], where the amplitude information T[x,y] represents the target image (e.g., a photograph). Since amplitude and phase are inherently combined in the Fourier transform, the transformed amplitude and phase contain useful information about the accuracy of the computed dataset. Therefore, the algorithm can be used iteratively with feedback of amplitude and phase information. However, in these embodiments, the phase-only information Ψ[u,v] is used as a hologram to form a holographic representation of the target image at the image plane. The hologram is a dataset of phase values ​​(e.g., a 2D array).

[0183] In other embodiments, an algorithm based on the Gerchberg-Saxton algorithm is used to compute a fully complex hologram. A fully complex hologram is a hologram having amplitude and phase components. A hologram is a dataset (e.g., a 2D array) comprising an array of complex data values, where each complex data value includes an amplitude component and a phase component.

[0184] In some embodiments, the algorithm processes complex data, and the Fourier transform is a complex Fourier transform. Complex data can be viewed as comprising (i) real and imaginary components, or (ii) amplitude and phase components. In some embodiments, the two components of the complex data are processed differently at different stages of the algorithm.

[0185] Figure 2AA first iteration of an algorithm for computing a phase-only hologram, according to some embodiments, is shown. The input to the algorithm is an input image 210 comprising a 2D array of pixel or data values, where each pixel or data value is an amplitude or oscillation value. That is, each pixel or data value of the input image 210 does not have a phase component. Therefore, the input image 210 can be considered as an amplitude-only, oscillation-only, or intensity-only distribution. An example of such an input image 210 is a photograph or a frame of a video comprising a time-series of frames. The first iteration of the algorithm begins with a data formation step 202A, which includes assigning random phase values ​​to each pixel of the input image using a random phase distribution (or random phase seed) 230 to form an initial complex dataset, where each data element of the dataset includes both amplitude and phase. In other words, the initial complex dataset represents the input image in the spatial domain.

[0186] First processing block 250 receives an initial complex dataset and performs a complex Fourier transform to form a complex dataset of Fourier transforms. Second processing block 253 receives the complex dataset of Fourier transforms and outputs a hologram 280A. In some embodiments, hologram 280A is a phase-only hologram. In these embodiments, second processing block 253 quantizes each phase value and sets each amplitude value to 1 to form hologram 280A. Each phase value is quantized according to the phase level that can be represented on the pixel of the spatial light modulator that will be used to "display" the phase-only hologram. For example, if each pixel of the spatial light modulator provides 256 different phase levels, each phase value of the hologram is quantized to one of the 256 possible phase levels. Hologram 280A is a phase-only Fourier hologram representing an input image. In other embodiments, hologram 280A is a fully complex hologram comprising an array of complex data values ​​(each including an amplitude component and a phase component) derived from the received complex dataset of Fourier transforms. In some embodiments, the second processing block 253 constrains each complex data value to one of a plurality of permissible complex modulation levels to form a hologram 280A. The constraint step may include setting each complex data value to the closest permissible complex modulation level in the complex plane. The hologram 280A can be said to represent an input image in the spectral, Fourier, or frequency domain. In some embodiments, the algorithm stops at this point.

[0187] However, in other embodiments, the algorithm continues, such as Figure 2A As shown by the dashed arrow in the image. In other words, follow... Figure 2A The steps indicated by the dashed arrows are optional (i.e., not essential for all embodiments).

[0188] The third processing block 256 receives the modified complex dataset from the second processing block 253 and performs an inverse Fourier transform to form a complex dataset with an inverse Fourier transform. The complex dataset with the inverse Fourier transform can be said to represent the input image in the spatial domain.

[0189] The fourth processing block 259 receives the complex dataset of the inverse Fourier transform and extracts the distribution of amplitude values ​​211A and the distribution of phase values ​​213A. Optionally, the fourth processing block 259 evaluates the distribution of amplitude values ​​211A. Specifically, the fourth processing block 259 can compare the distribution of amplitude values ​​211A of the complex dataset of the inverse Fourier transform with the input image 510, which itself is, of course, the distribution of amplitude values. If the difference between the distribution of amplitude values ​​211A and the input image 210 is sufficiently small, the fourth processing block 259 can determine that the hologram 280A is acceptable. That is, if the difference between the distribution of amplitude values ​​211A and the input image 210 is sufficiently small, the fourth processing block 259 can determine that the hologram 280A is a sufficiently accurate representation of the input image 210. In some embodiments, for comparison purposes, the distribution of phase values ​​213A of the complex dataset of the inverse Fourier transform is ignored. It will be understood that any number of different methods can be used to compare the distribution of amplitude values ​​211A with the input image 210, and this disclosure is not limited to any particular method. In some embodiments, the mean squared error is calculated, and if the mean squared error is less than a threshold, the hologram 280A is considered acceptable. If the fourth processing block 259 determines that the hologram 280A is unacceptable, further iterations of the algorithm can be performed. However, this comparison step is not required, and in other embodiments, the number of iterations of the algorithm performed is predetermined, preset, or user-defined.

[0190] Figure 2B This represents the second iteration of the algorithm and any further iterations of the algorithm. The distribution of the phase values ​​213A from previous iterations is fed back through the algorithm's processing block. Distributions of amplitude values ​​211A are rejected, favoring the distribution of amplitude values ​​of the input image 210. In the first iteration, data formation step 202A forms a first complex dataset by combining the distribution of amplitude values ​​of the input image 210 with the random phase distribution 230. However, in the second and subsequent iterations, data formation step 202B includes forming a complex dataset by combining (i) the distribution of phase values ​​213A from previous iterations of the algorithm with (ii) the distribution of amplitude values ​​of the input image 210.

[0191] Then, with reference Figure 2A The same method described is handled by Figure 2BThe complex dataset formed in step 202B is used to form the second iterative hologram 280B. Therefore, the description of this process will not be repeated here. The algorithm can stop when the second iterative hologram 280B has been computed. However, any number of further iterations of the algorithm can be performed. It will be understood that the third processing block 256 is only needed if a fourth processing block 259 is required or further iterations are needed. The output hologram 280B generally improves with each iteration. However, in practice, a point is often reached where measurable improvement is no longer observable, or the positive benefits of performing further iterations are offset by the negative impact of the additional processing time. Therefore, the algorithm is described as iterative and convergent.

[0192] Figure 2C This represents an alternative embodiment for the second and subsequent iterations. The distribution of the phase value 213A from the previous iteration is fed back through the algorithm's processing block. The distribution of the amplitude value 211A is rejected, favoring an alternative distribution of the amplitude value. In this alternative embodiment, the alternative distribution of the amplitude value is derived from the distribution of the amplitude value 211 from the previous iteration. Specifically, processing block 258 subtracts the distribution of the amplitude value of the input image 210 from the distribution of the amplitude value 211 from the previous iteration, scales the difference by a gain factor α, and subtracts the scaled difference from the input image 210. This is mathematically expressed by the following equation, where the subscript text and numbers represent the iteration number:

[0193] R n+1 [x,y]=F'{exp(iψ n [u,v])}

[0194] ψ n [u,v]=∠F{η×exp(i∠R n [x,y])}

[0195] h=T[x,y]-α(|R n [x,y]|-T[x,y])

[0196] in:

[0197] F' is the inverse Fourier transform;

[0198] F is the forward Fourier transform;

[0199] R[x,y] is the complex number dataset output by the third processing block 256;

[0200] T[x,y] is the input or target image;

[0201] ∠ is the phase component;

[0202] Ψ is a phase-only hologram 280B;

[0203] η is a new distribution of amplitude value 211B; and

[0204] α is the gain factor.

[0205] The gain factor α can be fixed or variable. In some embodiments, the gain factor α is determined based on the size and rate of the input target image data. In some embodiments, the gain factor α depends on the number of iterations. In some embodiments, the gain factor α is only a function of the number of iterations.

[0206] In all other respects, Figure 2C Implementation examples and Figure 2A and Figure 2B The implementation is the same. It can be said that only the phase hologram Ψ(u,v) includes the phase distribution in the frequency or Fourier domain.

[0207] In some embodiments, a spatial light modulator is used to perform a Fourier transform. Specifically, holographic data is combined with second data that provides optical power. That is, the data written to the spatial light modulator includes holographic data representing an object and lens data representing a lens. When displayed on the spatial light modulator and illuminated with light, the lens data simulates a physical lens, i.e., it focuses light in the same way as a corresponding physical optical element. Therefore, the lens data provides optical power or focused power. In these embodiments, [the following can be omitted] Figure 1A physical Fourier transform lens 120 is provided. Data representing the lens is known. This data can be referred to as a software lens. For example, a phase-only lens can be formed by calculating the phase delay caused by the optical path length at each point of the lens due to its refractive index and spatial variation. For example, the optical path length at the center of a convex lens is greater than the optical path length at the edge of the lens. An amplitude-only lens can be formed from Fresnel zone plates. In the field of computer-generated holography, it is also known how to combine data representing the lens with a hologram to perform a Fourier transform of the hologram without requiring a physical Fourier lens. In some embodiments, the lensed data is combined with the hologram by simple addition, such as simple vector addition. In some embodiments, a physical lens is used in combination with a software lens to perform the Fourier transform. Alternatively, in other embodiments, the Fourier transform lens is omitted entirely, allowing holographic reconstruction to occur in the far field. In further embodiments, the hologram can be combined with grating data (i.e., data arranged to perform grating functions such as image steering) in the same manner. Again, how to calculate such data is known in the art. For example, a phase-only grating can be formed by modeling the phase delay caused by each point on the surface of the blazed grating. An amplitude-only grating can be simply superimposed on an amplitude-only hologram to provide angular steering for holographic reconstruction. The second data providing lensing and / or steering may be referred to as an optical processing function or optical processing pattern to distinguish it from the holographic data, which may be referred to as an image forming function or image forming pattern.

[0208] In some embodiments, the Fourier transform is performed jointly by a physical Fourier transform lens and a software lens. That is, the software lens provides some of the optical power that contributes to the Fourier transform, while one or more physical optics provide the remaining optical power that contributes to the Fourier transform.

[0209] In some embodiments, a real-time engine is provided, arranged to receive image data using an algorithm and compute holograms in real time. In some embodiments, the image data is video comprising a sequence of image frames. In other embodiments, the holograms are pre-computed, stored in computer memory, and retrieved as needed for display on an SLM. That is, in some embodiments, a library of predetermined holograms is provided.

[0210] The embodiments described herein are by way of example only and involve Fourier holography and Gerchberg-Saxton type algorithms. This disclosure is equally applicable to Fresnel holography and Fresnel holograms that can be computed using similar methods. This disclosure is also applicable to holograms computed using other techniques, such as point cloud-based methods. As will be seen, the following figures herein are described as including point cloud methods for hologram computation. However, other methods for hologram computation, including those referenced above, can be used alternatively. Figures 2A to 2CThe Fourier method described.

[0211] Optical modulation

[0212] Spatial light modulators can be used to display diffraction patterns, including computer-generated holograms. If the hologram is a phase-only hologram, a spatial light modulator is needed to modulate the phase. If the hologram is a fully complex hologram, a spatial light modulator that modulates both the phase and amplitude can be used, or a first spatial light modulator that modulates the phase and a second spatial light modulator that modulates the amplitude can be used.

[0213] In some embodiments, the light modulation element (i.e., pixel) of the spatial light modulator is a cell comprising liquid crystal. That is, in some embodiments, the spatial light modulator is a liquid crystal device in which the optically active component is liquid crystal. Each liquid crystal cell is configured to selectively provide multiple light modulation levels. That is, each liquid crystal cell is configured at any time to operate at one light modulation level selected from multiple possible light modulation levels. Each liquid crystal cell can be dynamically reconfigured to a light modulation level different from the multiple light modulation levels. In some embodiments, the spatial light modulator is a reflective liquid crystal on silicon (LCOS) spatial light modulator, but this disclosure is not limited to this type of spatial light modulator.

[0214] LCOS devices provide a dense array of light-modulating elements or pixels within a small aperture (e.g., a few centimeters wide). Pixels are typically about 10 micrometers or smaller, resulting in a diffraction angle of a few degrees, meaning the optical system can be compact. The small aperture of an LCOS SLM is much easier to fully illuminate than the larger apertures of other liquid crystal devices. LCOS devices are typically reflective, meaning the circuitry driving the LCOS SLM pixels can be buried beneath the reflective surface. This results in a higher aperture ratio. In other words, the pixels are densely packed, meaning there are virtually no dead zones between pixels. This is advantageous because it reduces optical noise in the playback field. LCOS SLMs use a silicon substrate, which has the advantage of optically flat pixels. This is particularly important for phase modulation devices.

[0215] The following are just examples for reference. Figure 3 To describe a suitable LCOS SLM, an LCOS device is formed using a single-crystal silicon substrate 302. It has a 2D array of square planar aluminum electrodes 301, spaced apart by gaps 301a, arranged on the upper surface of the substrate. Each electrode 301 can be addressed by circuitry 302a buried in the substrate 302. Each electrode forms its own planar mirror. An alignment layer 303 is disposed on the electrode array, and a liquid crystal layer 304 is disposed on the alignment layer 303. A second alignment layer 305 is disposed on a planar transparent layer 306, for example, made of glass. A single transparent electrode 307, for example made of ITO, is disposed between the transparent layer 306 and the second alignment layer 305.

[0216] Each square electrode 301, together with the area covered by the transparent electrode 307 and the intermediate liquid crystal material, defines a controllable phase modulation element 308, commonly referred to as a pixel. Taking into account the space between pixels 301a, the effective pixel area, or fill factor, is the percentage of the total number of optically active pixels. By controlling the voltage applied to each electrode 301 relative to the transparent electrode 307, the properties of the liquid crystal material of the individual phase modulation elements can be altered, thereby providing a variable delay for light incident upon them. The effect is to provide phase-only modulation to the wavefront, i.e., without amplitude effects.

[0217] The described LCOS SLM outputs spatially modulated light in a reflective manner. The advantage of a reflective LCOS SLM is that the signal lines, grating lines, and transistors are located below the mirror, resulting in a high fill factor (typically greater than 90%) and high resolution. Another advantage of using a reflective LCOS spatial light modulator is that the thickness of the liquid crystal layer can be half that required when using a transmissive device. This significantly improves the switching speed of the liquid crystal (a key advantage for projecting moving video images). However, the teachings of this disclosure can also be implemented using a transmissive LCOS SLM.

[0218] Image projection using small display devices and long viewing distances

[0219] This disclosure relates to image projection, wherein the distance between the display device and the observer is much larger than the size of the display device. The viewing distance (i.e., the distance between the observer and the display device) can be at least one order of magnitude larger than the size of the display device. The viewing distance can be at least two orders of magnitude larger than the size of the display device. For example, the pixel area of ​​the display device can be 10 mm × 10 mm, and the viewing distance can be 1 m. The image projected by the system is formed on a display plane spatially separated from the display device. Compared to the viewing distance, the entrance aperture for the observer to view the image may also be relatively small.

[0220] According to this disclosure, the image is formed by holographic projection. The hologram is displayed on a display device. The hologram is illuminated by a light source (not shown), and the image is perceived on a display plane spatially separated from the hologram. The image can be real or virtual. For the purposes of the following explanation, it is helpful to consider a virtual image formed upstream of the display device—that is, appearing behind the display device. However, it is not essential that the image be virtual, and this disclosure applies equally to real images formed between the display device and the viewing system.

[0221] This disclosure enables the presentation of images (real or virtual) using very small display devices, even at relatively large viewing distances. This is achieved by providing a hologram simulating the presence of the image at the desired location and by intelligently guiding light that has been spatially modulated by the hologram, taking into account the location of the viewing system and the size and / or shape of the entrance aperture through which the light enters the viewing system.

[0222] Display devices consist of pixels that display holograms. The pixel structure of a display device is diffracted. Therefore, the size of the hologram is determined by diffraction rules. See below for reference. Figure 4 This can be explained using broad optical terminology to describe the result of display devices being extremely small in size.

[0223] Figure 4 An aperture 402 is shown, which forms a small viewing window between the real object or image 401 and the viewing system 405. Aperture 402 represents the aperture of the display device. Figure 4 The effect of aperture 402 on light from a real object or real image 401 located at a finite distance upstream of aperture 402 is shown. Aperture 402 is very small relative to the distance between it and the observation system 405. In this illustrative arrangement, image 401, display device 402, and observation system 405 are arranged on the optical axis Ax.

[0224] Figure 4 Only those rays (or beams) from image 401 are shown, which will pass through the very small viewing window defined by aperture 402 and propagate toward viewing plane 406, which is defined perpendicular to the optical axis Ax. A skilled reader will understand that other rays will propagate from image 401 but will not coincide with aperture 402, so that they cannot (in this example) reach viewing plane 406. Furthermore, five rays (or beams) are shown propagating from image 401 (each of five different portions of image 401)—however, a skilled reader will also recognize that this is merely illustrative and that this disclosure is not limited to five rays or beams. Viewing system 405 has an entrance aperture 404 located directly in front of viewing plane 406. Viewing system 406 can be a human eye. Thus, entrance aperture 404 can be the pupil of the eye, and viewing plane 406 can be the retina of the eye.

[0225] Figure 4 The very small size of the aperture 402 shows that only a portion of the image can be seen from each position on the viewing plane. Figure 4Five example light beams are shown, each characterized by a corresponding angle relative to the optical axis Ax, and each propagating from a different corresponding portion of image 401. The beam propagating along the optical axis Ax carries the central portion of the image; that is, it is the light from the center of the image. The other beams carry other portions of the image. The very small viewing window defined by aperture 402 and the very small entrance aperture of pupil 404, compared to the large viewing distance, result in not all image content being able to pass through the entrance pupil 404 at any given viewing position. In other words, not all image content is received by the eye. Figure 4 In the example, at any observation position, only one of the five beams shown passes through the pupil 404.

[0226] In this example, at the pupil position 404 shown, the central portion of the image is visible to the eye. The rest of the image information is blocked. The reader will understand that if the observer moves up or down, the eye may receive different beams of light; for example, the central portion of the image may be blocked. Therefore, the observer can only see a portion of the entire image. The rest of the image information is blocked. In other words, the observer's field of view is severely limited because they are actually viewing the image through a small aperture in the display device itself.

[0227] In short, light travels from the display device to the small viewing window within a certain angular range. At a viewing distance of 1 meter, for a given eye position, only a small range of angles from the small viewing window can travel through the pupil of the eye to form an image on the retina. Only the visible portion of the image falls on the retina. Figure 4 The portion shown is within a small angular range, passing through the entrance aperture 404. Therefore, the field of view is very small, and the specific angular range depends heavily on the eye position.

[0228] Reference Figure 4 The problems of a small field of view and sensitivity to eye position are a consequence of the large viewing distance and small aperture of the observation window, as well as the small entrance aperture of the observation system. Refer to Figures 5 through 7 for a further explanation of the importance of the viewing distance.

[0229] It is well known that holograms, displayed and illuminated on suitable display devices, are used to form images (real or virtual) at desired locations. However, the inventors have recognized that conventional holographic techniques are insufficient for forming images clearly and accurately using small display devices, especially for relatively large viewing distances or relatively small viewing apertures. They have also recognized that this is particularly true if the image is to be presented at a non-infinite distance from the observer, such as a virtual image. Furthermore, the inventors have recognized that conventional holographic techniques typically rely on holographic reconstruction to form an image between the display device and the observer, where the holographic reconstruction can be formed in free space or in a light-receiving member. However, formation relying on temporary holographic reconstruction often requires additional optical elements, such as diffusers, lenses, or mirrors, which can be impractical or undesirable, especially in applications requiring compactness and high real estate value.

[0230] Figure 5A A display device 502 is shown, which is arranged to display a hologram and propagate light that has been spatially modulated according to the hologram to an observation system including an incident aperture 504 and an observation plane 506. Figure 5A The display device in the middle has the same Figure 4 Similar to the small physical size of the observation aperture in [the text]. Figure 5A The light trails from the virtual image (not shown) represented by the hologram are also shown upstream of the display device 502. The virtual image 501 is at infinity, therefore the light trails traced between the virtual image and the display device 502 are collimated. The collimated light from the virtual image is depicted as comprising five rays or beams; however, it should be understood that this is merely illustrative and should not be considered a limitation of this disclosure.

[0231] Figure 5A The lower part shows an enlarged view of the observation system. This view is schematic and therefore does not show the physiological details of the eye. In reality, of course, there is a light source arranged to illuminate the display device 502. Figure 5A (Not shown in the image).

[0232] exist Figure 5A In this configuration, the distance between the display device and the viewing plane is small enough that the full diffraction angle of the light from the display device can form an image on the retina. In other words, the light propagation paths of all five beams (shown as coming from the virtual image) pass through the incident aperture. Therefore, all points on the virtual image are mapped onto the retina, and all image content is transmitted to the viewing plane. Consequently, the field of view for the perceived image is maximized. At the optimal position, the field of view equals the diffraction angle of the display device. Interestingly, different image points on the retina are formed by light propagating from different areas of the display device 502, such as those closest to the retina. Figure 5AThe image dot at the top is formed solely by light propagating from the bottom of the display device. Light propagating from other areas of the display device does not contribute to that image dot.

[0233] Figure 5B This shows what happens as the observation distance increases.

[0234] More in detail, Figure 5B A display device 502' is shown, which is arranged to display a hologram and propagate light modulated according to the hologram to an observation system including an incident aperture 504' and an observation plane 506'. The virtual image 501' is at infinity, so the light between the virtual image and the display device is collimated. Figure 5B The lower part shows an enlarged view of the observation system. This view is schematic and therefore does not show the physiological details of the eye. In reality, it certainly has a light source arranged to illuminate the display device 502'. Figure 5B (Not shown in the image).

[0235] Figure 5B Only light rays that can propagate through aperture 504' are shown; any other light rays that cannot pass through aperture 504' are omitted. However, it will be understood that those other light rays will also propagate from display device 502'. Figure 5B At larger viewing distances, the light cone spreads out on the viewing plane to such an extent that some light beams are blocked by the incident aperture 504' (i.e., they do not physically coincide). Specifically, in this example, light beams associated with the edge portions of the virtual image are blocked by the incident pupil 504'. However, if the incident aperture 504' moves to a position parallel to the viewing plane 506', different corresponding beams may coincide with the aperture 504', resulting in different portions of the virtual image being seen. Therefore, for any given aperture position, the entire virtual image is invisible, and the visible portion of the virtual image is heavily dependent on the aperture (e.g., eye) position. Consequently, large distances between the display device and the viewing system are problematic due to the small size of the display device, especially when combined with a relatively small incident aperture.

[0236] Figure 6A An improved system including a display device 602 is shown, which propagates light encoded with a hologram displayed on the display device 602 into an observation system including an entrance aperture 604 and an observation plane 606. In fact, a light source (not shown) is, of course, arranged to illuminate the display device 602. The improved system also includes a waveguide 608 located between the display device 602 and the entrance aperture 604. Figure 6A The lower part shows an enlarged view of the entrance pupil 604 and the viewing plane 606. This figure is schematic and therefore does not show the physiological details of the eye.

[0237] The observation distance in Figure 6 is... Figure 5BThe same. However, in Figure 5B The blocked beam of light is effectively recovered by waveguide 608, enabling the observation system to receive complete image information, even at a longer observation distance.

[0238] The presence of waveguide 608 allows content from all angles of the display device 602 to be received by the eye, even at this relatively large projection distance. This is because waveguide 608 acts as a pupil expander in a well-known manner, and therefore will only be briefly described here.

[0239] In short, waveguide 608 comprises a generally elongated structure. In this example, it comprises an optical plate of refractive material, but other types of waveguides are also well known and can be used. Waveguide 608 is positioned to intersect, for example, at an oblique angle, a light cone projected from display device 602. The size, position, and orientation of waveguide 608 are configured to ensure that light from each of the five light beams within the light cone enters waveguide 608. Light from the light cone enters waveguide 608 via a first planar surface 610 (located closest to display device 602) and is guided at least partially along the length of waveguide 608 before being emitted via a second planar surface 612 substantially opposite the first surface 610 (located closest to the eye). It is readily understood that the second planar surface 612 is partially reflective and partially transmissive. In other words, as each ray of light travels within waveguide 608 from its first planar surface 610 to its second planar surface 612, some light will be transmitted through waveguide 608, and some light will be reflected back to its first planar surface 610 by the second planar surface 612. The first planar surface 610 is reflective, such that all light striking it from within waveguide 608 will be reflected back to its second planar surface 612. Therefore, some light can simply be refracted between the two planar surfaces 610, 612 of waveguide 608 before being transmitted, while other light can be reflected and thus undergo one or more reflections (or "bouncing") between the planar surfaces 610, 612 of waveguide 608 before being transmitted. Therefore, the net effect of waveguide 608 is that the transmission of light is effectively extended to multiple locations on the second planar surface 612 of waveguide 608. Thus, compared to the case without waveguide 608, all angular content output by display device 602 can appear at a greater number of locations on the display plane (and at a greater number of locations on the aperture plane). This means that light from each beam can enter the entrance aperture 604 and contribute to the image formed by the viewing plane 606, despite the relatively large projection distance. In other words, the eye can receive content from all angles of the display device 602. Therefore, the full diffraction angle of the display device 602 is utilized, and the viewing window is maximized for the user. In turn, this means that all light contributes to the perceived virtual image 601.

[0240] Figure 6B The virtual image 601 (which is shown) is illustrated. Figure 6A The individual optical paths of each of the five light beams contributing to the five corresponding image points within the (formed in the image) are labeled R1 to R5 from top to bottom. As can be seen therein, the light from each of R1 and R2 is simply refracted and then transmitted through waveguide 608. On the other hand, the light from R4 encounters a single bounce before transmission. The light from R3 includes some light from the corresponding first portion of the display device 602, which is simply refracted by waveguide 608 before transmission, and some light from a different second corresponding portion of the display device 602, which encounters a single bounce before transmission. Similarly, the light from R5 includes some light from the corresponding first portion of the display device 602 that encounters a single bounce before transmission, and some light from a different second corresponding portion of the display device 602 that encounters two bounces before transmission. For each of R3 and R5, the two different portions of LCOS propagate light corresponding to the portion of the virtual image.

[0241] The inventors have recognized that, at least in some applications, the virtual image distance (i.e., the distance from the observer to the virtual image) is preferably finite, contrary to the notion of forming a virtual image at infinity. In some applications, there will be a preferred virtual image distance at which the appearance of the virtual image content is desired or necessary. This could be, for example, in a head-up display, such as in a car setting, if the virtual image content is to be superimposed on real content viewed by the observer through the vehicle's windshield. For example, a desired virtual image distance could include virtual image content formed a few meters in front of the observer's vehicle or windshield, such as 3 meters or 5 meters.

[0242] Figure 7 The upper portion shows a system including a display device 702 that propagates light 703 to an eye, which includes an entrance aperture 704 and an observation plane 706. This light 703 has been encoded (i.e., modulated according to) a hologram displayed on the display device 702. A light source (not shown) is present, arranged to illuminate the display device 702. The system also includes a waveguide 708 located between the display device 702 and the entrance aperture 704, acting as a pupil expander, as described in detail above with respect to FIG. 6a. Figure 7 The middle section shows an enlarged view of the entrance aperture 704 and the observation plane 706. Figure 7 The bottom portion shows a further magnified view of the viewing plane 706. This diagram is schematic and therefore does not show the physiological details of the eye. In this arrangement, the eye perceives the virtual image 701 as being located at a finite distance upstream of the display device 702. The light is divergent because the distance to the virtual image is finite.

[0243] According to the above Figure 6A , Figure 7 The presence of the middle waveguide 708 effectively enables the full diffraction angle of the display device 702 to be accessed at a relatively large projection distance, allowing the user to see the complete image content at the indicated observation position.

[0244] However, this introduces a further technical problem. Due to the presence of divergent light and the resulting different angles of light, for some light beams, the different optical paths of light from different parts of the display device 702 can cause these light beams to each form multiple image points on the retina 706 when a virtual image is formed at a finite virtual image distance. This is in Figure 7 The light beams labeled R3' and R5' are shown in diagram a. The resulting additional image points, attached to the main image points at a given point within the virtual image, may be referred to as "ghost image points," and together they form a "ghost image," or simply "ghost." As those skilled in the art of image formation will understand, from the observer's perspective, the formation of ghosts leads to blurring of the virtual image and an overall decrease in perceived quality. This is especially true if the "ghost" partially overlaps with the "main" image.

[0245] Figure 8 An example is shown where virtual images of the numbers "5" and "9" are displayed, which are created using a method similar to... Figure 7 The observation system shown in image 'a' creates an image that includes both the main image and a ghost image. The main image can be viewed as the brightest central image for each digit, with ghost images on both the left and right sides. Figure 8 In the example, when the observation distance is greater than that for '5', '9' is formed, so the blur is more noticeable for it.

[0246] Hologram Computation—Example 1

[0247] The inventors have solved the problem of ghosting. They have recognized the need for an observation system in which a virtual image can be formed at a finite virtual image distance, encompassing all angular image content output by the display device, while reducing or eliminating ghosting. Furthermore, the inventors have found that the risk of forming ghosting image points increases with the increase of the observation aperture size in conventional observation systems, as the aperture allows additional light to enter, potentially forming additional image points on the display plane. Therefore, they desire an improved observation system capable of accommodating different aperture sizes while still reducing or eliminating ghosting. The solutions provided by the inventors, detailed below, are applicable to a range of aperture, waveguide, and display device sizes and arrangements, and can be applied to different propagation distances for which one or more ghosting images can conventionally form. Moreover, they can be applied to relatively small display devices.

[0248] In summary, the inventors have recognized that an optical engine for generating holograms can be provided that avoids or reduces the formation of ghosting image points when the hologram is displayed on a display device and illuminated, while allowing the complete image to be observed accurately. The inventors have also recognized that a hologram engine for providing such holograms can be provided even when the projection distance in the observation system is relatively large and the display device and / or observation aperture is relatively small, and an improved observation system for displaying and illuminating the improved hologram for forming an improved image.

[0249] According to embodiments, the inventors have recognized that holograms can be determined using modeling methods, such as ray tracing methods, or point cloud hologram computation techniques. The modeling designed by the inventors effectively identifies one or more regions of a display device that, in conventional arrangements, would affect one or more ghost images. The hologram is derived to control the effects from these one or more regions of the display device, thereby avoiding or reducing the formation of ghost image points when the hologram is displayed on and illuminated on the display device.

[0250] Other methods for determining holograms are also disclosed here, as detailed in the following figures.

[0251] The inventors have recognized that, due to the presence of waveguides (e.g., here) Figure 6A and 7 The observation system (shown in A) imposes an angular constraint, comprising a relatively small observation aperture and optionally a relatively small display device, thus making it possible to consider different possible propagation paths within the waveguide separately. Furthermore, they have recognized that, as a result of this consideration, it is possible to identify each of the following: regions of the display device that are light sources contributing to the desired "master" image; regions of the display device that are light sources causing an unwanted "ghosting" image; and regions of the display device that are light sources blocked by the aperture and therefore contribute neither to the master image nor the ghosting image. The inventors also recognize that it is possible to limit hologram calculations to the regions of the display device that contribute only to the master image. They also recognize that, in further improvements, in some embodiments, improved holograms can be provided, which can effectively allow one or more ghosting images to be translated for superimposed onto the master image.

[0252] Referring to the accompanying drawings described in detail below, one can further understand the inventor's insights and the improved systems and methods that embody these insights.

[0253] Figure 9ADisplay device 902 is shown, which in this example is an LCOS spatial light modulator. "LCOS" as used below is short for "display device". The teachings of this disclosure are not limited to LCOS display devices. Figure 9B An LCOS 902 is shown and a ray of light from the LCOS 902 via a waveguide 908 toward an observation entity / system 905 associated with a virtual image point is depicted. In this example, the observation entity / system 905 includes the observer's eye. Figure 9C It also includes a magnified view of eye 905, showing the light rays at pupil 904 (i.e., the entrance aperture) and retina 906 (i.e., the sensor or observation plane). In this example, the entire LCOS region contributes to the formation of image points on retina 906. In other words, the entire LCOS 902 is "visible" to the observer. This contribution of the entire LCOS 902 to the image is shown by the entire LCOS being shaded, with its entire surface area represented as the "contributing area".

[0254] It can be seen that, from Figure 9B and 9C The light tracked by the LCOS 902 causes three image points—labeled G1, M, and G2—to be formed on the retina 906 for that particular virtual image point. The middle image point 'M' comprises the master image point, which contributes to the main / master virtual image perceived by the observer. The top image point G1 comprises the first ghost image point, and the bottom image point G2 comprises a different second ghost image point of the same virtual image point. Notably, in yet another advancement, the inventors have recognized the possibility of identifying regions of the LCOS 902 that contribute to the master image point M and / or the ghost image points G1 and G2.

[0255] Figures 10A to 10C The LCOS 902 and Figures 9A to 9C The ray pattern is divided into three corresponding propagation paths—the first path includes light contributing to the bottom ghost image point G2, the second path includes light contributing to the main image point M, and the third path includes light contributing to the top ghost image point G1. For example... Figure 10A As shown, the light contributing to G2 bounces three times before being transmitted through waveguide 908. (As...) Figure 10B As shown, the light contributing to M bounces twice before being transmitted through waveguide 908. Figure 10C As shown, the light contributing to G1 bounces once before being transmitted through waveguide 908.

[0256] Each figure (10A, 10B, 10C) also shows the portion of LCOS 902 that contributes to the corresponding image point through shading. Thus, it can be seen that the bottom ghost image point G2 is contributed by the region facing the lower part of LCOS 902, the top ghost image point G1 is contributed by the region facing the upper part of LCOS 902, and the main image point is contributed by the entire LCOS 902.

[0257] exist Figures 9A to 9C In the examples 10A to 10C, the aperture 904 (i.e., the observer's pupil) is relatively wide, which explains why the entire LCOS 902 contributes to the main image point. In other words, in this example, the f-number of the observation system is relatively low. Figures 10A to 10C As shown, while portions of LCOS 902 contribute to one or more of the ghosted images G1 and G2, there are also regions of LCOS 902 that contribute not to either of the ghosted images G1 or G2, but only to the main image point M. The inventors have recognized that this region can be identified as a contributing region, and more specifically, in this example, it can be identified as the "primary contributing region," as will be further understood from the description in the following figures. Therefore, it can be seen that, in this case, the primary contributing region is not limited to a circle or an ellipse, but can take other more complex shapes.

[0258] Figures 11A to 11C The diagram shows the corresponding ray patterns at different points in the virtual image when the incident aperture is relatively small (i.e., the f-number is relatively high). Figure 11A The first field point (i.e., the first virtual image point) involves the virtual image. Figure 11B Involving the second field point of the virtual image, Figure 11C The third field point involves the virtual image. Figures 11A to 11C This shows that not all LCOS 902 pixels contribute to the main image points. In fact, Figures 11A to 11C The diagram shows that the first region of LCOS corresponds to the main image point (here referred to as the "major contribution region"), while the second region of LCOS corresponds to the ghost image point (here referred to as the "minor contribution region").

[0259] The inventors have recognized that, under certain conditions, different corresponding regions of the LCOS 902 (or other display devices in the observation system) will contribute to the main image or the ghost image, or may not contribute to any visible portion of the image. They further recognize that this information can be used to optimize the hologram determination process. For example, light from certain portions of the display device can be omitted, or in some cases, the way the hologram encodes these portions of the display device can be altered to actively contribute to the main image rather than the ghost image. Furthermore, additional regions of the display device can be identified, which can be configured to actively contribute to the main image.

[0260] The inventors' insights will be described below using an example in conjunction with point cloud holograms. However, these insights can be applied to other types of holograms, such as Fourier or Fresnel holograms, as detailed later in this disclosure with reference to subsequent figures. That is, other hologram calculation methods, as described in this disclosure, can be optimized based on the inventors' insights.

[0261] As is well known, to compute a point cloud hologram of an image (e.g., a virtual image), the image is typically decomposed (i.e., represented by it) into multiple individual points—here called “virtual points”, since we are describing the formation of a virtual image. Spherical waves (or “wavelets”) are then propagated from the expected or desired location of each virtual point within the virtual image to the plane of the display device, such as the plane of LCOS in the example above, through computation (i.e., using a model or other theoretical tools). Considering the way these wavelets interfere with each other, and calculating the final amplitude and / or phase of the wavelets received at each pixel of the display device, the display device can then be tuned in a well-known manner, and therefore not described here, to illustrate the amplitude and / or phase modulation required at each pixel location to simulate the computed wavelets, thereby creating a hologram of the image.

[0262] The inventors have recognized that, for observation systems with waveguides and large viewing distances as described herein, if the entire display device has the net amplitude and phase of the corresponding wavelets for all virtual points, the hologram to be created during display and illumination may produce one or more ghost images as well as the main image. This can occur particularly when the observation system is configured to perceive virtual images at a finite distance from the observer. Furthermore, in many cases, light emitted from pixels in some parts of the device will be wasted (i.e., they will not contribute to the image seen or perceived by the observer) because the physical limitations of the observation system (e.g., small aperture and / or small display device and / or large projection distance) will determine which parts of the device the light from will not enter the observer's eyes. Therefore, the inventors have recognized that intelligent selection can be applied regarding which parts of the display device are tuned to provide the hologram. Specifically, if only those portions (or regions) of the LCOS that contribute to the main image are selected, and if the wavelets are propagated from virtual points of the expected virtual image only to those portions of the LCOS, and not to other portions of the LCOS that do not contribute to the main image, then the composite amplitude and / or phase of the wavelets received at each pixel within the selected area of ​​the display device can be calculated. No other corresponding portions of the display device need to be calculated.

[0263] Then, based on the improved calculations, the display device can be tuned to display the required amplitude and phase modulation at each pixel location within the selected section, in order to simulate the calculated wavelets and thus create a hologram of the main image. When this is done, no other parts of the LCOS need to be tuned, so when the calculated hologram is displayed on the display device and illuminated, no image information will propagate from those other parts to the observer's eye (or other observed entity). Therefore, no information is available to the observer, which could lead to the formation of unwanted "ghosting" image points. As a result, the ghosting is eliminated or "extinguished." Furthermore, no computational or image information is wasted because, for a given set of conditions (e.g., for a specific aperture width and position of the eye), only those pixels of the display device known to provide light that will be allowed through the observer's pupil (or through the aperture of the corresponding other observed entity) will be tuned.

[0264] Figure 12A A system 1200 for forming a virtual image including example virtual point 1201 is shown. The observation system 1200 includes a display device 1202, which in this example is an LCOS SLM, including contributing regions 1203 and non-contributing regions 1207 as identified according to this disclosure. The display device 1202 is arranged to display a hologram of the virtual image and to project light encoded according to the hologram onto an eye 1205, which includes a pupil (not shown) acting as an aperture, a lens 1209, and a retina 1206 acting as an observation plane. The lens 1209 and the retina are separated by a distance 'A'. A light source (not shown) is present, arranged to illuminate the display device 1202. The observation system 1200 also includes a waveguide 1208 located between the LCOS 1202 and the eye 1205. This image is schematic and therefore does not show physiological details of the eye.

[0265] Virtual point 1201 is located upstream of display device 1202, which is in Figure 12AThe image is described by virtual point 1201 being located to the left of display device 1202. Virtual point 1201 has a position defined by spatial coordinates, which in this example include Cartesian (x,y,z) coordinates, but other coordinate systems or other means of identifying the virtual point's position may be used. A distance 'z' is defined between virtual point 1201 and display device 1202 in a direction substantially parallel to the optical axis of display device 1202. A distance 'l' from display device to lens is also defined between display device 1201 and eye lens 1209 in a direction substantially parallel to the optical axis of display device 1201. At any given time, the values ​​of 'z' and 'l' will vary depending on the specific arrangement of the viewing system 1200, including the observer's position. For example, the distance 'l' from display device to lens may be on the order of approximately 1 meter, while the distance 'z' from display device to image may be much larger, such as on the order of several meters. However, these numerical examples are purely illustrative and should not be considered limiting.

[0266] The inventors have recognized that if a virtual image including virtual image point 1201 is to be perceived by an observer at the position shown in Figure 12, then the corresponding image point 1211 must be formed on the retina 1206. Light rays can be traced from the virtual point 1201 of the virtual image to the corresponding point 1211 on the retina 1211 via LCOS 1202.

[0267] It should be understood that, due to the possible paths created / generated by waveguide 1208, more than one possible optical path can be used between virtual point 1201 and its corresponding point 1211 on the retina via LCOS 1202. According to an embodiment, a master ray can be determined, which includes one of multiple ray paths between virtual image point 1201 and its corresponding point 1211 on the viewing plane (i.e., retina 1206). When this master ray path is identified, the number of bounces experienced by the light of the master ray within the waveguide is determined. This number of bounces (B) can be set as the number of bounces the ray should be tracked between the virtual image and the viewing plane. According to an embodiment, as an initial step, the master ray and the associated number of bounces (B) can be identified.

[0268] In this example, ray tracing can determine the portion of the LCOS 1202 traversed by the "main ray" as it travels between the virtual image point 1201 and the corresponding point 1211 on the retina, in order to identify the "contribution region" 1203 of the virtual image point 1201. Therefore, in Figure 12, there exists a ray 'r' depicted as propagating between the virtual image point 1201 and the contribution region 1203 of the display device 1202. According to the inventors, based on the virtual image point 1201 and the display device 1202, only wavelets contributing to the contribution region of the LCOS need to be modeled (or otherwise computationally considered). In other words, only the identified contribution region 1203 of the display device 1202 needs to be encoded (or "tuned") to generate a proper hologram. When encoded and properly illuminated on the display device, this hologram will allow the observer to perceive the virtual image point 1201 without any ghosting of it. This can be discussed below. Figure 13 and 14 Further understanding is needed.

[0269] Figure 12B The contribution region 1203 and the following about Figure 13 and 14 The contributing region discussed can be located based on the intersection of the main ray and the display device. For example, the contributing region can be centered on the point where the main ray intersects the display device. The size and shape of the contributing region can be determined based on the size and shape of the incident aperture of the corresponding observed entity and associated optics (e.g., waveguide geometry, any reflections within a larger optical system, etc.). Thus, when the observed entity is the human eye, in some cases, the contributing region on the display device can include a generally circular or elliptical shape, or any other suitable shape, such as a complex shape similar to the receiving pupil size. However, this disclosure includes more complex contributing region shapes. The pupil diameter can be measured or estimated in any suitable manner. For example, the pupil diameter can be measured using an eye-tracking system. Alternatively, the pupil diameter can be estimated based on a known range of pupil diameters (e.g., 2-6 mm) or based on another estimate under ambient light conditions at a given time.

[0270] A contribution area can be set to intentionally contribute an area slightly larger than the pupil (in the aperture plane), and / or contribute an area whose shape is slightly different from the pupil (or other aperture) (in the aperture plane). In this case, not all light from the "contribution area" can pass through the pupil all the way through, but the eye will be able to move slightly while still collecting enough light to form a good image on the retina.

[0271] Figure 13A method for determining contributing and non-contributing regions of a display device according to one aspect of this disclosure is illustrated. Optionally, these determinations can then be used to optimize the generation of one or more holograms for display and illumination by an observation system such as system 1200 of Figure 12. (Refer to...) Figure 13 In the described method, the observation system includes a lens and a camera with "f" numbers (i.e., focal length and aperture). The photosensitive component of the camera can be, for example, a CCD array and is located on the observation plane. Functionally, the lens and camera replace the lens and retina of the observer's eye and are used solely for the process of determining contributing and non-contributing areas of the display device. These areas of the display device can be determined for multiple observation positions (e.g., eye position within an eye-tracking box) and / or multiple image distances (e.g., virtual image distance in front of a vehicle). In some aspects, references Figure 13 The published method can be considered a pioneer in holographic computation. This method can be considered an optimization or even a calibration process.

[0272] It is easy to understand that each virtual image to be generated can be represented by one or more virtual image points, each virtual image point having a corresponding position, for example defined by (x,y,z) coordinates. Figure 13 Steps 1302 through 1312 of method 1300 (detailed below) can be applied to each virtual image point within the virtual image to be created. Furthermore, method 1300 is applicable to a specific set of conditions of the observation system, namely specific measurements and constraints. Therefore, any given iteration (or “run”) of method 1300 is suitable for establishing the specific image to be created (one virtual image point after another), and is suitable when the system has a specific display device-to-image distance “z”, a specific distance “d” between the display device and the retina, a specific aperture (pupil) width, and a specific virtual image distance for eye focusing. Iterations of method 1300 are also specific to display devices of a specific size and type, and have a permissible viewing window for a specific eye position. Other measurements and / or constraints may exist specific to each iteration of the method. According to embodiments, if any of these measurements or constraints change, method 1300 can be rerun to redetermine the contribution area of ​​the display device under the changed conditions. However, it will be understood that, according to embodiments, certain tolerances may be applied to one or more of these measurements or constraints such that if their variation is less than a predetermined amount and / or less than a predetermined time length, the method need not be rerun. Rules regarding when the method should be repeated can be determined on a per-system basis.

[0273] Method 1300 can be executed by a suitable processor. The processor may include, be included in, or communicate with a holographic engine. The processor or holographic engine may be contained within a light engine.

[0274] Prior to executing method 1300, the processor may acquire or receive boundary information about the observation system. For example, it may acquire or receive information such as the size of components of the display device, information about the absolute and / or relative positions of various components and the observation system (e.g., a potential human observer), information about the light source, etc.

[0275] According to method 1300, in the first step 1302, the position of the virtual image point (hereinafter referred to as "virtual point") is obtained based on the position of the virtual image to be perceived, for example, the coordinates [x...]. virtual ,y virtual ,z virtual Then, the virtual image distance between lens 1209 and the virtual point is obtained or determined. This virtual image distance can be set or determined by the processor executing method 1300, or it can be set or determined by another entity and transmitted to the processor. In some arrangements, it can be preset or selected from multiple possible virtual image distances. In real-world operation, when the observation system is an eye, eye-tracking or head-tracking information can be used to determine the virtual image distance.

[0276] In the second step 1304, the desired distance 'A' between the lens and the sensor is determined to focus on the virtual image point. Each virtual image point can also be defined by an angle, see [reference needed]. Figure 4 The "angle content" mentioned here refers to virtual image points relative to the virtual image.

[0277] In the third step 1306, the number of reflections or bounces "B" of light associated with the master image or primary image formed by the observation system within the waveguide is determined. Those skilled in optics will understand that the waveguide produces multiple copies of light associated with virtual image points, and each copy can be associated with a different number of light bounces / reflections within the waveguide. For example, one way to determine B is to determine the intersection point of the principal ray of each possible light propagation path in the waveguide with the display device, and select the number of reflections / bouncings that brings the principal ray closest to the center of the display device. Advantageously, this method maximizes the area of ​​the display device that contributes to the observation system.

[0278] Alternatively, another method for calculating the number of bounces used in step 3 1306 includes the following sub-steps 1 to 5:

[0279] 1. Know the eye position and use it as input.

[0280] 2. For the first bounce B, ray tracing is performed from the center of the display device to the determined eye position. The extrapolation of the rays into the virtual image defines the field of view (θ) for that bounce B. B ).

[0281] 3. For the second bounce number B+1, ray tracing is performed from the center of the display device to the determined eye position. The extrapolation of the rays into the virtual image defines the field of view (θ) for this bounce number (B+1). B+1 ).

[0282] 4. B is used for θ B and θ B +(θ B+1 -θ B Number of bounces of content at an angle between ) / 2

[0283] 5. B+1 is used for θ B +(θ B+1 -θ B ) / 2 and θ B Number of bounces of content at the angle between

[0284] In step 1308, the output of step 1302 (i.e., the coordinates of the virtual image point) and the output of step 1306 (parameter B) are used to determine the corresponding image position / point [x] on the sensor. sensor ,y sensor ,z sensor In other words, step 1308 in the fourth step determines the point on the sensor where the light from the virtual image point is received. In other words, the point on the sensor where the virtual image point is imaged. Regarding... Figure 14 This point on the sensor is referred to below as the master image point [x]. sensor ,y sensor ,z sensor As an example only, computational ray tracing from a virtual point to a sensor can be used for B-order bounces within a waveguide, but this disclosure is not limited to this method in the fourth step.

[0285] Those skilled in the art will understand that it is possible to identify from virtual point [x] virtual ,y virtual ,z virtual [x] to the point on the sensor sensor ,y sensor ,z sensor The principal ray (or simply principal line) is the light source. Similarly, computational ray tracing can be used to identify or trace the principal ray, but other methods are equally applicable. In step 5, 1310, the intersection point of the display device [x] is identified. LCOS (B),y LCOS (B),z LCOS (B)], where the display device intersection is the location where the main ray on the display device intersects with the display device. The display device intersection can be determined, calculated, or measured, for example, by ray tracing.

[0286] In step 6, 1312, the intersection point with the display device is identified [x]. LCOS (B),y LCOS (B),z LCOS (B)] The area of ​​the related display device. The area of ​​the display device can be geometrically defined by this point [x] LCOS (B),y LCOS (B),z LCOS Centered on [B], this region can be circular or elliptical, but other more complex shapes are also conceivable. If the region is a regular shape, such as a circle or ellipse, its radius can be determined, for example, based on the f-number of the observation system's lens. This region is referred to here as the "primary contributing region" because it corresponds to the primary image formed by the observation system. The term "contribution" reflects that pixels of the display device within the recognition area of ​​the display device are pixels that provide the necessary information content to the sensor. Other areas of the display device (i.e., other pixels of the display device) do not contribute to the formation of image points on the sensor. Of course, other pixels can contribute to other image points on the sensor that are related to other virtual image points.

[0287] The method according to the main aspects of this disclosure concludes by determining the main contributing region of the display device. Alternatively, the hologram can be determined based on the main contributing region rather than the entire area of ​​the display device.

[0288] Therefore, in the optional seventh step 1314, holographic components are determined for the main contributing region based on the virtual points. Specifically, the optical parameters of the main contributing region are determined. The optical parameters can be the amplitude and / or phase of each pixel in the main contributing region. For example, point cloud methods familiar to those skilled in the art can be used to determine the optical amplitude and phase for each pixel in the main contributing region based on the propagation of light from the virtual points to the main contributing region. The holographic components of the virtual points can be stored and combined with the holographic components of other virtual points as part of the iterative process described in the next paragraph to construct a complete hologram for the entire virtual image.

[0289] In summary, in step 7, 1314, light modulation values ​​(e.g., amplitude and / or phase values) are assigned to each pixel value of the display device within the primary contribution area. This is achieved by considering the light wave from [x] virtual ,y virtual ,z virtual The propagation to the main contributing region, and the addition of amplitude and / or phase to [x] LCOS (B),y LCOS (B),z LCOS(B)] is achieved by considering the pixels of the display device within the desired radius. That is, the amplitude and / or phase of the light emitted from the virtual image point and arriving at each point (i.e., pixel) in the main contributing region is determined by taking into account the propagation of the light wave; specifically, the amplitude and / or phase of the light wave after traveling a certain distance from the virtual image point to each pixel. This determination can be performed using any of a variety of different techniques known to those skilled in the art of optics. This determination can also be made through experimental measurement.

[0290] Within the virtual image to be projected using a hologram, steps one through seven can be repeated for each virtual point. For example, multiple hologram components can be added together to produce a composite hologram for each pixel of the display device. For example, for propagation from all virtual image points, complex amplitudes can be added at each pixel. If the hologram is to be displayed on a pure phase modulator, the amplitude components of the resulting complex amplitudes can be ignored, leaving only the phase. More broadly, this result corresponds to the diffraction structure of the virtual image, which, if displayed and illuminated on a display device within the observation system, forms the virtual image.

[0291] Holograms can be displayed or encoded on a display device. As a result, the display device will be tuned to modulate the light in a way that allows the observer to perceive the virtual image at the desired virtual image distance.

[0292] For each of multiple virtual points within a virtual image, method 1300 can be executed substantially simultaneously (or very rapidly consecutively), thereby allowing a suitable hologram of the entire virtual image to be derived and encoded onto a display device very quickly for a given observation setup and specific numerical measurements and constraints. The method can be rerun if any changes occur that might affect the identification of the contributing region and / or the desired tuning of the display device. The processor can be configured to rerun the method on a time-controlled loop, and / or in response to a signal indicating that a change has occurred, and / or when the content or identity of the desired virtual image changes. The processor may include, or communicate with, memory for storing previously calculated data. For example, a lookup table or other storage device may be provided that indicates the effective area of ​​the display device for a specific virtual image or virtual point under a specific set of measurements and / or constraints.

[0293] Method 1300 can run (or rerun) very quickly to rapidly and continuously display multiple different virtual images and / or accurately respond to changes in conditions, such as user movement. Although only one eye is shown in the system of Figure 12, Method 1300 can be configured to consider the observer's eye and / or another observation system with two or more incident apertures. Furthermore, although some of the descriptions above may relate to aperture width, it should be understood that the pupil (and most other apertures used for observing entities) is two-dimensional and its size can vary in each of these two dimensions. Method 1300 can be configured to consider two-dimensional aperture dimensions and their variations.

[0294] The inventor discovered the use of reference Figure 13 The published method can efficiently determine the hologram of a virtual image. However, the inventors also observed that in some cases, only a relatively small portion of the LCOS is utilized when all regions of light that traditionally propagate and would form a ghosting image are not used. In another notable technological advancement, in addition to the main contributing region, the inventors found a method to utilize additional regions of the LCOS and calculate holographic values ​​for these additional regions, which would enable them to contribute light to enhance the main image rather than form an unwanted ghosting image.

[0295] As is well known, the path length of light rays passing through the waveguide in an observation system can be increased relative to the path length of other light rays. Typically, this increase is likely to be small compared to the distance 'v' of the virtual image, and therefore imperceptible to the naked eye.

[0296] Figure 14 A further improved method 1400 based on the inventor's additional understanding is shown, which can be applied to systems such as system 1200 of FIG12. Figure 14 Method 1400 includes Figure 13 The method 1300 includes all steps, and further includes the processing of one or more ghost image points corresponding to virtual points, which may also exist and generally result in the perception of one or more ghost images of the virtual image.

[0297] Method 1400 can be executed by a suitable processor. The processor may include, be included in, or communicate with a holographic engine. The processor or holographic engine may be included within a light engine.

[0298] Before executing the method, the processor can acquire or receive boundary information about the system. For example, it can acquire or receive information such as the size of components of the display device, the absolute and / or relative positions of various components and the observer, information about the light source, etc.

[0299] In some cases, the inventors have discovered that ghosting image points occur because light from the corresponding virtual point passes through a portion of the "primary contribution region" of the display device, different from the portion traversed by the main ray of the main image. In the preceding figures, these portions of the display device are referred to as "secondary contribution regions." The light that produces one or more ghosting image points may be referred to as including one or more "ghosting rays." The rays that produce the ghosting image may undergo a different number of bounces within the waveguide than the rays corresponding to the main image, so as to also pass through the narrow pupil of the observer's eye and coincide with the retina. Therefore, if it is determined that the main ray corresponding to the main image undergoes "B" bounces within the waveguide, it can be determined that the light corresponding to the ghosting image undergoes "B+ΔB" bounces, where ΔB can be a negative or positive integer, typically a single digit, for example, in the range of -5 to +5.

[0300] according to Figure 14 The improved method 1400, in Figure 13 Following step 1308 of method 1300, the position of the main image point on the observation plane is established, for example, its coordinates (x... sensor ,y sensor ,z sensor ), Figure 13 Subsequent steps of method 1300 can continue, and furthermore, for example in parallel or at a later time, for at least one ΔB value, another set of steps can be performed as follows. In summary, Figure 14 The improved method 1400 uses the coordinates of the virtual point [x] virtual ,y virtual ,z virtual The method determines how many bounces 'B+ΔB' the ghosting ray will undergo to form a ghosting image point at the observation plane. Then, the improved method 1400 determines the translational (or corrected) position of a virtual point from which light can travel and undergo 'B+ΔB' bounces within the waveguide to reach the main image point on the observation plane, instead of forming a separate ghosting image point. The position on the LCOS through which the light travels from the translational position of the virtual point to the main image point can then be identified and correspondingly encoded with a hologram. Therefore, one or more additional regions of the LCOS (in addition to the main contributing region) can be encoded with hologram values ​​to contribute to the main image while still avoiding the generation of ghosting images.

[0301] More detailed, the improved method 1400 is as follows:

[0302] In the first further step 1402, from the main image point (x sensor ,y sensor ,z sensor The light rays are traced back to the virtual image, but this is for light rays that have undergone 'B+ΔB' bounces / reflections within the waveguide (instead of B bounces).

[0303] In the second further step 1404, the position of the sub-virtual point of the virtual image is determined (e.g., coordinates [x]). virtual (ΔB),y virtual (ΔB),z virtual (ΔB)] (for example, as a result of ray tracing performed in the first further step 1402), this virtual point will be imaged onto the main image point [x]. sensor ,y sensor ,z sensor That is, the virtual point will propagate light that will pass through the display device, waveguide, and incident aperture to coordinate with the position [x]. sensor ,y sensor ,z sensor The observation plane at point [x] coincides if the light undergoes 'B+ΔB' bounces. The term "secondary virtual point" is used here as an abbreviation for the secondary (i.e., shifted or modified) position of the (primary) virtual point. That is, the inventors have recognized that if the position of the virtual point is moved to the 'secondary virtual point' position [x], the observation plane at point [x] coincides with the light's position if the light undergoes 'B+ΔB' bounces. virtual (ΔB),y virtual (ΔB),z virtual [(ΔB)] Any light from the 'sub-virtual point' that undergoes 'B+ΔB' bounces / reflections in the waveguide will contribute to the master image at the observation plane.

[0304] In summary, the third further step 1406 includes targeting the B+ΔB bounces within the waveguide from [x] virtual (ΔB),y virtual (ΔB),z virtual The propagation of light to the observation plane determines the coordinates of the principal ray at the display device. LCOS (B+ΔB),y LCOS (B+ΔB),z LCOS In some cases, z virtual It can be adjusted to account for different path lengths through the waveguide (i.e., due to different bounce counts). This primary ray can be referred to as the "secondary primary ray".

[0305] More specifically, in the third further step 1406, a point on the display device is identified, wherein the distance from the secondary virtual point to the primary image point [x] sensor ,y sensor ,z sensor The "secondary principal ray" will experience B+ΔB bounces in the waveguide through this point. This point has coordinates [x] on the display device. LCOS (B+ΔB),y LCOS (B+ΔB),z LCOS ].

[0306] In the fourth additional step 1408, point [x] LCOS (B+ΔB),y LCOS (B+ΔB),z LCOS The radius or other suitable indicator of the range of the area to which it is assigned, or the size of the area to which it is assigned. (x) LCOS (B+ΔB),y LCOS (B+ΔB),z LCOS The relevant region is referred to here as the "additional contribution region" because it propagates light that contributes to the master image point at the observation plane, but only if that light originates from the displacement or modified position of the (master) virtual point, i.e., [x virtual (ΔB),y virtual (ΔB),z virtual Instead of [x] virtual ,y virtual ,z virtual As determined in the second further step 1404.

[0307] The fourth further step 1408 is similar to the sixth step 1312. Specifically, the fourth further step 1408 includes identifying the intersection with the display device [x] LCOS (B+ΔB),y LCOS (B+ΔB),z LCOS (B)] The area of ​​the related display device. The area of ​​the display device can be geometrically defined by this point [x] LCOS (B+ΔB),y LCOS (B+ΔB),z LCOS Centered on [B]. For example, this region could be circular or elliptical, but other more complex shapes could also be envisioned. If the region is a regular shape, such as a circle or ellipse, its radius can be determined, for example, based on the f-number of the lens in the observation system. This region is referred to here as the "additional contribution region" because it will propagate light that contributes to the virtual image if a proper hologram is calculated based on the displacement or modified position of the (master) virtual point.

[0308] The fifth further step 1410 is similar to the seventh step 1314. The fifth further step 1410 is optional. In the fifth further step 1410, the position [x] is modified based on the (master) virtual point. virtual (ΔB),y virtual (ΔB),z virtual The holographic components for the additional contributing region are determined. Specifically, the optical parameters of the additional contributing region are determined. The optical parameters can be the amplitude and / or phase of each pixel in the additional contributing region. For example, point cloud methods familiar to those skilled in the art can be used, based on light from different virtual points [x]. virtual (ΔB),y virtual(ΔB),z virtual The propagation to the additional contribution region determines the optical amplitude and phase for each pixel within that region. Different virtual points [x] can be stored. virtual (ΔB),y virtual (ΔB),z virtual The holographic components of the virtual image are combined with the holographic components of other virtual points as part of the iteration described in the following paragraphs to build a complete hologram for the entire virtual image.

[0309] The required light modulation performed by the display device (which is output relative to a single virtual dot) can be referred to as the "holographic component" of that virtual dot. In subsequent repetitions of method 1300 for one or more other virtual dots within the virtual image to be created, the holographic component can be stored by the processor.

[0310] Figure 14 The further improved method 1400, steps 1402 to 1410, can be combined with Figure 13 Steps 1302 through 1314 of method 1300 are repeated for each virtual point within the virtual image to be created. Once the modulation behavior of each virtual point and its corresponding hologram component have been determined, the hologram components can be summed to produce the synthetic modulation behavior of each pixel of the display device. This synthetic modulation behavior represents the diffraction structure or hologram of the virtual image, which, if displayed and illuminated on a display device within the observation system, will only result in the formation of the main image without any ghosting. As an execution... Figure 14 The improved method 1400 results in a main image that may be more powerful than that formed by the method alone. Figure 13 The corresponding main image produced by method 1300 is brighter.

[0311] The processor can output data corresponding to the hologram in any suitable manner. The hologram can then be encoded onto a display device. As a result, the display device will be tuned to modulate the light so that an observer can perceive the virtual image at the desired virtual image distance without any ghosting.

[0312] For each of a plurality of virtual points within a virtual image, method 1400 can be executed substantially simultaneously (or very rapidly consecutively), thereby allowing a suitable hologram of the entire virtual image to be derived very quickly and encoded onto a display device for a given observation setting and specific numerical measurements and constraints. The method can be rerun if any changes occur that may affect the recognition and / or desired tuning of the display device. The processor can be configured to rerun the method on a time-controlled loop, and / or in response to a signal indicating that a change has occurred, and / or when the content or identity of the desired virtual image changes. The processor may include, or be in communication with, memory for storing previously calculated data. For example, a lookup table or other storage device may be provided that indicates the effective area of ​​the display device for a particular virtual image or virtual point under a specific set of measurements and / or constraints.

[0313] Method 1400 can run (or rerun) very quickly to rapidly and continuously display multiple different virtual images and / or accurately respond to changes in conditions, such as user movement. Although only one eye is shown in the system of Figure 12, Method 1400 can be configured to consider both of the observer's eyes. Furthermore, although some of the descriptions above may relate to aperture width, it should be understood that the pupil (and most other apertures used for observing entities) is two-dimensional and its size can vary in each of these two dimensions. Method 1400 can be configured to consider the two-dimensional aperture size and its variations.

[0314] Hologram Computation—Example 2

[0315] above Figure 13 and 14 Methods include point cloud hologram calculation methods; however, the inventors' understanding extends to hologram calculation by any suitable method, including Fresnel and Fourier hologram calculation methods. In other words, the inventors have recognized that holograms can be calculated in one of a variety of different possible ways and displayed on a relatively small display device located at a relatively large distance from the observer, with a waveguide appropriately positioned between the display device and the observer so that the observer sees a virtual image at a finite distance upstream of the display device without ghosting. Furthermore, this can be achieved by projecting the light of the hologram directly into the observer's eye without the need for an intermediate holographic reconstruction.

[0316] For example, the inventor designed a computing Figure 15 The method for creating a hologram using the optical system shown is important. The display device is relatively small, and the projection distance is relatively long. The hologram is projected directly onto the observation system, and this method can be implemented in real time. (As above...) Figure 6AAs shown in the diagram, the relatively small size of the display device and the relatively long projection distance necessitate a pupil expander. This method addresses the different paths through the pupil expander and takes into account the size and shape of the entrance aperture of the observation system. Furthermore, this method, for example using a hologram, allows image content to appear at different and / or multiple distances from the observation system, optionally simultaneously. This method also allows image content to appear downstream and upstream of the display device, optionally simultaneously, for example using a hologram.

[0317] Figure 15 A spatial light modulator 1501 for displaying a hologram is shown. In this embodiment, the spatial light modulator 1501 is a liquid crystal on silicon device arranged to modularize the phase of received light. The spatial light modulator 1501 is illuminated by at least partially coherent light from a light source (not shown). The light source may be a laser diode. The spatial light modulator 1501 outputs light spatially modulated according to the displayed hologram. For example, Figure 15 A single ray 1502 of spatially modulated light is shown. It should be understood that multiple rays of spatially modulated light output from the spatial light modulator 1501 are typically present. A pupil expander 1503 receives the spatially modulated light. The pupil expander 1503 is tilted relative to the plane of the display device 1501. The pupil expander 1503 therefore receives light that is not perpendicularly incident. The angle of incidence (the angle formed by the optical axis and the pupil expander) can be less than 25 degrees, for example, 10 to 20 degrees. The pupil expander includes an input surface 1503a and an output surface 1503b for receiving the spatially modulated light. The input surface 1503a and the output surface 1503b are substantially parallel and extend in the direction of pupil expansion. The input surface 1503a includes at least a portion that is substantially totally internally reflected (e.g., R = 1). The output surface 1503b includes at least a portion that is highly reflective but partially transmissive (e.g., R = 0.9 and T = 0.1). The reflective surfaces are arranged such that spatially modulated light bounces back and forth between them, and the light is emitted at multiple points along the output surface 1503b, as shown in the reference above. Figure 6A As described in waveguide 608. In this embodiment, the pupil expander is substantially elongated. The pupil expander provides pupil expansion in one direction, i.e., the elongation direction, but this disclosure can be extended to include a second pupil expander arranged to expand the pupil in an orthogonal direction.

[0318] Figure 15The diagram illustrates how ray 1502 is effectively replicated twice to form three propagation paths 1505, each associated with a different distance Z0, Z1, and Z2. The shortest propagation path corresponds to Z0, and in this example, the light has passed through the waveguide without any internal reflection. The mid-range propagation path of the three shown corresponds to Z1 and two internal reflections in the pupil expander (once per surface). The longest propagation path shown corresponds to Z2 and four internal reflections in the pupil expander (twice per surface). Planes x0, x1, and x2 respectively show the spatial extent of the light field associated with each of the three propagation paths Z0, Z1, and Z2. More specifically, Figure 15 It shows how the three planes x0, x1 and x2 are offset from each other in the x-direction.

[0319] Figure 15 An observation system 1513 is also shown, which includes an entrance pupil 1507, a lens 1509, and a light sensor 1511. In an embodiment, the observation system 1513 is a human eye, and the light sensor 1511 is the retina of the eye. Figure 15 It shows how only some light fields associated with each propagation path pass through the incident pupil 1507. Figure 15 The diagram shows the light rays associated with the center of the mid-range propagation path passing through the center of the entrance pupil 1507. However, for example, the light rays associated with the center of the light field of the shortest propagation path are blocked by the top of aperture 1507. However, other light rays associated with the light field of the shortest propagation path can pass through aperture 1507. The light rays associated with the center of the light field of the longest propagation path are blocked by the bottom of aperture 1507. However, other light rays associated with the light field of the longest propagation path can also pass through aperture 1507.

[0320] Light passing through aperture 1507 is focused by lens 1509 onto light sensor 1511. The plane of light sensor 1511 is substantially parallel to the plane of display device 1501, and is therefore also tilted relative to the elongated dimension of pupil expander 1503.

[0321] For example only, Figure 15 Three possible light propagation paths for a single ray 1502 of spatially modulated light are shown. This disclosure is not limited to the number of propagation paths. That is, those skilled in the art will understand from the following description that the method can be extended to consider any number of light propagation paths. Similarly, the pupil expander is not necessarily tilted relative to the display plane and the sensor plane. The inventors have devised a method, which will be described below in conjunction with… Figure 16The method is described as follows: for a range of different pupil expander settings and any possible number of light bounces within the pupil expander, and therefore for any number of light propagation paths, this method can be used to calculate a suitable hologram to ensure that spatially modulated light correctly reaches the observer's eye. Importantly, it takes into account the size and shape of the entrance aperture of the observation system so that all the necessary light for the image reaches the observer.

[0322] Figure 16 This is a flowchart illustrating the steps of the method. The method is similar to a Gerchberg-Saxton type algorithm, which uses mathematical transformations back and forth between the image plane and the hologram to converge on a phase hologram corresponding to the image, which can be a virtual image and can be formed at a finite distance upstream of the spatial light modulator 1501. After each propagation to the image plane or holographic plane, the amplitude components of the light field are modified or constrained, but the phase components are preserved.

[0323] The zeroth stage of this method includes steps 1602 and 1604. The zeroth stage includes forming a zeroth composite light field. Step 1602 provides a random phase seed to form the phase component of the zeroth composite light field. Step 1604 provides the amplitude component of the zeroth composite light field. The amplitude component can be a unit or amplitude distribution representing the light source light used to reconstruct the image from the hologram.

[0324] In step 1606, the zeroth composite light field propagates Fresnelly from the spatial light modulator 1501 (i.e., from the holographic plane) to the entrance pupil 1507 of the observation system 1513 (more specifically, to the plane containing the entrance pupil 1507 of the observation system 1513). Furthermore, this embodiment refers to Fresnel propagation as one example of a variety of different mathematical transformations that can be used without departing from the spirit or scope of this disclosure. Step 1506 is performed for each number of bounces or internal reflections provided by the pupil expander 1503 to form a composite light field with respect to each light propagation path. Step 1606 includes taking into account the lateral position of the composite light field in the x-direction at the plane of the entrance pupil 1507, and the phase shift of each reflection within the pupil expander 1503. Different composite light fields can be combined, for example, by addition. The first stage also includes step 1608, tailoring the combined composite light field according to the size and shape of the entrance pupil 1507 to form a first composite light field at the entrance pupil 1507.

[0325] The second phase of the method includes steps 1610 and 1612. In step 1610, a second composite light field is determined by propagating a first composite light field from the entrance pupil through lens 1509 to the plane of the photosensor 1511. Step 1612 includes modifying the amplitude component of the composite light field reaching the photosensor 1511. More specifically, step 1612 includes replacing the amplitude component of the composite light field with the amplitude component of the target image or an amplitude component based on the amplitude component of the target image (e.g., a weighted version of the amplitude component of the target image). The position of lens 1509 used in the propagation determines the image distance—that is, the space in which the image content will appear. In some embodiments, the image is a virtual image, and this distance may be referred to as the virtual image distance “VID”.

[0326] Advantageously, the method disclosed herein allows the formation of image content at multiple different image distances (e.g., multiple VIDs) using the same hologram. The inventors recognize that this can be achieved by repeating the second stage for each image distance, taking into account the different positions of the lens 1509 in the z-direction. For example, the composite light fields determined according to this method for each different image distance can be combined by addition.

[0327] The third stage of the method includes step 1614, in which the second composite light field propagates back to the entrance pupil 1507 via lens 1509. This can be referred to as backpropagation, simply to reflect the travel of light in the opposite z-direction. In some embodiments, backpropagation is the mathematical inverse of the corresponding "forward" propagation. The third stage also includes tailoring the propagated light field according to the size and shape of the entrance pupil 1507 to form a third composite light field.

[0328] The fourth stage comprises steps 1616 and 1618. In step 1616, light propagates back to the plane of the spatial light modulator 1502 via multiple light propagation paths of the pupil expander, in the manner described above with respect to the first stage—but of course, in the opposite light direction (i.e., “reverse” propagation). Step 1618 includes trimming the propagated light field according to the size and location of the effective / pixel area of ​​the display device. The number of complex values ​​of each composite light field may be equal to or less than the number of pixels of the display device.

[0329] Step 1620 includes extracting a hologram from the fourth composite optical field. The hologram may include the phase values ​​of the fourth composite optical field; in this case, the hologram may be referred to as a phase hologram. The method can also begin from the image plane (i.e., the third stage). According to this disclosure, each stage requires at least one iteration. Figure 17A and 17B The method and the way described Figure 13 and 14 Holograms formed using this method.

[0330] Channelized Hologram

[0331] The inventors have discovered that, regardless of the calculation method, the holograms (or "phase holograms" or "diffraction structures") calculated according to the present invention possess unique properties that are not observable or cannot be achieved using conventional hologram calculation methods.

[0332] In summary, the hologram calculated according to this disclosure enables a display device (e.g., but not limited to LCOS) to output channels of spatially modulated light, where each channel corresponds to a different corresponding portion of a corresponding image on which the hologram is displayed and illuminated. This unique channel configuration allows the display device to work with a suitable pupil expander, such as a waveguide, to allow an observer to accurately see the entire image through the relatively small aperture of their eye, even at relatively large viewing distances and when the display device is relatively small, without requiring eye movement. For example, a virtual image located at a finite distance upstream of the display device can be observed (correctly and completely) at a relatively large distance, even if both the observer's eye aperture and the display device displaying the hologram are relatively small. This was previously impossible, whether using conventional holography or non-holography.

[0333] According to one aspect of this disclosure, the inventors have discovered that when calculating holograms using the "point cloud" method described above, light from each virtual image point is limited according to the different corresponding principal contribution areas of the display device. The inventors further recognize that this means light from different parts of the virtual image (i.e., different virtual image points) passes through the system along different optical paths. Similarly, according to one aspect of this disclosure, the inventors have discovered that when calculating holograms using Fresnel propagation, for example via the above... Figure 16 The method shown corresponds to spatially modulated (i.e., “holographic”) light following different corresponding optical paths for different parts of the image. Therefore, the inventors recognized that a hologram (however calculated) could be used to simultaneously direct each of these optical paths to the observer's eye, enabling the observer to receive all the holographic light needed for their eye / brain to reconstruct the entire image without moving their eyes or making any other physical changes. As illustrated in the detailed examples above, to achieve this, waveguides or other pupil expanders can be used in conjunction with a display device that displays the calculated hologram.

[0334] exist Figure 17A and 17BIn the illustrated embodiment, the inventors configured an optical system to display a virtual image comprising multiple discrete virtual image components or regions to aid in understanding the unique properties of the holograms disclosed herein. However, this disclosure is equally applicable to calculating and displaying holograms corresponding to images having continuous (i.e., non-discrete) image content, and / or applicable to holograms of images having any number / size / division of discrete image portions. Figure 17A and 17B In simple terms, (i) the virtual image comprises multiple discrete virtual image components or regions, and (ii) the light of each virtual image component is associated with a different number of bounces / reflections within waveguide 1708. However, in some other embodiments, the light of two or more discrete virtual image components may experience the same number of bounces within the waveguide.

[0335] Figure 17A An image 1752 for projection is shown, comprising eight discrete image regions / components V1 to V8. Figure 17A Eight image components are shown as an example only, and image 1752 can be divided into any number of components. Figure 17A Also shown is an encoded light pattern (i.e., a holographic light pattern) 1754 formed when a hologram (calculated as disclosed herein) is properly displayed and illuminated. The encoded light pattern 1754 can reconstruct an image 1752, for example, when transformed by a lens of a suitable observation system (e.g., an observer's eye). The encoded light pattern 1754 includes first to eighth components or channels H1 to H8 corresponding to the first to eighth image components / regions V1 to V8. Therefore, a hologram can be characterized by the channelization of the holographic light it performs. This channelization of light occurs due to the manner of computation, such as... Figure 17B As shown. Specifically, according to the hologram of this disclosure, holographic light is guided into multiple discrete channels, which can be formed as discrete corresponding regions on a plane. In the example shown, the discrete regions are disks, but other shapes are also conceivable. As mentioned above, the hologram is calculated (e.g., cropped) specifically using the size / shape of the light field at the display device and / or the size / shape of the light field at the viewing aperture. Therefore, the optimal size and shape of the disk may be related to the size and shape of the entrance pupil of the observation system.

[0336] It should be understood from this disclosure that this type of hologram can be calculated by any method, and the inventors' core understanding is that this type of hologram can be used to provide a relatively large field of view using relatively small display devices. The holographic behavior disclosed herein is synergistic with holographic replicators such as pupil expanders. The holographic system disclosed herein is particularly synergistic with head-up displays.

[0337] The channels of holographic light output from the hologram effectively decompose the image content (of the image to be holographically reconstructed by the observer) according to angles. This can be achieved through the above... Figure 4 To further understand this, we need to compare the optical arrangements. Figure 4 In the optical arrangement, light beams from multiple discrete positions on the real image 401 travel to the aperture (or viewing window) 402 at multiple discrete corresponding angles, but at any given eye position, only one of these light beams can pass through the observer's eye. A hologram calculated as described herein and displayed by a suitable display device can form a holographically reconstructed virtual image to simulate the presence of image 401 (or, in fact, any desired image / object) at a desired image distance. However, with... Figure 4 Compared to traditional holographic systems, a significant advantage of this optical system is that the hologram calculated herein can still be seen or perceived by the observer even when the display device is relatively small, the entrance aperture of the observation system (e.g., the observer's eye) is relatively small, and the observation distance is relatively large. In other words, as a non-limiting example, the hologram will enable... Figure 4 All five light beams shown arrive at the observer simultaneously, thus fully forming the desired virtual image.

[0338] Importantly, when properly displayed and illuminated, this hologram causes the display device to output channels of holographic light, where each channel corresponds to the angle (or in some cases, a beam angle) at which light from a corresponding portion of the desired image / object arrives at the display device. Thus, it can be said that the channels of holographic light correspond to different corresponding angular portions of the image content. This is not the case with conventional holograms. Furthermore, unlike unmodulated light from a real image / object or spatially modulated light formed by conventional holograms, the channels of holographic light disclosed herein are specifically configured such that they can be guided by a suitable waveguide or other pupil expander located between the display device and the observer to ensure that the observer can simultaneously receive each channel and thus receive holographic light corresponding to each (i.e., every) portion of the image. Moreover, in at least some embodiments, each channel may be received only once.

[0339] Figure 17C It shows according to Figure 17A and 17B The improved observation system 1700, as shown, can be applied. Figure 13 Method 1300 or Figure 14 Method 1400 or Figure 16 The method or any other suitable method to calculate Figures 17A to 17C The hologram in the scheme shown.

[0340] The observation system 1700 includes a display device comprising an LCOS 1702 in this arrangement. The LCOS 1702 is arranged to display a modulation pattern (or “diffraction pattern”) including a hologram and to project holographically encoded light onto an eye 1705, which includes a pupil acting as an aperture 1704, a lens 1709, and a retina (not shown) acting as an observation plane. A light source (not shown) is arranged to illuminate the LCOS 1702. The light source may include, for example, a laser diode. The hologram is configured such that the entire hologram can be illuminated by a single light beam (or a single beam of light). For it to function as described herein, multiple light sources or, for example, multiple light beams each with a different wavelength are not required to illuminate the hologram.

[0341] The lens 1709 of eye 1705 performs the holographic-to-image conversion. Therefore, there is no holographic reconstruction of the image between LCOS and eye 1705.

[0342] The observation system 1700 also includes a waveguide 1708 located between the LCOS 1702 and the eye 1705. Figure 17C The projection distance may be relatively large. However, as described with respect to the preceding figures, the presence of waveguide 1708 allows all angular content from LCOS 1702 to be received by eye 1705, even at this relatively large projection distance. This is because waveguide 1708 acts as a pupil expander in the manner already described above.

[0343] Furthermore, in this arrangement, when the LCOS 1702 has been encoded according to the method described herein, the waveguide 1708 can be oriented at an angle relative to the LCOS 1702 to establish a unique relationship between the light from the LCOS 1702 and the virtual image perceived by the observer. The size, position, and orientation of the waveguide 1708 are configured to ensure that light from each holographic channel and light from each part of the virtual image enters the waveguide 1708 and is guided along its long axis, bouncing between the substantially flat surfaces of the waveguide 1708. Whenever light reaches the second planar surface (closest to the eye 1705), some light is transmitted and some is reflected.

[0344] Figure 17C A total of nine “bounce” points B0 to B8 are shown along the length of waveguide 1702. The reader will notice that the center of image 1752 remains blank. Figure 17CThe 0th to 9th light "bounce" or reflection points B0 to B8 within the waveguide are shown. Although the light associated with all points of the image (V1-V8) (i.e., the light from each of the eight holographic light channels H1 to H8) is transmitted out of the waveguide at each "bounce" from the second planar surface of waveguide 1708, only the light from one angular portion of the image (e.g., the light from one of channels H1 to H8, which corresponds to a specific corresponding one of the image portions V1 to V8) has a trajectory that enables it to reach the eye 1705 from each corresponding "bounce" point B0 to B8. Furthermore, in this embodiment, light from different channels, and therefore from different corresponding angular portions (V1 to V8) of the image, reaches the eye 1705 from each corresponding "bounce" point. Figure 17C The light from all the different holographic light channels emitted at each "bounce" point is shown (represented by multiple short arrows at each transmission point), but only the light path to the eye 1705 for the corresponding channel is shown, which corresponds to the unique corresponding image portion (i.e., the unique corresponding angular image content) that will actually reach the eye 1705 from that bounce point. For each bounce point, its light path is shown as the channel that will contribute to the corresponding portion of the virtual image from the corresponding portion of the waveguide. For example, for the zeroth bounce B0, the light transmitted by waveguide 1708 is simply refracted and does not undergo any reflection therein. The light from the eighth holographic channel H8 reaches the eye from the zeroth bounce B0. For the next bounce B1, the light transmitted by waveguide 1702 undergoes one bounce therein before transmission. The light from the seventh hologram H7 reaches the eye from the next bounce B1. This continues sequentially until the light transmitted by waveguide 1708 at the last bounce B8 has undergone eight bounces before being transmitted and reaching the eye 1705, and includes the light encoded according to the first hologram H1. In this arrangement, light from each channel arrives at the observer simultaneously—from multiple different corresponding bounce points on the waveguide. Therefore, the observer receives holographic light corresponding to the entire virtual image simultaneously without moving their eyes or making any other changes, even if their eyes and the display device are relatively small and the viewing distance is relatively large.

[0345] exist Figures 17A to 17C In the example shown, light from only one image region reaches the eye from each bounce point. Therefore, when determining the hologram as described herein, a spatial correlation is established between regions of the virtual image and their corresponding bounce points on the waveguide. In some other examples, there may be relatively small overlap, such that a region of the image originates from two adjacent transmission points and is thus contained within two adjacent disks propagating from the waveguide to the observation plane.

[0346] Therefore, the inventors recognize that the methods and apparatus described herein can generate diffraction patterns (or light modulation patterns) including holograms, which, when displayed on an LCOS or other suitable display device, enable the efficient emission of spatially modulated light from multiple “disks” or holographic light channels, each “disk” or holographic light channel corresponding to (more specifically, encoding) a different corresponding portion of a corresponding virtual image.

[0347] Therefore, this paper describes improved methods and apparatuses that enable holograms to be computed and displayed on a suitable display device in such a way that an observer can see a clear image when the display device is illuminated by a suitable light source. The image seen by the observer can be free of ghosting and, at least in some embodiments, can be made brighter by the contribution of light, which would normally produce a ghosted image rather than a single main image. Furthermore, even if the viewing apertures (i.e., the eyes) of the display device and the observer are relatively small and the viewing distance is relatively large, they enable the observer to see an image, such as a virtual image, at a finite distance (rather than infinite distance) from the display device.

[0348] The improved methods and apparatus described herein can be executed in real time and can be repeated, for example, very rapidly, to adapt to changes in the position / orientation of the observation aperture. They can be implemented for more than one observation aperture, such as two eyes. They can be repeated, for example, very rapidly, to enable the display of multiple different holograms, thereby allowing the continuous and / or sequential, serial, patterned, or cyclical observation of multiple different corresponding images.

[0349] The improved methods and apparatus described herein can be implemented in a variety of different applications and viewing systems. For example, they can be implemented in head-up displays (HUDs). In many improvements to conventional HUDs, where the virtual image is formed at infinity, the improved methods and apparatus described herein can be implemented to create virtual images at a finite image distance (which can be selected and adjusted by a suitable controller) while still eliminating ghosting.

[0350] Although virtual images have been discussed here, which require the eye to convert received modulated light to form a perceived image, the improved methods and apparatus described here can be applied to real images.

[0351] Additional features

[0352] The embodiments relate to electrically activated LCOS spatial light modulators by way of example only. The teachings of this disclosure can be equivalently implemented on any spatial light modulator capable of displaying computer-generated holograms according to this disclosure, such as any electrically activated SLM, optically activated SLM, digital micromirror device, or microelectromechanical device.

[0353] In some embodiments, the light source is a laser, such as a laser diode.

[0354] The systems disclosed herein can be used to provide improved head-up displays (HUDs) or head-mounted displays. In some embodiments, a vehicle is provided that includes a holographic projection system mounted in the vehicle to provide a HUD. The vehicle may be a motorized vehicle, such as a car, truck, van, lorry, motorcycle, train, airplane, boat, or ship.

[0355] The quality of holographic reconstruction can be affected by the so-called zero-order problem, which is a result of the diffraction characteristics of using a pixelated spatial light modulator. This zero-order light can be considered as "noise" and includes, for example, specular reflections and other unwanted light from the SLM.

[0356] In this embodiment, only the primary playback field is utilized, and the system includes physical blocks, such as baffles, arranged to restrict the propagation of higher-level playback fields through the system.

[0357] In some embodiments, the holographic reconstruction is in color. In some embodiments, a method known as Spatial Separated Color (SSC) is used to provide color holographic reconstruction. In other embodiments, a method known as Frame Sequential Color (FSC) is used.

[0358] The SSC method uses three spatially separated arrays of light-modulated pixels for three monochrome holograms. An advantage of the SSC method is that the images can be very bright because all three holographic reconstructions can be formed simultaneously. However, if three spatially separated arrays of light-modulated pixels are provided on a common SLM due to spatial constraints, the quality of each monochrome image is suboptimal because only a subset of the available light-modulated pixels is used for each color. Therefore, relatively low-resolution color images are provided.

[0359] The FSC method can use all pixels of a common spatial light modulator to sequentially display three monochrome holograms. Cyclic monochrome reconstruction (e.g., red, green, blue, red, green, blue, etc.) is fast enough that a human observer can perceive a multicolor image from the synthesis of the three monochrome images. The advantage of FSC is that the entire SLM is used for each color. This means the quality of the resulting three color images is optimal because all pixels of the SLM are used for each color image. However, the disadvantage of the FSC method is that the brightness of the synthesized color image is approximately three times lower than that of the SSC method because each monochrome illumination event can only occur within one-third of the frame time. This disadvantage can be addressed by over-exciting the laser or by using a higher-power laser, but this requires greater power, leading to higher cost and increased system size.

[0360] The example describes illuminating an SLM with visible light, but those skilled in the art will understand that, for example, the light source and the SLM can also be used to guide infrared or ultraviolet light, as disclosed herein. For example, to provide information to a user, those skilled in the art will know the techniques used to convert infrared and ultraviolet light into visible light. For example, this disclosure extends to the use of phosphors and / or quantum dot techniques for this purpose.

[0361] Some embodiments describe 2D holographic reconstruction by way of example only. In other embodiments, the holographic reconstruction is 3D holographic reconstruction. That is, in some embodiments, each computer-generated hologram forms a 3D holographic reconstruction.

[0362] The methods and processes described herein can be embodied on a computer-readable medium. The term "computer-readable medium" includes media arranged for temporary or permanent storage of data, such as random access memory (RAM), read-only memory (ROM), buffer memory, flash memory, and cache memory. The term "computer-readable medium" should also be considered to include any medium or combination of media capable of storing instructions for machine execution, such that when the instructions are executed by one or more processors, the machine performs, wholly or partially, any or all of the methods described herein.

[0363] The term "computer-readable medium" also covers cloud-based storage systems. The term "computer-readable medium" includes, but is not limited to, one or more tangible and non-transitory data repositories (e.g., data volumes) in the example forms of solid-state storage chips, optical discs, disks, or any suitable combinations thereof. In some example embodiments, instructions for execution may be transmitted by a carrier medium. Examples of such carrier media include transient media (e.g., propagation signals for transmitting instructions).

[0364] It will be apparent to those skilled in the art that various modifications and variations can be made without departing from the scope of the appended claims. This disclosure covers all modifications and variations within the scope of the appended claims and their equivalents.

Claims

1. A diffraction structure arranged to spatially modulate light that can be converted into an image by an observation system, wherein, The diffraction structure is configured to route light into multiple holographic channels, each holographic channel corresponding to a different sub-region of the image, wherein the diffraction structure includes multiple pixels; The diffraction structure is arranged to spatially modulate light according to the hologram of the image; The diffraction structure is arranged to divide the content of the image by angle, such that the central axis of each hologram channel propagates from the diffraction structure at its own different angle. The diffraction structure is arranged such that each pixel of the diffraction structure contributes light to each hologram channel.

2. The diffraction structure as described in claim 1, wherein, Each hologram channel essentially consists of spatially modulated light based on the corresponding different sub-regions of the image.

3. The diffraction structure as described in claim 1, wherein it is arranged to spatially modulate the phase of light.

4. The diffraction structure as described in claim 1, wherein it is arranged to route light via a waveguide.

5. The diffraction structure as described in claim 4, wherein, The waveguide is arranged for pupil expansion.

6. The diffraction structure as described in claim 5, wherein, The cross-sectional shape of the light pattern that can be formed by each holographic channel basically corresponds to the shape of the incident aperture of the observation system.

7. The diffraction structure as described in claim 1, wherein, The holographic channels are spatially separated or at least partially spatially separated.

8. A system for image projection, comprising a diffraction structure as claimed in any one of claims 1 to 7, a waveguide arranged to receive spatially modulated light from the diffraction structure, and an observation system arranged to receive spatially modulated light via the waveguide.

9. The system of claim 8, wherein the light from each hologram channel arrives at the observation system from the diffraction structure along a different optical path.

10. The system of claim 9, wherein, The different optical paths include different numbers of reflections within the waveguide.

11. The system of claim 9, wherein, The different optical paths have different lengths.

12. The system as claimed in any one of claims 9 to 11, wherein, The different optical paths pass through the entrance aperture of the observation system at different corresponding angles.

13. The system as claimed in any one of claims 9 to 11, wherein, The waveguides are arranged such that at any observation position on the observation plane, all holographic channels are routed through the entrance aperture of the observation system.

14. The system of claim 13, wherein, For each permitted observation location, the waveguide routes each hologram channel to the observation system via only one optical path.

15. The system of claim 14, wherein, At least two of the plurality of holographic channels partially overlap at the entrance aperture of the observation system.

16. The system of claim 8, wherein, The diffraction structure is a phase hologram or a hologram.

17. A method for calculating a hologram of an image, the method comprising at least one step, the at least one step comprising cropping an optical path during calculation according to an incident pupil of an observation system to form a hologram, the hologram forming spatially modulated light when illuminated, wherein a continuous optical path of the spatially modulated light corresponds to a continuous region of the image, wherein the hologram comprises a plurality of pixels; in, The holograms are arranged to divide the content of the image by angle, such that the central axis of each hologram channel propagates from the hologram at its own different angle; The holograms are arranged such that each pixel of the hologram contributes light to each hologram channel.

Images