Method of reconstructing an image from a hologram

Abstract

A method of reconstructing an image from a hologram, comprising: receiving an image for display within a display area of a display system, wherein the display area is visible from a viewing area spatially separated therefrom; determining a first image component of the image; calculating a hologram of the image, wherein the hologram is configured to angularly distribute light in dependence on position within the image such that angularly distributed light corresponds to respective contiguous regions of the image; displaying the hologram on a display device and spatially modulating light in dependence on the displayed hologram; propagating the spatially modulated light through a pupil expander arranged to provide a plurality of different light propagation paths for the spatially modulated light from the display device to the viewing area, wherein each light propagation path corresponds to a respective contiguous region of the image due to the angular distribution of light from the hologram, wherein the method comprises allocating more data processing resources to calculating the hologram in respect of the first image component of the image than the second image component.

Description

Technical Field

[0001] This disclosure relates to image projection and methods for projecting images. This disclosure relates to image reconstruction and methods for reconstructing images from diffraction structures such as holograms or phase holograms. Embodiments relate to projecting images via a pupil dilator, such as a waveguide pupil dilator. This disclosure also relates to a method for optimizing the allocation of data processing resources, such as holographic computing resources. Some embodiments relate to optical engines, such as image projectors, holographic projectors, or image generation units. Some embodiments relate to head-up displays or vehicles housing head-up displays. Background Technology

[0002] Light scattered from an object contains amplitude and phase information. This amplitude and phase information can be captured on, for example, a photosensitive plate using well-known interferometry techniques to form a holographic record, or "hologram," including interference fringes. The hologram can be reconstructed by illuminating it with appropriate light to form a two-dimensional or three-dimensional holographic reconstruction or replay image representing the original object.

[0003] Computer-generated holography can numerically simulate interference processes. Computer-generated holograms can be calculated using techniques based on mathematical transformations such as Fresnel or Fourier transforms. These types of holograms are called Fresnel / Fourier transform holograms or simply Fresnel / Fourier holograms. A Fourier hologram can be considered a Fourier domain / plane representation of an object or a frequency domain / plane representation of an object. For example, computer-generated holograms can also be calculated using coherent ray tracing or point cloud techniques.

[0004] Computer-generated holograms can be encoded on spatial light modulators arranged to modulate the amplitude and / or phase of incident light. For example, optical modulation can be achieved using electrically addressable liquid crystals, optically addressable liquid crystals, or micromirrors.

[0005] Spatial light modulators typically comprise multiple individually addressable pixels, which may also be referred to as cells or elements. The light modulation scheme can be binary, multilevel, or sequential. Alternatively, the device can be sequential (i.e., excluding pixels), so the light modulation can be continuous on the device. Spatial light modulators can be reflective, meaning the modulated light is output as reflected light. Spatial light modulators can also be transmissive, meaning the modulated light is output as transmitted light.

[0006] The system described herein can be used to provide holographic projectors. For example, such projectors have already been used in head-up displays (HUDs) and light detection and ranging (LiDAR). Summary of the Invention

[0007] For ease of explanation and illustration, this disclosure and the accompanying drawings generally depict a one-dimensional case. However, those skilled in the art of optics will understand that the concepts described and illustrated can be extended to two dimensions to provide a two-dimensional image from a two-dimensional hologram. For example, although only one-dimensional pupil dilation is described and illustrated, the reader should understand that this disclosure extends to two-dimensional pupil dilation—for example, using two one-dimensional pupil dilations in series.

[0008] In summary, this disclosure relates to image projection. It relates to methods of image projection and image projectors including display devices. This disclosure also relates to projection systems including image projectors and observation systems. This disclosure is equally applicable to monocular and binocular observation systems. An observation system may include one or more eyes of an observer. An observation system includes optical elements with optical power (e.g., the lens of the human eye) and an observation plane (e.g., the retina of the human eye). The projector may be referred to as a "light engine." The display device and the image formed (or perceived) using the display device are spatially separated from each other. An observer forms or perceives the image on the display plane. In some embodiments, the image is a virtual image, and the display plane may be referred to as a virtual image / display plane. The image is formed by illuminating a diffraction pattern (e.g., a hologram) displayed on the display device.

[0009] Display devices consist of pixels. The pixels of a display device diffract light. According to well-known optical principles, the maximum diffraction angle depends on the pixel size (and other factors, such as the wavelength of light).

[0010] In this embodiment, the display device is a spatial light modulator, such as a liquid crystal on silicon (“LCOS”) spatial light modulator (SLM). Light travels from the LCOS to the observing entity / system, such as a camera or eye, within a diffraction angle range (e.g., from zero to the maximum diffraction angle). In some embodiments, amplification techniques may be used to increase the range of available diffraction angles beyond the conventional maximum diffraction angle of the LCOS.

[0011] In one embodiment, the image is a real image. In other embodiments, the image is a virtual image perceived by the human eye (or multiple eyes). The projection system or light engine can therefore be configured so that the observer looks directly at the display device. In such an embodiment, holographically encoded light propagates directly to the eye, and no intermediate holographic reconstruction is formed in the free space between the display device and the observer, or on the screen or other light-receiving surface. In such an embodiment, the pupil of the eye can be considered as the entrance aperture of the observation system, and the retina of the eye can be considered as the observation plane of the observation system. It is sometimes said that in this configuration, the lens of the eye performs the holographic-to-image conversion or transformation.

[0012] According to well-known optical principles, the range of angles at which light propagating from a display device can be observed by an eye or other observation entity / system varies with the distance between the display device and the observation entity. For example, at an observation distance of 1 meter, only a small range of angles from the LCOS can propagate through the pupil of the eye to form an image on the retina at a given eye location. The range of angles at which light propagates from the display device determines the portion of the image that the observer can successfully propagate through the pupil of the eye to form an image on the retina at a given eye location. In other words, not all parts of the image are visible from any point on the observation plane (e.g., any eye position within the observation window of an eye-tracking box).

[0013] In some embodiments, the image perceived by the observer is a virtual image appearing upstream of the display device; that is, the image is perceived by the observer as being farther away than the display device. Therefore, conceptually, the observer can be thought of viewing the virtual image through a "window the size of the display device," which can be very small, for example, 1 cm in diameter, at a relatively large distance, such as 1 meter. And the user will view the window the size of the display device through the pupil of their eye, which can also be very small. Therefore, at any given time, the field of view is small, and the range of specific angles that can be seen is highly dependent on eye position.

[0014] A pupil dilator addresses the problem of increasing the field of view—that is, increasing the angular range of light propagating from a display device so that the light can successfully pass through the pupil of the eye to form an image. The display device is (relatively) small, and the projection distance is (relatively) large. In some embodiments, the projection distance is at least one (e.g., at least two) orders of magnitude larger than the diameter or width of the entrance pupil and / or aperture of the display device (i.e., the size of the pixel array). This disclosure relates to so-called direct-view holography, in which a hologram of an image, rather than the image itself, is propagated to the human eye. In other words, the light received by the observer is modulated according to the hologram of the image.

[0015] A pupil expander increases the field of view, thus increasing the maximum propagation distance usable across the full diffraction angle of the display device. The use of a pupil expander also laterally enlarges the user's eyebox, allowing for some eye movement while still enabling the user to see the image. In this embodiment, the pupil expander is a waveguide pupil expander. This disclosure generally relates to non-infinite virtual image distances, i.e., near-field virtual images.

[0016] This paper discloses a method for computing holograms of an image, comprising at least one step of cropping a hologram according to the entrance pupil of an observation system to form a hologram that, when illuminated, forms spatially modulated light, wherein consecutive optical channels of the spatially modulated light correspond to consecutive regions of the image. The consecutive optical channels can be defined by a consecutive range of ray angles of the spatially modulated light. All consecutive optical channels of the spatially modulated light correspond to consecutive regions of the image. The spatially modulated light can be divided into any number of consecutive optical channels. In other words, the method disclosed herein determines a diffraction structure arranged to spatially modulate light, which can be varied by the observation system, into an image, wherein the diffraction structure is configured to route light into multiple hologram channels, each hologram channel corresponding to a different portion of the image.

[0017] In some cases, this holographic computation method for optical systems including waveguide pupil dimmers can utilize eye tracking to determine the position of the eye's pupil. When viewed from the pupil's position, a hologram is computed to provide a good image across the field of view. The eye-tracking tool determines the pupil's position and can also estimate the eye's gaze direction (i.e., the direction the eye is pointing). Some foveal imaging concepts provide higher image quality for the foveal region of the retina than for the rest of the field of view. However, this disclosure relates to much more than these. In a general sense, this disclosure relates to using foveal imaging concepts to manage hologram computation resources. More specifically, hologram computation resources dedicated to different regions of the field of view are selected based on gaze-tracking input, and the hologram is propagated through a pupil dimmer, where different regions of the field of view correspond to different optical paths through the pupil dimmer, which are in fact different sub-holograms.

[0018] Various aspects of this disclosure are defined in the appended independent claims.

[0019] A first aspect of this disclosure is a method for reconstructing an image from a hologram. The method includes first to fifth steps. These steps can be performed sequentially. The first step includes receiving an image for display within a display area of ​​a display system (e.g., a head-up display). The display area is visible or perceptible from an observation area spatially separated from it. That is, the observation area is spatially separated from the display area, for example, by 1-2 meters. In some examples, the display area may be an active display area of ​​a display device, such as a spatial light modulator visible from the observation area. In other examples, the display area may include a virtual display area formed by one or more virtual images / display planes visible from the observation area. The observation area may be an observation window, such as the eye box of a head-up display. The second step includes determining a first image component of the image. The third step includes determining or calculating a hologram of the image. The hologram is configured to distribute light at an angle according to positions within the image, such that angular channels of the angled light correspond to corresponding consecutive regions of the image. The fourth step includes displaying the hologram on a display device and spatially modulating the light according to the displayed hologram. The fifth step involves propagating each spatially modulated light beam through a pupil dilator, which is arranged such that the spatially modulated light provides multiple distinct light propagation paths from the display device to the viewing area. Due to the angular distribution of the light from the hologram, each light propagation path corresponds to a corresponding consecutive region of the image. The fifth step can be implemented by arranging the pupil dilator to receive the spatially modulated light from the display device. It is noteworthy that this method (e.g., the third step) involves allocating more data processing resources to compute the hologram relative to the first image component than to the second image component.

[0020] Readers will be familiar with the idea that in real-world holographic display devices, infinite time and processing power cannot be allocated to hologram computation. For example, if a holographic display device is required to operate at video rate, it is necessary to compute each hologram from each corresponding image frame at a rate faster than the frame / display / video rate of the input image stream. Similarly, if a holographic display device is required to meet maximum size or cost constraints, it may be necessary to use lower-specification electronic components, such as processors or memory. Therefore, in real-world devices, a limited number or amount of data processing resources are available for computing each hologram. The term "data processing resources" includes processing time and processing power only as an example, but readers will be familiar with the idea that a maximum amount or amount of data processing is available (i.e., allocated) for hologram computation. These data processing resources—regardless of their form—are typically distributed evenly during hologram computation. In fact, it is generally impossible to allocate hologram computation resources in any other way because each part of the hologram contributes to each part of the reconstruction. That is, there is no one-to-one pixel association between the hologram and the reconstructed image. In embodiments, the quality or fidelity of a hologram to the image it represents depends on the amount of data processing resources allocated to computation. That is, if more data processing resources are allocated to the computation of the hologram, the hologram will produce a higher quality image reconstruction. In other words, the difference between the source / target image and the holographically reconstructed image is reduced. It is generally desirable to allocate as many data processing resources as possible to the computation of each hologram, but according to this disclosure, due to the nature of holograms and the use of waveguide pupil dilators, data processing resources are unevenly allocated to each image. For example, some aspects or components of each image (e.g., sub-regions or color components) are allocated or assigned more holographic computation resources than other aspects or components of the same image. This will be further understood from the detailed description below.

[0021] In summary, the method according to this disclosure results in high image quality perceived by the user, but with low holographic computational resource requirements. Furthermore, it is highly compatible with holographic computation methods used for the optical system and utilizes a system that has already integrated eye tracking into holographic computation.

[0022] Importantly, the hologram is propagated to the observation system, not a holographic reconstruction (i.e., an image) formed from the hologram. It can be said that the spatially modulated light received by the observation system is in the holographic domain, not in the spatial or image domain. It can also be said that the observation system performs the holographic-to-image conversion. More specifically, optical elements such as lenses of each observation system perform the conversion. In this embodiment, no holographic reconstruction or image is formed between the display device and the observation system. In some embodiments, optionally, an interleaving scheme is used to compute different holograms and propagate them to each eye of the observer.

[0023] The display device has an active / pixel display area, which has a first dimension of less than 10 cm, for example, less than 5 cm or less than 2 cm. The propagation distance between the display device and the observation system can be greater than 1 m, for example, greater than 1.5 m or greater than 2 m. The optical propagation distance within the waveguide can be up to 2 m, for example, up to 1.5 m or up to 1 m. This method is capable of receiving images and determining corresponding holograms of sufficient quality in less than 20 ms, such as less than 15 ms or less than 10 ms.

[0024] The method disclosed herein forms a hologram configured to route light into multiple channels, each corresponding to a different portion (i.e., a sub-region) of an image. The hologram can be displayed, for example, on a display device such as a spatial light modulator. When displayed on a suitable display device, the hologram can spatially modulate the light that can be converted into an image by an observation system. The channels formed by the diffraction structure are referred to herein as “hologram channels” simply to reflect that they are light channels encoded by a hologram containing image information. It can be said that the light in each channel is in the holographic domain, rather than in the image or spatial domain. In some embodiments, the hologram is a Fourier or Fourier transform hologram, and therefore the hologram domain is Fourier or the frequency domain. The hologram can also be a Fresnel or Fresnel transform hologram. The hologram is described herein as routing light into multiple hologram channels simply to reflect that the image reconstructed from the hologram has a finite size and can be arbitrarily divided into multiple image sub-regions, where each hologram channel corresponds to each image sub-region. Importantly, the hologram of this disclosure is characterized by how it distributes image content when illuminated. Specifically, holograms divide image content by angles. That is, each point on the image is associated with a unique ray angle in the spatially modulated light formed by the hologram when illuminated—at least a unique pair of angles, since holograms are two-dimensional. To avoid confusion, this holographic behavior is not conventional. When illuminated, the spatially modulated light formed by this particular type of hologram can be arbitrarily divided into multiple holographic channels, each defined by a series of ray angles (two-dimensional). As can be understood above, any holographic channel (i.e., a subrange of ray angles) that can be considered in the spatially modulated light will be associated with a corresponding part or subregion of the image. That is, all the information needed to reconstruct that part or subregion of the image is contained within the subrange of angles of the spatially modulated light formed by the hologram of the image. When the spatially modulated light is viewed as a whole, there is not necessarily any evidence of multiple discrete optical channels. However, in some embodiments, multiple spatially separated holographic channels are formed by intentionally leaving the region of the target image for hologram computation blank or empty (i.e., without image content). In some embodiments, the optical channels are non-overlapping. In other embodiments—for example, embodiments that additionally include an optical combiner with optical power (e.g., a vehicle windshield) between the waveguide and the observer—some optical channels may at least partially overlap.

[0025] However, holograms can still be identified. For example, if only a continuous portion of a sub-region of spatially modulated light formed by the hologram is reconstructed, then only that sub-region of the image should be visible. If different continuous portions or sub-regions of spatially modulated light are reconstructed, then different sub-regions of the image should be visible. Another identifying feature of this type of hologram is that the shape of the cross-sectional region of any hologram channel substantially corresponds to the shape of the entrance pupil (i.e., substantially the same), although the dimensions may differ. Each light hologram channel propagates from the hologram at a different angle or angular range. While these are example ways of characterizing or identifying this type of hologram, other methods may also be used. In summary, the holograms disclosed herein are characterized and identified by how the image content is distributed within the hologram-encoded light, and the appended claims are stated accordingly.

[0026] Different propagation paths can pass through the entrance aperture of the observation system at different angles. The pupil dilator can be arranged such that all holographic channels are routed through the entrance aperture of the observation system at any observation position on the observation plane. For each permitted observation position, the pupil dilator routes each holographic channel to the observation system via only one propagation path. At least two of the multiple holographic channels can partially overlap at the entrance aperture of the observation system.

[0027] The image comprises multiple image components. In other words, the image can be decomposed into multiple image components. The term "image component" is used to encompass many different concepts disclosed herein. Image components can take different forms. Each image component can be a sub-region of the image. That is, a region or portion of the image, where different regions or portions of the image together constitute the complete image. By way of example only, a first image component may include half of the image, and a second image component may include the other half. Multiple image components may not overlap. In some embodiments, a first image component may correspond to a first sub-region of the image, and a second image component may correspond to a second sub-region of the image, wherein the first sub-region is different from the second sub-region, and optionally, the first and second sub-regions do not overlap. In some embodiments, the first sub-region corresponds to a first consecutive group of image pixels of the image, and the second sub-region corresponds to a second consecutive group of image pixels of the image. In some embodiments, the second consecutive group of image pixels partially or completely surrounds the first consecutive group of image pixels. In some embodiments, the first consecutive group of image pixels includes fewer image pixels than the second consecutive group of image pixels. In some other embodiments, the distribution of image pixels between the first and second groups changes repeatedly or continuously based on an input. This input may indicate the observer's pupil position. Alternatively, the input can indicate the image's attributes or features.

[0028] The computation of a hologram may include the computation of multiple sub-holograms. Each sub-hologram may correspond to a different region of the image. Allocating more data processing resources relative to a first image component than to a second image component may include allocating more data processing resources relative to the computation of the first sub-hologram corresponding to the first sub-region than to the computation of the second sub-hologram corresponding to the second sub-region.

[0029] It is noteworthy that the inventors have identified the synergistic effect between the feature holograms and pupil dilators described herein, as well as user tracking (e.g., eye tracking) that enables the computational resources associated with hologram computation to be optimized. Because each sub-hologram corresponds to a different part of the image, and each sub-hologram is computed at least semi-independently, the inventors recognized that, based on the visual importance of corresponding regions (i.e., sub-regions) of the image, they could spend more or less time on some sub-holograms relative to others. That is, a low-quality hologram might be suitable for unimportant regions of the image, but a high-quality hologram in other regions would result in an overall improvement in the perceived “quality” of the reconstructed image. The disclosure herein concerning the quantity or amount of “data processing resources” used for hologram computation can refer to processing power or time, etc. For example, a low-quality hologram can be computed very quickly, while a high-quality (or more accurate) hologram may require more processing time. In fact, according to this disclosure, each sub-hologram can be computed to different levels of precision. In some embodiments, a limited amount of time or a limited amount of data processing resources are available for hologram computation, and determining or computing at least one sub-hologram takes more time than determining or computing at least one other sub-hologram. Therefore, the reader will understand that some embodiments involve the allocation or, for example, continuous reallocation / distribution of processing resources within certain boundaries (e.g., time or processing power) set by real-world applications or systems. In some embodiments, the distribution of processing resources among sub-holograms is evaluated continuously, for example, at least once per frame of a video rate sequence of image frames. Furthermore, the inventors have identified unique synergies between different aspects of the disclosed holographic system that enable the valuable resources associated with hologram computation to be optimized (e.g., in real time), thereby providing the user with an improved visual experience.

[0030] The hologram can be a point cloud hologram, and allocating more data processing resources relative to the first sub-region than to the second sub-region can include using a higher density of point cloud data points relative to the first sub-region than to the second sub-region.

[0031] Computing a hologram may include performing iterative algorithms, and allocating more data processing resources relative to the first sub-region may include performing more algorithm iterations relative to the second sub-region of the image.

[0032] The inventors used an iterative phase retrieval algorithm, for example, as referenced here.Figure 7 and Figure 8 The described algorithm reveals a more favorable synergy. In the described algorithm, each sub-hologram is computed independently—at least to some extent—so the number of iterations for each sub-hologram can be determined independently. While some degree of convergence has been observed in practice, the more iterations used, the more accurate the phase retrieval / sub-hologram should be, and thus the higher the quality of the reconstructed image. Thus, in one example, the x-iteration of the algorithm is used to determine the first sub-hologram associated with an image region having relatively low visual impact / importance, and the y-iteration is used to determine the second sub-hologram associated with an image region having relatively high visual impact / importance, where y > x. In some examples, the region associated with the observer's foveal visual region is allocated more processing resources (e.g., algorithm iterations) than the observer's peripheral visual region. Similarly, the inventors have identified and utilized a unique synergy with the specific type of hologram disclosed herein, namely, holograms corresponding to image location guide lights in the holographic domain.

[0033] The method may include five stages. The first stage includes determining a first composite light field at the entrance pupil of the observation system. The first composite light field is generated by the propagation of light from the display plane of the display device along at least one light propagation path of the pupil dilator. The first stage also includes cropping according to the entrance pupil of the observation system. The second stage includes determining a second composite light field at the sensor plane of the sensor of the observation system. The second composite light field is generated by the propagation of light from the first composite light field from the entrance pupil through a lens of the observation system. The second stage also includes modifying the amplitude component according to the image. The third stage includes determining a third composite light field at the entrance pupil. The third composite light field is generated by the backpropagation of light from the second composite light field from the sensor plane through a lens. The third stage also includes cropping according to the entrance pupil. The fourth stage includes determining a fourth composite light field on the display plane. The fourth composite light field is generated by the propagation of light from the third composite light field along at least one light propagation path of the pupil dilator. The fourth stage also includes cropping according to the display device. The hologram is extracted from the fourth composite light field. Steps one through four can be repeated iteratively. With each iteration, the hologram converges, may improve, but tends to plateau. For example, the method can stop when the hologram that can be extracted from the fourth stage is deemed to have acceptable quality, or when the rate of change in each iteration is below a threshold, or when the allocated time has expired. To avoid ambiguity, the extracted hologram is the one intended for display on a display device.

[0034] The term "backward propagation" is used only to reflect that the direction of light propagation in the third and fourth stages is different from or substantially opposite to that in the first and second stages. In this respect, the light propagation in the first and second stages can be referred to as "forward propagation." In some embodiments, "forward propagation" and "backward propagation" are mathematical inverses of each other.

[0035] As used herein, the term "clipping" refers to the process of selectively discarding information (e.g., light field information) outside the region or range of interest (e.g., outside the optical aperture). In some embodiments, "clipping" is a data processing step that includes discarding data points outside the aperture, zeroing out data points, or simply ignoring data points.

[0036] The term "composite light field" is mentioned here. The term "light field" simply refers to a light pattern of finite size in at least two orthogonal spatial directions (x and y). The term "complex" as used here simply means that the light at each point in the light field can be defined by amplitude and phase values, and therefore can be represented by a complex number or a pair of values. For holographic computation purposes, the composite light field can be a complex two-dimensional array, where the complex numbers define the light intensity and phase at multiple discrete locations within the light field. According to the method disclosed herein, the composite light field propagates forward and backward along the +z and -z directions between the holographic plane (in the hologram / frequency / Fourier domain) and the image plane (in the image / spatial domain). Light propagation can be simulated or modeled using any of a variety of different methods or mathematical transformations familiar to those skilled in the art of wave optics.

[0037] At least one light propagation path may be just one of multiple light propagation paths provided by the pupil dilator. First through fourth stages can be performed on each of the multiple light propagation paths to extract a sub-hologram for each light propagation path. The multiple sub-holograms corresponding to the multiple light propagation paths are combined to form a hologram for display on a display device.

[0038] Before the step of extracting the sub-hologram from the final iteration, the first to fourth stages can be repeated iteratively for each light propagation path.

[0039] The first sub-hologram corresponding to the first sub-region can be computed using point cloud methods / algorithms. Alternatively, the second sub-hologram corresponding to the second sub-region can be computed using iterative methods / algorithms such as phase retrieval methods / algorithms.

[0040] If the changes in size and / or position within the image of the first sub-region are determined, the method can be repeated.

[0041] The method may also include recalculating the hologram relative to the first image component rather than the second image component if changes in size and / or position within the image with respect to the first sub-region are determined. In these embodiments, it is not always necessary to recalculate the second sub-region unless the image content has changed significantly, which can save frame-by-frame hologram recalculation.

[0042] The first sub-region of the image can be determined by (firstly) determining the corresponding first sub-region of the display area. The first sub-region of the image display can correspond to the foveal visual region of the observation system at the observation area. The observer can be a camera. The observer can be a human observer, in which case the observation system can be one eye or both eyes. The second sub-region of the image can be determined by (firstly) determining the corresponding second sub-region of the display area. The second sub-region of the display area can correspond to the non-foveal or peripheral visual region of the observation system.

[0043] The first sub-region of the image may extend beyond the corresponding foveal visual region. Alternatively, the first sub-region of the image may extend beyond the corresponding foveal visual region. The first sub-region may extend to include the entire image feature of the image partially contained within the foveal visual region. Therefore, the method may further include identifying at least one image feature, wherein the image feature is an identifiable object or display element of the image.

[0044] It is worth noting that the method may further include determining the position of the observer / observation system within the observation window to identify a first sub-region of the display area. For example, this method may include an eye, head, or gaze-tracking observation system.

[0045] This method may include determining the image region corresponding to the observer's blind spot based on eye, head, or gaze tracking. The method may also include processing the image before computing the hologram to remove image content corresponding to the observer's blind spot. This can save processing resources. For example, the blind spot of an eye can be determined through gaze tracking.

[0046] The method may also include determining the rate of change of eye or head position or gaze direction, and reducing the allocation of data processing resources for holographic computation if the rate of change of eye or head position or gaze direction is greater than a stored value. The human eye can move approximately 1000° per second. If the head moves rapidly, a high-quality image may not be necessary, thus saving holographic computation time in such cases.

[0047] The display device can have a variable display / frame rate. The method can also include updating the display device more quickly if the rate of change in eye or head position or gaze direction is greater than a stored value. This leverages the potential of using faster, lower-quality holographic computing. For example, smooth but low-quality updates can be used during head movement, rather than jerky, potentially unnecessary high-quality updates.

[0048] This method can also include predicting future eye or head position or gaze direction based on stored, associated data. Therefore, more data processing resources can be allocated to compute the hologram.

[0049] The method may also include increasing the intensity of a first sub-region of the image relative to a second sub-region before calculating the hologram of the image. For example, an increased intensity adjustment may be applied to make the image in the foveal visual region of the retina brighter as well (thus saving energy used in the rest of the image display).

[0050] According to this disclosure, image components can take different forms. That is, they may relate to different aspects or parts of the image. The image can be a multicolor image. The first image component can be a first monochrome image component of the image, and the second image component can be a second monochrome image component of the image. For each monochrome image component of the image, the steps of calculation, display, and propagation can be performed at least partially independently.

[0051] For example, the computation, display, and propagation steps associated with the first monochrome component of an image can be allocated more data processing resources compared to the same steps used for the second monochrome component of the same image. Image components can be, for example, the red, green, and blue components of a color image. In these embodiments, each image component is actually a monochrome image in itself, but for consistency and clarity of distinction between the (composite color) image and the (monochrome) image components, they are referred to herein as image components. In these embodiments—unlike other embodiments associated with sub-regions of the image—each image component can include the same number of pixels as each other, and the image is treated as a whole. The computation, display, and propagation steps for each monochrome image component can be performed in parallel or serially. A (monochrome) hologram is computed for each monochrome image component. Each (monochrome) hologram reconstructs the corresponding monochrome image component within the display area. Multiple monochrome reconstructions can be overlapped on the display area to reconstruct a pancolor image. In these embodiments, it can therefore be said that the method includes computed multiple holograms from the image or computed a hologram for each image component. However, it is clearly understood that in these cases, a hologram relative to an image component refers to the hologram of that image component.

[0052] The described methods for dividing an image into sub-regions and into monochrome components can be combined. For example, in some embodiments, a first sub-region of a first monochrome component may be allocated a first amount of data processing resources, a second sub-region of the first monochrome component may be allocated a second amount of data processing resources, and the second monochrome component may be allocated a third amount of data processing resources (spanning its entire region), wherein the first, second, and third amounts of data processing resources are all different from each other. For example, the second amount of data processing resources may be greater than the third amount of data processing resources, and the third amount of data processing resources may be greater than the first amount of data processing resources. The reader will understand that any combination, permutation, or mixture of these two concepts can be implemented according to this disclosure.

[0053] The light sensor of the observation system, which is arranged in the observation area to receive spatially modulated light, is more sensitive to light corresponding to the first monochromatic image component than to light corresponding to the second monochromatic image component.

[0054] The method may also include determining that a first monochrome image component is visually more dominant than a second monochrome image component in the image.

[0055] The method may also include determining parameters of the image background, such as color, and based on the determined image parameters, determining that a first monochrome image component is more likely to be visible on the background than a second monochrome image component.

[0056] Therefore, this disclosure includes prioritizing holographic computational resources for different primary colors (red, green, and blue). For example, more computation may be allocated to green, which is most sensitive to the eye, or to the primary colors that dominate the field of view, or to colors that may be more visible in the background scene.

[0057] The method may also include eye, head, or gaze tracking of the observation system to determine the foveal visual region of the display area, and altering the color balance of the reconstructed multicolor image in the image region corresponding to the non-foveal visual region of the display area.

[0058] The method may also include altering the color balance of the reconstructed polychromatic image in the non-retinal foveal visual region, including shifting the color balance toward 500 nm and / or reducing the intensity of any image light with wavelengths greater than 600 nm.

[0059] Some embodiments differentiate the color balance of the non-retinal fovea to make the overall display more efficient (utilizing the lower color sensitivity of peripheral vision (the visual rod), such as using more colors that the eye is most sensitive to detect and / or having higher electro-optical conversion efficiency). The visual rod of the human eye is most sensitive at a wavelength of 500 nm (therefore, green is preferred), while it is not sensitive to wavelengths greater than 640 nm (therefore, few dots of red are used in the peripheral area).

[0060] This paper discloses a light engine (e.g., an image projector, such as a holographic projector) arranged to reconstruct an image from a hologram. The light engine includes an image processor arranged to receive an image for display within a display area and to determine a first image component of the image. The display area is visible from an observation area spatially separated from it. The light engine may also include a holographic engine arranged to compute a hologram of the image, wherein the hologram is configured to distribute light at an angle according to positions within the image. Thus, angular channels of the angularly distributed light correspond to corresponding contiguous regions of the image. The light engine also includes a display device arranged to display the hologram and spatially modulate the light according to the displayed hologram. The light engine may also include a pupil dilator arranged to propagate spatially modulated light, thereby providing multiple distinct light propagation paths for the spatially modulated light from the display device to the observation area. Due to the angular distribution of light from the hologram, each light propagation path corresponds to a corresponding contiguous region of the image. The light engine is arranged to allocate more data processing resources to compute the hologram relative to a first image component than to a second image component of the image.

[0061] A holographic engine is also disclosed herein, arranged to determine a hologram of an image for observation using a head-up display (HUD). The HUD includes a display device and a pupil dilator. The HUD is configured to operate in conjunction with at least one observation system. Each observation system includes an entrance pupil on an entrance pupil plane, a lens on a lens plane, and a sensor on a sensor plane. The HUD may be configured to operate in conjunction with a pair of observation systems, such as a pair of eyes. The display device (e.g., a spatial light modulator) is arranged to display the hologram. The pupil dilator is arranged to receive light spatially modulated according to the hologram. For example, the displayed hologram may be illuminated with at least partially coherent light from a light source. The display device spatially modulates the received light according to the displayed hologram.

[0062] The holographic engine can be incorporated into the display driver, such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). The display driver can also be part of the image generation unit (PGU) used in a head-up display (HUD).

[0063] The term "hologram" is used to refer to a record containing amplitude or phase information about an object, or some combination thereof. The term "holographic reconstruction" is used to refer to the optical reconstruction of an object formed by illuminating a hologram. The system disclosed herein is described as a "holographic projector" because the holographic reconstruction is a real image and spatially separate from the hologram. The term "reproduced field" is used to refer to the 2D region within which the holographic reconstruction is formed and fully focused. If the hologram is displayed on a spatial light modulator including pixels, the reproduced field will be repeated in the form of multiple diffraction orders, where each diffraction order is a copy of the zero-order reproduced field. The zero-order reproduced field generally corresponds to the preferred or master reproduced field because it is the brightest reproduced field. Unless otherwise explicitly stated, the term "reproduced field" should be considered to refer to the zero-order reproduced field. The term "reproduced plane" is used to refer to a plane in space containing all reproduced fields. The terms "image," "reproduced image," and "image region" refer to the region of the reproduced field illuminated by the light reconstructed by the hologram. In some embodiments, an “image” may include discrete points, which may be referred to as “image points” or simply as “image pixels” for convenience.

[0064] The terms “encoding,” “writing,” and “addressing” are used to describe the process of providing multiple corresponding control values ​​to multiple pixels of an SLM, each determining the modulation level of the pixel. In other words, the pixels of an SLM are configured to “display” an optical modulation distribution in response to the receipt of multiple control values. Therefore, it can be said that an SLM “displays” a hologram, and a hologram can be considered an array of optical modulation values ​​or levels.

[0065] It has been found that acceptable-quality holographic reconstructions can be formed from "holograms" containing only phase information related to the Fourier transform of the original object. Such holographic records can be referred to as phase-only holograms. While the embodiments relate to phase-only holograms, this disclosure is equally applicable to amplitude-only holography.

[0066] This disclosure is equally applicable to forming holographic reconstructions using amplitude and phase information associated with the Fourier transform of the original object. In some embodiments, this is achieved by using complex modulation of a so-called fully complex hologram containing amplitude and phase information associated with the original object. Because the value (gray level) assigned to each pixel of the hologram has amplitude and phase components, such a hologram may be called a fully complex hologram. The value (gray level) assigned to each pixel can be represented as a complex number with amplitude and phase components. In some embodiments, a fully complex computer-generated hologram is computed.

[0067] The term "phase delay" can be a shorthand for the phase value, phase component, phase information, or simply phase of a pixel in a computer-generated hologram or spatial light modulator. That is, any phase value described is actually a number representing the amount of phase delay provided by that pixel (e.g., in the range of 0 to 2π). For example, a spatial light modulator is described as a pixel having a π / 2 phase value causing a π / 2 radian phase delay in the received light. In some embodiments, each pixel of a spatial light modulator can operate on one of a plurality of possible modulation values ​​(e.g., phase delay values). The term "gray level" can be used to refer to a plurality of available modulation levels. For example, the term "gray level" can be used for convenience to refer to a plurality of available phase levels in a phase modulator, even if different phase levels do not provide different shades of gray. For convenience, the term "gray level" can also be used to refer to a plurality of available complex modulation levels in a complex modulator.

[0068] Therefore, a hologram comprises an array of gray levels, i.e., an array of optical modulation values, such as phase delay values ​​or complex modulation values. A hologram is also considered a diffraction pattern because it is a pattern that causes diffraction when displayed on a spatial light modulator and illuminated with light whose wavelength is relative to (typically less than) the pixel spacing of the spatial light modulator. Reference is made herein to combining holograms with other diffraction patterns, such as those used as lenses or gratings. For example, a diffraction pattern used as a grating can be combined with a hologram to translate the playback field on the playback plane, or a diffraction pattern used as a lens can be combined with a hologram to focus the holographic reconstruction onto the playback plane in the near field.

[0069] Although different embodiments and groups of embodiments may be disclosed separately in the detailed description below, any feature of any embodiment or group of embodiments may be combined with any other feature or combination of features of any embodiment or group of embodiments. That is, all possible combinations and substitutions of the features disclosed in this disclosure are contemplated. Attached Figure Description

[0070] Specific embodiments are described by way of example only with reference to the following figures:

[0071] Figure 1 This is a schematic diagram illustrating a reflective SLM that generates holographic reconstruction on a screen;

[0072] Figure 2A The first iteration of the example Gerchberg-Saxton type algorithm is shown;

[0073] Figure 2B The second and subsequent iterations of the example Gerchberg-Saxton type algorithm are shown;

[0074] Figure 2CAlternative second and subsequent iterations of the example Gerchberg-Saxton type algorithm are shown;

[0075] Figure 3 This is a schematic diagram of a reflective LCOS SLM;

[0076] Figure 4 This shows the angular content of the virtual image effectively propagating from the display device to the aperture;

[0077] Figure 5A An observation system with a relatively short propagation distance is shown;

[0078] Figure 5B An observation system with a relatively large propagation distance is shown;

[0079] Figure 6A An observation system with a relatively large propagation distance is shown, which includes waveguides for forming virtual images at infinity;

[0080] Figure 6B It shows Figure 6A A magnified view of the optical path;

[0081] Figure 7 An optical system according to an embodiment is shown;

[0082] Figure 8 This is a flowchart illustrating the steps of a method according to an embodiment;

[0083] Figure 9A An image comprising multiple image regions (bottom) and a corresponding hologram comprising multiple holographic components (top) are shown;

[0084] Figure 9B A hologram according to the present disclosure is shown, characterized in that holographically encoded light is routed or directed into multiple discrete hologram channels;

[0085] Figure 10 An optimized system is shown, which is arranged to deliver the light content of each hologram channel to the eye via different optical paths;

[0086] Figure 11 The display area and example image are shown in the first gaze direction; and

[0087] Figure 12 The display area and example image for the second gaze direction are shown.

[0088] In all the accompanying drawings, the same reference numerals will be used to refer to the same or similar parts. Detailed Implementation

[0089] This invention is not limited to the embodiments described below, but extends to the full scope of the appended claims. That is, the invention may be implemented in different forms and should not be construed as limited to the described embodiments, which are illustrated for illustrative purposes.

[0090] Unless otherwise stated, singular terms may include plural forms.

[0091] A structure described as being formed above or below another structure should be interpreted as including situations where the structures are in contact with each other, and also including situations where a third structure is placed between them.

[0092] When describing temporal relationships, such as when the chronological order of events is described as “after,” “following,” “next,” “before,” etc., this disclosure should be considered to include both consecutive and discontinuous events, unless otherwise stated. For example, unless terms such as “exactly,” “immediately,” or “directly” are used, the description should be considered to include discontinuous cases.

[0093] Although the terms “first,” “second,” etc., may be used herein to describe various elements, these elements are not limited by these terms. These terms are used only to distinguish individual elements. For example, without departing from the scope of the appended claims, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element.

[0094] Features of different embodiments may be coupled or combined with each other in part or in whole, and may interoperate differently with each other. Some embodiments may be implemented independently of each other, or may be implemented together in a mutually dependent relationship.

[0095] Optical configuration

[0096] Figure 1 An embodiment is shown in which a computer-generated hologram is encoded on a single spatial light modulator. The computer-generated hologram is a Fourier transform of the object used for reconstruction. Therefore, the hologram can be described as a Fourier domain, frequency domain, or spectral domain representation of the object. In this embodiment, the spatial light modulator is a reflective liquid crystal on silicon (LCOS) device. The hologram is encoded on the spatial light modulator, and a holographic reconstruction is formed at the reproduction field, such as a light-receiving surface like a screen or diffuser.

[0097] A light source 110, such as a laser or laser diode, is configured to illuminate the SLM 140 via a collimating lens 111. The collimating lens causes the approximately plane wavefront of the light to be incident on the SLM. Figure 1In this embodiment, the wavefront is oriented off-normally (e.g., two or three degrees away from a plane that is truly orthogonal to the transparent layer). However, in other embodiments, the approximately planar wavefront is provided with normal incidence, and beam splitters are arranged to separate the input and output optical paths. Figure 1 In the illustrated embodiment, the arrangement is such that light from the light source is reflected from the mirrored rear surface of the SLM and interacts with the light modulation layer to form an outgoing wavefront 112. The outgoing wavefront 112 is applied to an optics device including a Fourier transform lens 120, the focal point of which is located at screen 125. More specifically, the Fourier transform lens 120 receives the modulated beam from the SLM 140 and performs a frequency-space transformation to produce a holographic reconstruction at screen 125.

[0098] It is worth noting that in this type of hologram, each pixel of the hologram contributes to the overall reconstruction. There is no one-to-one correlation between a specific point (or image pixel) on the playback field and a specific optical modulation element (or hologram pixel). In other words, the modulated light leaving the optical modulation layer is distributed across the entire playback field.

[0099] In these embodiments, the spatial position of the holographic reconstruction is determined by the diopter (focusing) of the Fourier transform lens. Figure 1 In the illustrated embodiment, the Fourier transform lens is a physical lens. That is, the Fourier transform lens is an optical Fourier transform lens and performs a Fourier transform optically. Any lens can act as a Fourier transform lens, but the performance of the lens will limit the accuracy of the Fourier transform it performs. Those skilled in the art will understand how to use lenses to perform optical Fourier transforms.

[0100] Traditional hologram calculation

[0101] In some embodiments, the computer-generated hologram is a Fourier transform hologram, or simply a Fourier hologram or a Fourier-based hologram, wherein the image is reconstructed in the far field by utilizing the Fourier transform properties of a positive lens. The Fourier hologram is computed by Fourier transforming the desired light field in the reproduction plane back to the lens plane. The Fourier transform can be used to compute computer-generated Fourier holograms.

[0102] Algorithms such as the Gerchberg-Saxton algorithm can be used to compute Fourier transform holograms. Furthermore, the Gerchberg-Saxton algorithm can be used to compute holograms in the Fourier domain (i.e., Fourier transform holograms) based solely on amplitude-only information in the spatial domain (e.g., a photograph). This effectively "retrieves" phase information related to the object from the amplitude-only information in the spatial domain. In some embodiments, the Gerchberg-Saxton algorithm or its variants are used to compute computer-generated holograms from amplitude-only information.

[0103] The Gerchberg-Saxton algorithm takes into account the fact that the intensity cross section I of the beams in planes A and B is known. A (x,y) and I B (x,y) and I A (x,y) and I B The case where (x,y) is correlated via a single Fourier transform. For a given intensity cross section, the approximate phase distribution Ψ in planes A and B is obtained. A (x,y) and Ψ B (x,y). The Gerchberg-Saxton algorithm finds a solution to the problem by following an iterative process. More specifically, the Gerchberg-Saxton algorithm iteratively applies spatial and spectral constraints while repeatedly transferring the representation I between the spatial and Fourier (spectral or frequency) domains. A (x,y) and I B A dataset (x, y) containing amplitude and phase. A corresponding computer-generated hologram in the spectral domain is obtained through at least one iteration of the algorithm. The algorithm is convergent and arranged to produce a hologram representing the input image. The hologram can be an amplitude-only hologram, a phase-only hologram, or a fully complex hologram.

[0104] In some embodiments, the phase-only hologram is computed using an algorithm based on the Gerchberg-Saxton algorithm, such as the algorithm described in British Patents 2498170 or 2501112, the entire contents of which are incorporated herein by reference. However, the embodiments disclosed herein are described by way of example only when calculating the phase-only hologram. In these embodiments, the Gerchberg-Saxton algorithm retrieves the phase information Ψ[u,v] of the Fourier transform of a dataset, which produces known amplitude information T[x,y], where the amplitude information T[x,y] represents the target image (e.g., a photograph). Since amplitude and phase are inherently combined in the Fourier transform, the transformed amplitude and phase contain useful information about the accuracy of the computed dataset. Therefore, the algorithm can be used iteratively with feedback of amplitude and phase information. However, in these embodiments, the phase-only information Ψ[u,v] is used as a hologram to form a holographic representation of the target image at the image plane. The hologram is a dataset of phase values ​​(e.g., a 2D array).

[0105] In other embodiments, an algorithm based on the Gerchberg-Saxton algorithm is used to compute a fully complex hologram. A fully complex hologram is a hologram having amplitude and phase components. A hologram is a dataset (e.g., a 2D array) comprising an array of complex data values, where each complex data value includes an amplitude component and a phase component.

[0106] In some embodiments, the algorithm processes complex data, and the Fourier transform is a complex Fourier transform. Complex data can be viewed as comprising (i) real and imaginary components, or (ii) amplitude and phase components. In some embodiments, the two components of the complex data are processed differently at different stages of the algorithm.

[0107] Figure 2A A first iteration of an algorithm for computing a phase-only hologram, according to some embodiments, is shown. The input to the algorithm is an input image 210 comprising a 2D array of pixel or data values, where each pixel or data value is an amplitude or oscillation value. That is, each pixel or data value of the input image 210 does not have a phase component. Therefore, the input image 210 can be considered as an amplitude-only, oscillation-only, or intensity-only distribution. An example of such an input image 210 is a photograph or a frame of a video comprising a time-series of frames. The first iteration of the algorithm begins with a data formation step 202A, which includes assigning random phase values ​​to each pixel of the input image using a random phase distribution (or random phase seed) 230 to form an initial complex dataset, where each data element of the dataset includes both amplitude and phase. In other words, the initial complex dataset represents the input image in the spatial domain.

[0108] First processing block 250 receives an initial complex dataset and performs a complex Fourier transform to form a complex dataset of Fourier transforms. Second processing block 253 receives the complex dataset of Fourier transforms and outputs a hologram 280A. In some embodiments, hologram 280A is a phase-only hologram. In these embodiments, second processing block 253 quantizes each phase value and sets each amplitude value to 1 to form hologram 280A. Each phase value is quantized according to the phase level that can be represented on the pixel of the spatial light modulator that will be used to "display" the phase-only hologram. For example, if each pixel of the spatial light modulator provides 256 different phase levels, each phase value of the hologram is quantized to one of the 256 possible phase levels. Hologram 280A is a phase-only Fourier hologram representing an input image. In other embodiments, hologram 280A is a fully complex hologram comprising an array of complex data values ​​(each including an amplitude component and a phase component) derived from the received complex dataset of Fourier transforms. In some embodiments, the second processing block 253 constrains each complex data value to one of a plurality of permissible complex modulation levels to form a hologram 280A. The constraint step may include setting each complex data value to the closest permissible complex modulation level in the complex plane. The hologram 280A can be said to represent an input image in the spectral, Fourier, or frequency domain. In some embodiments, the algorithm stops at this point.

[0109] However, in other embodiments, the algorithm continues, such as Figure 2AAs shown by the dashed arrow in the image. In other words, follow... Figure 2A The steps indicated by the dashed arrows are optional (i.e., not essential for all embodiments).

[0110] The third processing block 256 receives the modified complex dataset from the second processing block 253 and performs an inverse Fourier transform to form a complex dataset with an inverse Fourier transform. The complex dataset with the inverse Fourier transform can be said to represent the input image in the spatial domain.

[0111] The fourth processing block 259 receives the complex dataset of the inverse Fourier transform and extracts the distribution of amplitude values ​​211A and the distribution of phase values ​​213A. Optionally, the fourth processing block 259 evaluates the distribution of amplitude values ​​211A. Specifically, the fourth processing block 259 can compare the distribution of amplitude values ​​211A of the complex dataset of the inverse Fourier transform with the input image 510, which itself is, of course, the distribution of amplitude values. If the difference between the distribution of amplitude values ​​211A and the input image 210 is sufficiently small, the fourth processing block 259 can determine that the hologram 280A is acceptable. That is, if the difference between the distribution of amplitude values ​​211A and the input image 210 is sufficiently small, the fourth processing block 259 can determine that the hologram 280A is a sufficiently accurate representation of the input image 210. In some embodiments, for comparison purposes, the distribution of phase values ​​213A of the complex dataset of the inverse Fourier transform is ignored. It will be understood that any number of different methods can be used to compare the distribution of amplitude values ​​211A with the input image 210, and this disclosure is not limited to any particular method. In some embodiments, the mean squared error is calculated, and if the mean squared error is less than a threshold, the hologram 280A is considered acceptable. If the fourth processing block 259 determines that the hologram 280A is unacceptable, further iterations of the algorithm can be performed. However, this comparison step is not required, and in other embodiments, the number of iterations of the algorithm performed is predetermined, preset, or user-defined.

[0112] Figure 2B This represents the second iteration of the algorithm and any further iterations of the algorithm. The distribution of the phase values ​​213A from previous iterations is fed back through the algorithm's processing block. Distributions of amplitude values ​​211A are rejected, favoring the distribution of amplitude values ​​of the input image 210. In the first iteration, data formation step 202A forms a first complex dataset by combining the distribution of amplitude values ​​of the input image 210 with the random phase distribution 230. However, in the second and subsequent iterations, data formation step 202B includes forming a complex dataset by combining (i) the distribution of phase values ​​213A from previous iterations of the algorithm with (ii) the distribution of amplitude values ​​of the input image 210.

[0113] Then, with reference Figure 2AThe same method described is handled by Figure 2B The complex dataset formed in step 202B is used to form the second iterative hologram 280B. Therefore, the description of this process will not be repeated here. The algorithm can stop when the second iterative hologram 280B has been computed. However, any number of further iterations of the algorithm can be performed. It will be understood that the third processing block 256 is only needed if a fourth processing block 259 is required or further iterations are needed. The output hologram 280B generally improves with each iteration. However, in practice, a point is often reached where measurable improvement is no longer observable, or the positive benefits of performing further iterations are offset by the negative impact of the additional processing time. Therefore, the algorithm is described as iterative and convergent.

[0114] Figure 2C This represents an alternative embodiment for the second and subsequent iterations. The distribution of the phase value 213A from the previous iteration is fed back through the algorithm's processing block. The distribution of the amplitude value 211A is rejected, favoring an alternative distribution of the amplitude value. In this alternative embodiment, the alternative distribution of the amplitude value is derived from the distribution of the amplitude value 211 from the previous iteration. Specifically, processing block 258 subtracts the distribution of the amplitude value of the input image 210 from the distribution of the amplitude value 211 from the previous iteration, scales the difference by a gain factor α, and subtracts the scaled difference from the input image 210. This is mathematically expressed by the following equation, where the subscript text and numbers represent the iteration number:

[0115] R n+1 [x,y]=F'{exp(iψ n [u,v])}

[0116] ψ n [u,v]=∠F{η·exp(i∠R n [x,y])}

[0117] η=T[x,y]-α(|R n [x,y]|-T[x,y])

[0118] in:

[0119] F' is the inverse Fourier transform;

[0120] F is the forward Fourier transform;

[0121] R[x,y] is the complex dataset output by the third processing block 256;

[0122] T[x,y] is the input or target image;

[0123] ∠ is the phase component;

[0124] Ψ is a phase-only hologram 280B;

[0125] η is a new distribution of amplitude value 211B; and

[0126] α is the gain factor.

[0127] The gain factor α can be fixed or variable. In some embodiments, the gain factor α is determined based on the size and rate of the input target image data. In some embodiments, the gain factor α depends on the number of iterations. In some embodiments, the gain factor α is only a function of the number of iterations.

[0128] In all other respects, Figure 2C Implementation examples and Figure 2A and Figure 2B The implementation is the same. It can be said that only the phase hologram Ψ(u,v) includes the phase distribution in the frequency or Fourier domain.

[0129] In some embodiments, a spatial light modulator is used to perform a Fourier transform. Specifically, holographic data is combined with second data that provides optical power. That is, the data written into the spatial light modulator includes holographic data representing an object and lens data representing a lens. When displayed on the spatial light modulator and illuminated with light, the lens data simulates a physical lens—that is, it focuses light in the same way as a corresponding physical optical element. Therefore, the lens data provides optical power or focusing power. In these embodiments, [the following can be omitted] Figure 1A physical Fourier transform lens 120 is used. Data representing the lens is known. This data can be referred to as a software lens. For example, a phase-only lens can be formed by calculating the phase delay caused by the optical path length at each point of the lens due to its refractive index and spatial variation. For example, the optical path length at the center of a convex lens is greater than the optical path length at the edge of the lens. An amplitude-only lens can be formed from Fresnel zone plates. In the field of computer-generated holography, it is also known how to combine data representing the lens with a hologram to perform a Fourier transform of the hologram without requiring a physical Fourier lens. In some embodiments, the lensed data is combined with the hologram by simple addition, such as simple vector addition. In some embodiments, a physical lens is used in combination with a software lens to perform the Fourier transform. Alternatively, in other embodiments, the Fourier transform lens is omitted entirely, allowing holographic reconstruction to occur in the far field. In further embodiments, the hologram can be combined with grating data—i.e., data arranged to perform grating functions such as image steering—in the same manner. Again, how to calculate such data is known in the art. For example, a phase-only grating can be formed by modeling the phase delay caused by each point on the surface of the blazed grating. An amplitude-only grating can be simply superimposed on an amplitude-only hologram to provide angular steering for holographic reconstruction. The second data providing lensing and / or steering may be referred to as an optical processing function or optical processing pattern to distinguish it from the holographic data, which may be referred to as an image forming function or image forming pattern.

[0130] In some embodiments, the Fourier transform is performed jointly by a physical Fourier transform lens and a software lens. That is, the software lens provides some of the optical power that contributes to the Fourier transform, while one or more physical optics provide the remaining optical power that contributes to the Fourier transform.

[0131] In some embodiments, a real-time engine is provided, arranged to receive image data using an algorithm and compute holograms in real time. In some embodiments, the image data is video comprising a sequence of image frames. In other embodiments, the holograms are pre-computed, stored in computer memory, and retrieved as needed for display on an SLM. That is, in some embodiments, a library of predetermined holograms is provided.

[0132] The embodiments described herein are by way of example only and involve Fourier holography and Gerchberg-Saxton type algorithms. This disclosure is equally applicable to Fresnel holography and Fresnel holograms that can be computed using similar methods. This disclosure is also applicable to holograms computed using other techniques, such as point cloud-based methods.

[0133] Optical modulation

[0134] Spatial light modulators can be used to display diffraction patterns, including computer-generated holograms. If the hologram is a phase-only hologram, a spatial light modulator is needed to modulate the phase. If the hologram is a fully complex hologram, a spatial light modulator that modulates both the phase and amplitude can be used, or a first spatial light modulator that modulates the phase and a second spatial light modulator that modulates the amplitude can be used.

[0135] In some embodiments, the light modulation element (i.e., pixel) of the spatial light modulator is a cell comprising liquid crystal. That is, in some embodiments, the spatial light modulator is a liquid crystal device in which the optically active component is liquid crystal. Each liquid crystal cell is configured to selectively provide multiple light modulation levels. That is, each liquid crystal cell is configured at any time to operate at one light modulation level selected from multiple possible light modulation levels. Each liquid crystal cell can be dynamically reconfigured to a light modulation level different from the multiple light modulation levels. In some embodiments, the spatial light modulator is a reflective liquid crystal on silicon (LCOS) spatial light modulator, but this disclosure is not limited to this type of spatial light modulator.

[0136] LCOS devices provide a dense array of light-modulating elements or pixels within a small aperture (e.g., a few centimeters wide). Pixels are typically about 10 micrometers or smaller, resulting in a diffraction angle of a few degrees, meaning the optical system can be compact. The small aperture of an LCOS SLM is much easier to fully illuminate than the larger apertures of other liquid crystal devices. LCOS devices are typically reflective, meaning the circuitry driving the LCOS SLM pixels can be buried beneath the reflective surface. This results in a higher aperture ratio. In other words, the pixels are densely packed, meaning there are virtually no dead zones between pixels. This is advantageous because it reduces optical noise in the playback field. LCOS SLMs use a silicon substrate, which has the advantage of optically flat pixels. This is particularly important for phase modulation devices.

[0137] The following are just examples for reference. Figure 3 To describe a suitable LCOS SLM, an LCOS device is formed using a single-crystal silicon substrate 302. It has a 2D array of square planar aluminum electrodes 301, spaced apart by gaps 301a, arranged on the upper surface of the substrate. Each electrode 301 can be addressed by circuitry 302a buried in the substrate 302. Each electrode forms its own planar mirror. An alignment layer 303 is disposed on the electrode array, and a liquid crystal layer 304 is disposed on the alignment layer 303. A second alignment layer 305 is disposed on a planar transparent layer 306, for example, made of glass. A single transparent electrode 307, for example made of ITO, is disposed between the transparent layer 306 and the second alignment layer 305.

[0138] Each square electrode 301, together with the area covered by the transparent electrode 307 and the intermediate liquid crystal material, defines a controllable phase modulation element 308, commonly referred to as a pixel. Taking into account the space between pixels 301a, the effective pixel area, or fill factor, is the percentage of the total number of optically active pixels. By controlling the voltage applied to each electrode 301 relative to the transparent electrode 307, the properties of the liquid crystal material of the individual phase modulation elements can be altered, thereby providing a variable delay for light incident upon them. The effect is to provide phase-only modulation to the wavefront, i.e., without amplitude effects.

[0139] The described LCOS SLM outputs spatially modulated light in a reflective manner. The advantage of a reflective LCOS SLM is that the signal lines, grating lines, and transistors are located below the mirror, resulting in a high fill factor (typically greater than 90%) and high resolution. Another advantage of using a reflective LCOS spatial light modulator is that the thickness of the liquid crystal layer can be half that required when using a transmissive device. This significantly improves the switching speed of the liquid crystal (a key advantage for projecting moving video images). However, the teachings of this disclosure can also be implemented using a transmissive LCOS SLM.

[0140] Image projection using small display devices and long viewing distances

[0141] This disclosure relates to image projection in which the distance between a display device and an observer is much larger than the size of the display device. The viewing distance (i.e., the distance between the observer and the display device) can be at least one order of magnitude larger than the size of the display device. The viewing distance can be at least two orders of magnitude larger than the size of the display device. For example, the pixel area of ​​the display device can be 10 mm × 10 mm, and the viewing distance can be 1 m. The image projected by the system is formed on a display plane that is spatially separated from the display device.

[0142] According to this disclosure, the image is formed by holographic projection. The hologram is displayed on a display device. The hologram is illuminated by a light source (not shown), and the image is perceived on a display plane spatially separated from the hologram. The image can be real or virtual. For the purposes of the following explanation, it is helpful to consider a virtual image formed upstream of the display device—that is, appearing behind the display device. However, it is not important whether the image is virtual or not, and this disclosure applies equally to real images formed between the display device and the viewing system.

[0143] Display devices consist of pixels that display holograms. The pixel structure of a display device is diffracted. Therefore, the size of the hologram is determined by diffraction rules. See below for reference. Figure 4 The results of the interpretation of the diffraction characteristics of the display device are presented.

[0144] Figure 4A pixelated display device 402 is shown, arranged to display a hologram forming a virtual image 401 upstream of the display device 402. The diffraction angle q of the display device determines the size of the virtual image 401. The virtual image 401, the display device 402, and the observation system 405 are arranged on the optical axis Ax.

[0145] The observation system 405 has an entrance aperture 404 and an observation plane 406. The observation system 405 can be a human eye. Therefore, the entrance aperture 404 can be the pupil of the eye, and the observation plane 406 can be the retina of the eye.

[0146] The light propagating between the display device 402 and the observation system 405 is modulated by the hologram of the image (not the image itself). However, Figure 4 This illustrates how a hologram divides the content of a virtual image by angle. Each illustrated beam of light relates to a different portion of the virtual image 401. More specifically, the light in each beam is encoded by the hologram with information about a portion of the virtual image. Figure 4 Five example light beams are shown, each characterized by a corresponding angle relative to the optical axis Ax, and each beam represents a corresponding portion of the virtual image. In this example, one beam passes through the pupil 404, while the other four beams are blocked by the pupil 404. Similarly, the five different light beams correspond to five different portions of the virtual image 401. The complete image content of the virtual image is effectively divided by angle. The beams propagating along the optical axis Ax carry the central portion of the image information, i.e., information related to the image center. The other beams carry other portions of the image information. The two beams shown at the ends of the light cone carry the edge portions of the image information. The result of dividing the image information by angle is that not all image content can pass through the entrance aperture 404 of the observation system at a given viewing position. In other words, not all image content is received by the eye. Figure 4 In the example shown, only one of the five beams passes through the pupil 404 at any viewing position. The reader will understand that the five beams are shown only by way of example, and the process described is not limited to dividing the image information of a virtual image into just five beams.

[0147] In this example, the central portion of the image information is received by the eye. The edges of the image information are blocked by the pupil. The reader will understand that if the observer moves up or down, the eye may receive different beams of light; for example, the central portion of the image information may be blocked. Therefore, the observer can only see a portion of the entire image. The remaining image information is blocked by the incident pupil. The observer's field of vision is severely limited because they are actually viewing the image through a small aperture in the display device itself.

[0148] In short, light travels from a display device within a diffraction angle range. At a viewing distance of 1 meter, for a given eye position, only a small range of angles from the display device can travel through the pupil of the eye to form an image on the retina. The visible portion of a virtual image is only what falls within the diffraction angle. Figure 4 The field of view is shown as a small angular range passing through the incident aperture. Therefore, the field of view is very small, and the specific angular range depends heavily on the eye position.

[0149] refer to Figure 4 The issues of a small field of view and sensitivity to eye position are a result of the large viewing distance and small aperture of the display device. The importance of viewing distance is further explained with reference to Figures 5 through 7.

[0150] Figure 5A A display device 502 is shown, arranged to display a hologram and propagate light modulated according to the hologram to an observation system including an incident aperture 504 and an observation plane 506. The virtual image 501 is at infinity, so the light rays tracked between the virtual image and the display device are collimated. Figure 5A The lower part shows an enlarged view of the observation system. This view is schematic and therefore does not show the physiological details of the eye. In reality, of course, there is a light source arranged to illuminate the display device 502. Figure 5A (Not shown in the image).

[0151] Figure 5A Only light rays that can propagate through aperture 504 are shown; any other light rays that cannot pass through aperture 504 are omitted. However, it will be understood that in practice, those other light rays will also propagate from display device 502. Figure 5A In this configuration, the distance between the display device and the viewing plane is small enough that the full diffraction angle from the display device can form an image on the retina. All light propagation paths from the virtual image pass through the incident aperture. Therefore, all points on the virtual image are mapped onto the retina, and all image content is transmitted to the viewing plane. Consequently, the field of view for perceiving the image is maximized. At the optimal position, the field of view equals the diffraction angle of the display device. Interestingly, different image points on the retina are formed by light propagating from different areas of the display device 502, for example, the closest to the retina. Figure 5A The image dot at the top is formed solely by light propagating from the bottom of the display device. Light propagating from other areas of the display device does not contribute to that image dot.

[0152] Figure 5B This shows what happens as the observation distance increases.

[0153] More in detail, Figure 5BA display device 502' is shown, which is arranged to display a hologram and propagate light modulated according to the hologram to an observation system including an incident aperture 504' and an observation plane 506'. The virtual image 501' is at infinity, so the light between the virtual image and the display device is collimated. Figure 5B The lower part shows an enlarged view of the observation system. This view is schematic and therefore does not show the physiological details of the eye. In reality, it certainly has a light source arranged to illuminate the display device 502'. Figure 5B (Not shown in the image).

[0154] Figure 5B Only light rays that can propagate through an aperture of 504' are shown. Figure 5B At larger viewing distances, some light beams are blocked by the entrance aperture 504'. Specifically, light beams associated with the edges of the virtual image are blocked by the entrance pupil 504'. Therefore, the entire virtual image is invisible, and the visible portion of the virtual image heavily depends on eye position. Thus, large distances between the display device and the viewing system are problematic due to the small size of the display device.

[0155] Figure 6A An improved system including a display device 602 is shown, which propagates light encoded with a hologram displayed on the display device 602 into an observation system including an entrance aperture 604 and an observation plane 606. In fact, a light source (not shown) is, of course, arranged to illuminate the display device 602. The improved system also includes a waveguide 608 located between the display device 602 and the entrance aperture 604. Figure 6A The lower part shows an enlarged view of the entrance pupil 604 and the viewing plane 606. This figure is schematic and therefore does not show the physiological details of the eye.

[0156] The observation distance in Figure 6 is... Figure 5B The same. However, in Figure 5B The blocked beam of light is effectively recovered by waveguide 608, allowing the observation system to receive complete image information—despite the longer observation distance.

[0157] The presence of waveguide 608 allows content from all angles of the display device 602 to be received by the eye, even at this relatively large projection distance. This is because waveguide 608 acts as a pupil dilator in a well-known manner, and will therefore be described only briefly here.

[0158] In short, waveguide 608 comprises a generally elongated structure. In this example, it comprises an optical plate of refractive material, but other types of waveguides are also well known and can be used. Waveguide 608 is positioned to intersect, for example, at an oblique angle, a light cone projected from display device 602. The size, position, and orientation of waveguide 608 are configured to ensure that light from each of the five beams within the light cone enters waveguide 608. Light from the light cone enters waveguide 608 via a first planar surface 610 (located closest to display device 602) and is guided at least partially along the length of waveguide 608 before being emitted via a second planar surface 612 substantially opposite the first surface 610 (located closest to the eye). It is readily understood that the second planar surface 612 is partially reflective and partially transmissive. In other words, as each ray of light propagates within waveguide 608 from its first planar surface 610 to its second planar surface 612, some light is transmitted through waveguide 608, and some is reflected back to its first planar surface 610 by the second planar surface 612. The first planar surface 610 is reflective, such that all light striking it from within waveguide 608 is reflected back to its second planar surface 612. Therefore, some light can be simply refracted between the two planar surfaces 610, 612 of waveguide 608 before transmission, while other light can be reflected and thus undergo one or more reflections (or "bouncing") between the planar surfaces 610, 612 of waveguide 608 before transmission. Thus, the net effect of waveguide 608 is that the transmission of light is effectively extended to multiple locations on the second planar surface 612 of waveguide 608. Therefore, compared to the case without waveguide 608, all angular content output by display device 602 can appear at a greater number of locations on the display plane (and at a greater number of locations on the aperture plane). This means that light from each beam can enter the entrance aperture 604 and contribute to the image formed by the viewing plane 606, despite the relatively large projection distance. In other words, the eye can receive content from all angles of the display device 602. Therefore, the full diffraction angle of the display device 602 is utilized, and the viewing window is maximized for the user. In turn, this means that all light contributes to the perceived virtual image 601.

[0159] Figure 6B The diagram shows the individual optical path of each of the five ray beams contributing to five corresponding image points within the virtual image 601. Figure 6AThe light from each of R1 and R2 is simply refracted and then transmitted through waveguide 608. On the other hand, the light from R4 experiences a single bounce before transmission. The light from R3 comprises some light from the corresponding first portion of display device 602, which is simply refracted by waveguide 608 before transmission, and some light from a different second corresponding portion of display device 602, which experiences a single bounce before transmission. Similarly, the light from R5 comprises some light from the corresponding first portion of display device 602 that experiences a single bounce before transmission, and some light from a different second corresponding portion of display device 602 that experiences two bounces before transmission. For each of R3 and R5, the two different portions of LCOS propagate light corresponding to the virtual image portion.

[0160] The inventors have recognized that, at least in some applications, the virtual image distance (i.e., the distance from the observer to the virtual image) is preferably finite, contrary to the notion of forming a virtual image at infinity. In some applications, there will be a preferred virtual image distance at which the appearance of the virtual image content is desired or necessary. This could be, for example, in a head-up display, such as in a car setting, if the virtual image content is to be superimposed on real content viewed by the observer through the vehicle's windshield. For example, a desired virtual image distance could include virtual image content formed a few meters in front of the observer's vehicle or windshield, such as 3 meters or 5 meters.

[0161] Holographic computation for small display devices, long viewing distances, and pupil dilators

[0162] The inventor designed a computing Figure 7 A method for creating a hologram using the illustrated optical system. Importantly, the display device is relatively small, and the projection distance is relatively long. The hologram is projected directly onto the observation system, and the method can be implemented in real time. The relatively small size of the display device and the relatively long projection distance necessitate a pupil dilator. The method handles different paths via the pupil dilator. This method allows image content to appear at different and / or multiple distances from the observation system, optionally simultaneously, for example, using a single hologram. This method also allows image content to appear downstream and upstream of the display device, optionally simultaneously, for example, using a single hologram.

[0163] Figure 7 A spatial light modulator 701 for displaying an image hologram is shown. In this embodiment, the spatial light modulator 701 is a liquid crystal on silicon device arranged to modularize the phase of received light. The spatial light modulator 701 is illuminated by at least partially coherent light from a light source (not shown). The light source may be a laser diode. The spatial light modulator 701 outputs light spatially modulated according to the displayed hologram. Figure 7 A ray 702 of spatially modulated light is shown. A pupil dilator 703 receives the spatially modulated light. The pupil dilator 703 is tilted relative to the plane of the display device 701. The pupil dilator 703 therefore receives light that is not perpendicularly incident. The angle of incidence (the angle formed by the optical axis and the pupil dilator) can be less than 25 degrees, for example, 10 to 20 degrees. The pupil dilator includes an input surface 703a and an output surface 703b for receiving the spatially modulated light. The input surface 703a and the output surface 703b are substantially parallel and elongate in the direction of pupil dilation. The input surface 703a includes at least a portion that is substantially totally internally reflected (e.g., R = 1). The output surface 703b includes at least a portion that is highly reflective but partially transmissive (e.g., R = 0.9 and T = 0.1). The reflective surfaces are arranged such that the spatially modulated light is reflected back and forth between them, and the light is emitted at multiple points along the output surface 703b, as described above with reference to waveguide 608 in FIG. 6. In this embodiment, the pupil dilator is substantially elongated. A pupil dilator provides pupil dilation in one direction—that is, the elongation direction—but this disclosure can be extended to include the presence of a second pupil dilator arranged to dilate the pupil in an orthogonal direction.

[0164] Figure 7 The diagram illustrates how ray 702 is effectively replicated twice to form three propagation paths 705, each associated with a different distance Z0, Z1, and Z2. The shortest propagation path corresponds to Z0, and in this example, the light has passed through the waveguide without any internal reflections. The mid-range propagation path of the three shown corresponds to Z1 and two internal reflections in the pupil dilator (one on each surface). The longest propagation path shown corresponds to Z2 and four internal reflections in the pupil dilator (two on each surface). Planes x0, x1, and x2 respectively show the spatial extent of the light field associated with each of the three propagation paths Z0, Z1, and Z2. More specifically, Figure 7 This shows how the three planes x0, x1, and x2 are offset from each other in the x-direction.

[0165] Figure 7 An observation system 713 is also shown, which includes an entrance pupil 707, a lens 709, and a light sensor 711. In an embodiment, the observation system 713 is a human eye, and the light sensor 711 is the retina of the eye. Figure 7 It shows that only some of the light fields associated with each propagation path pass through inlet 707. Figure 7The diagram shows the light rays associated with the center of the mid-range propagation path passing through the center of the entrance pupil 707. However, for example, the light rays associated with the center of the light field of the shortest propagation path are blocked by the top of the aperture 707. However, other light rays associated with the light field of the shortest propagation path can pass through the aperture 707. The light rays associated with the center of the light field of the longest propagation path are blocked by the bottom of the aperture 707. However, other light rays associated with the light field of the longest propagation path can also pass through the aperture 707.

[0166] Light passing through aperture 707 is focused onto light sensor 711 by lens 709. The plane of light sensor 711 is substantially parallel to the plane of display device 701, and therefore also tilted relative to the elongated dimension of pupil dilator 703.

[0167] Figure 7 Three possible light propagation paths are shown by way of example only. This disclosure is not limited to the number of propagation paths. That is, those skilled in the art will understand from the following description that the method can be extended to consider any number of light propagation paths. Similarly, the pupil dilator is not necessarily tilted relative to the display plane and the sensor plane.

[0168] Figure 8 This is a flowchart illustrating the steps of the method. The method is similar to a Gerchberg-Saxton type algorithm, which uses mathematical transformations back and forth between the image plane and the hologram to converge on a phase hologram corresponding to the image. After each propagation to the image plane or the holographic plane, the amplitude components of the light field are modified or constrained, but the phase components are preserved.

[0169] The zeroth stage of the method includes steps 802 and 804. The zeroth stage includes forming a zeroth composite light field. Step 802 provides a random phase seed to form the phase component of the zeroth composite light field. Step 804 provides the amplitude component of the zeroth composite light field. The amplitude component can be a unit or amplitude distribution representing the light source light used to reconstruct the image from the hologram.

[0170] In step 806, the zeroth composite light field propagates Fresnelly from the spatial light modulator 701 (i.e., from the holographic plane) to the entrance pupil 707 of the observation system 713 (more specifically, to the plane containing the entrance pupil 707 of the observation system 713). Furthermore, this embodiment refers to Fresnel propagation as one example of a variety of different mathematical transformations that can be used without departing from the spirit or scope of this disclosure. Step 806 is performed for each number of bounces or internal reflections provided by the pupil dilator 703 to form a composite light field with respect to each light propagation path. Step 806 includes taking into account the lateral position of the composite light field in the x-direction at the plane of the entrance pupil 707, and the phase shift of each reflection within the pupil dilator 703. Different composite light fields can be combined, for example, by addition. The first stage also includes step 808, tailoring the combined composite light field according to the size and shape of the entrance pupil 707 to form a first composite light field at the entrance pupil 707.

[0171] The second phase of the method includes steps 810 and 812. In step 810, a second composite light field is determined by propagating a first composite light field from the entrance pupil through lens 709 to the plane of the light sensor 711. Step 812 includes modifying the amplitude component of the composite light field reaching the light sensor 711. More specifically, step 812 includes replacing the amplitude component of the composite light field with the amplitude component of the target image or an amplitude component based on the amplitude component of the target image (e.g., a weighted version of the amplitude component of the target image). The position of lens 709 used in the propagation determines the image distance, i.e., the space in which the image content will appear. In some embodiments, the image is a virtual image, and this distance may be referred to as the virtual image distance "VID".

[0172] Advantageously, the method disclosed herein allows the formation of image content at multiple different image distances (e.g., multiple VIDs) using the same hologram. The inventors recognize that this can be achieved by repeating the second stage for each image distance, taking into account the different positions of the lens 709 in the z-direction. For example, the composite light fields determined according to this method for each different image distance can be combined by addition.

[0173] The third stage of the method includes step 814, in which the second composite light field propagates back to the entrance pupil 707 via lens 709. This can be referred to as backpropagation, simply to reflect the travel of light in the opposite z-direction. In some embodiments, backpropagation is the mathematical inverse of the corresponding "forward" propagation. The third stage also includes tailoring the propagated light field according to the size and shape of the entrance pupil 707 to form a third composite light field.

[0174] The fourth stage comprises steps 816 and 818. In step 816, light propagates back to the plane of the spatial light modulator 702 via multiple light propagation paths of the pupil dilator, in the manner described above with respect to the first stage—but of course, in the opposite light direction (i.e., “reverse” propagation). Step 818 includes trimming the propagated light field according to the size and position of the effective / pixel area of ​​the display device. The number of complex values ​​of each composite light field can be equal to or less than the number of pixels of the display device.

[0175] Step 820 includes extracting a hologram from the fourth composite optical field. The hologram may include the phase value of the fourth composite optical field; in this case, the hologram may be referred to as a phase hologram. As explained earlier in this disclosure, the method can also begin from the image plane (i.e., the third stage). According to this disclosure, each stage requires at least one iteration. Figures 9 and 10 illustrate holograms formed using this method.

[0176] Optical Channel

[0177] Figure 9A An image 1552 for projection is shown, comprising eight image regions / components V1 to V8. Figure 9A Eight image components are shown as an example only, and image 1552 can be divided into any number of components. Figure 9A Also shown is an encoded light pattern 1554 (i.e., a hologram) that can reconstruct image 1552—for example, when transformed by a lens of a suitable observation system. The encoded light pattern 1554 includes first to eighth sub-holograms or components H1 to H8, corresponding to first to eighth image components / regions V1 to V8. Figure 9A This further illustrates how the hologram calculated according to this disclosure effectively decomposes image content through angles. Therefore, a hologram is characterized by its guidance of light. This is in Figure 9B As shown in the diagram. Specifically, light is guided to multiple discrete regions according to the hologram of this disclosure. In the example shown, the discrete regions are disks, but other shapes are also conceivable. After propagation through the waveguide, the optimal size and shape of the disk can be correlated with the size and shape of the entrance pupil of the observation system. The existence of this light channel is solely due to the specific method for determining the hologram disclosed herein.

[0178] Figure 10 It shows according to Figure 9A and 9B The improved observation system 1500 shown is an example of this.

[0179] The observation system 1500 includes a display device comprising an LCOS 1502 in this arrangement. The LCOS 1502 is arranged to display a modulation pattern (or "diffraction pattern") including a hologram and to project holographically encoded light onto an eye 1505, which includes a pupil serving as an aperture 1504, a lens 1509, and a retina (not shown) serving as an observation plane. A light source (not shown) is arranged to illuminate the LCOS 1502. The lens 1509 of the eye 1505 performs a hologram-to-image conversion.

[0180] The observation system 1500 also includes a waveguide 1508 located between the LCOS 1502 and the eye 1505. Figure 10 The projection distance may be relatively large. However, as described with respect to the preceding figures, the presence of waveguide 1508 allows all angular content from LCOS 1502 to be received by eye 1505, even at this relatively large projection distance. This is because waveguide 1508 acts as a pupil dilator in the manner already described above.

[0181] Furthermore, in this arrangement, when the LCOS1502 has been encoded according to the method described herein, the waveguide 1508 can be oriented at an angle relative to the LCOS1502 to establish a unique relationship between the light from the LCOS1502 and the virtual image perceived by the observer. The size, position, and orientation of the waveguide 1508 are configured to ensure that light from each portion of the virtual image enters the waveguide 1508 and is guided along its elongated axis, bouncing between the substantially flat surfaces of the waveguide 1508. Whenever light reaches the second planar surface (closest to the eye 1505), some light is transmitted and some is reflected.

[0182] Figure 10 A total of nine “bounce” points B0 to B8 are shown along the length of waveguide 1502. The reader will notice that the center of image 1552 remains blank. Figure 10 The diagram shows the 0th to 9th light "bounce" or reflection points B0 to B8 within the waveguide. Although light associated with all points in the image (V1-V8) is transmitted out of the waveguide at each "bounce" from the second planar surface of waveguide 1508, only light from a corner portion of the image (e.g., light from one of V1 to V8) has a trajectory that allows it to reach eye 1505 from each corresponding "bounce" point B0 to B8. Furthermore, light from different angular portions of the image V1 to V8 reaches eye 1505 from each corresponding "bounce" point. Figure 10The image shows light emitted from all the different angular contents at each “bounce” point (represented by multiple short arrows at each transmission point), but only the light path from each angular content to the eye 1505 is shown subsequently. This path will actually travel from the corresponding portion of the waveguide to the eye 1505—thus helping the observer perceive the corresponding portion of the virtual image. For example, for the zeroth bounce B0, the light transmitted by waveguide 1508 is simply refracted and undergoes no reflection within it. The light from the eighth sub-hologram H8 reaches the eye from the zeroth bounce B0. For the next bounce B1, the light transmitted by waveguide 1502 undergoes one bounce within it before transmission. The light from the seventh hologram H7 reaches the eye from the next bounce B1. This continues sequentially until the light transmitted by waveguide 1508 at the last bounce B8 has undergone eight reflections before being transmitted and reaching the eye 1505, and includes light encoded according to the first hologram H1.

[0183] exist Figure 10 In the example shown, light from only one image region reaches the eye from each bounce point. Therefore, when determining the hologram as described herein, a spatial correlation is established between the regions of the virtual image and their corresponding bounce points on the waveguide. In some other examples, there may be relatively small overlap, such that a region of the image originates from two adjacent transmission points and is thus contained within two adjacent optical discs propagating from the waveguide to the observation plane.

[0184] Therefore, the inventors recognize that the methods and apparatus described herein can generate diffraction patterns (or light modulation patterns) including holograms, which, when displayed on an LCOS or other suitable display device, enable light to be effectively emitted from them in the form of multiple “disks” or beams, each disk or beam corresponding to (more specifically, encoding) a different corresponding portion of a corresponding virtual image.

[0185] In summary, this disclosure relates to the calculation of a hologram based on the angular distribution of light (in a holographic domain) at positions within an image, and the propagation of said light through a pupil dilator providing multiple light propagation paths, each light propagation path corresponding to a corresponding consecutive region of the image. A further feature of this disclosure is the determination of a first image component of the image, and the allocation of more data processing resources relative to the first image component than to a second image component for calculating the hologram.

[0186] First set of examples: Sub-regions of an image

[0187] In short, Figure 11 and 12 It shows how to compute the foveal region of the observer's retina to obtain high image quality, while computing the rest of the image to obtain lower quality (but sufficient for peripheral vision).

[0188] To be more detailed, Figure 11 A display area 1101 for displaying an image is shown. For example, display area 1101 could be the display area of ​​a display system such as a head-up display. According to this disclosure, the image changes (e.g., over time). The image may change in real time, for example, at a video rate. Each image can be an image frame in a sequence of image frames. Each image can include image content. Each image can include multiple different image elements. This is merely an example. Figure 11 An image comprising three image elements is shown. The first image element represents the speedometer. The second image element represents the vehicle's headlight indicator. The third image element represents the warning indicator. The first, second, and third image elements are separated within the display area. That is, the first, second, and third image elements are separated by blank space. In other words, the first, second, and third image elements are disconnected. Figure 11 A first eye position 1105 and a second eye position 1107 are also shown. The first eye at the first eye position 1105 has a corresponding first eye gaze direction 1106, and the second eye at the second eye position 1107 has a corresponding second eye gaze direction 1108. The reader will understand that the first and second eyes are examples of an observation system set up on an observation area or observation plane. Of course, the observation system can be a human observer. The reader will also understand that the observer may have foveal vision and peripheral vision. Figure 11 The first sub-region 1103, corresponding to the display area of ​​the observer's foveal vision, is highlighted. The first sub-region 1103 corresponds to the first sub-region or first image component of the image. In this example, the first image component corresponds to half of the first image element (velocity sensor). Figure 11 It shows how the first eye gaze direction 1106 and the second eye gaze direction point to the first sub-region 1103.

[0189] It is worth noting that, Figure 11 This illustrates how the image quality associated with the observer's foveal visual region (i.e., the first sub-region 1103) is greater than the image quality of the observer's peripheral visual region (i.e., the remainder of display area 1101). In this example, half of the first image element within the first sub-region 1103 is sharper / less blurry than the other half of the first image element outside the first sub-region 1103. For example, the image contrast within the first sub-region may be greater than the image contrast outside the first sub-region. The image content corresponding to the observer's peripheral visual region is less sharp / less blurry than the image content corresponding to the observer's foveal visual region. Figure 11 This shows how the image quality of the second and third image elements is relatively low.

[0190] In embodiments, each image is formed or reconstructed from a corresponding hologram. Each hologram can be determined or calculated based on a source image or a target image. Naturally, the reconstructed image should be a faithful copy or reproduction of the source or target image. However, the reader will understand that holograms (diffractive elements with complex structures) inherently affect the quality or accuracy of the reconstructed image, and therefore the "quality" of holograms can be considered or compared. By way of example only, the quality of an image reconstructed from a hologram can be evaluated or even measured by the signal-to-noise ratio of the reconstructed image. However, the reader will understand that many other methods for measuring image quality are also possible. In some embodiments, each hologram is calculated using mathematical methods that require computational resources such as time or processing power. The image quality of the reconstructed image is determined at least in part by the amount of processing resources allocated to the computation of the hologram.

[0191] According to this disclosure, more processing resources are allocated to the computation of a hologram relating to the foveal visual region of the retina compared to the observer's peripheral visual region. Therefore, the method includes determining a first sub-region 1103 of the display region 1101 and selectively allocating processing resources based on this determination. In some embodiments, eye tracking is used to determine the first sub-region 1103 of the display region 1101, but the reader will be familiar with other methods for determining the observer's foveal visual region and thus non-foveal visual regions, and vice versa.

[0192] Figure 11 It can represent the image display based on the first foveal visual region of the retina at the first moment. Figure 12 It can be represented as an image display based on the second foveal visual region of the second retina at a second time, where the first time is different from the second time. Figure 11 The image shown in the display area 1101 can be reconstructed from a first hologram calculated based on the first foveal visual region 1103 of the first retina. Figure 12 The image shown in the display area 1201 can be reconstructed from a second hologram calculated based on the second retinal foveal visual region 1203.

[0193] Figure 12 Roughly corresponding to Figure 11 However, it shows a first gaze direction 1206 and a second gaze direction 1208 pointing to the second foveal visual region 1203 of the retina. Figure 12 The position of the second foveal visual region 1203 within the display area 1201 is different from that of the second foveal visual region 1203. Figure 11 The location of the first foveal visual region 1103 within the display area 1101. In this example, the second foveal visual region 1203 is substantially aligned with the third image element. The image quality of the third image element is higher than that of the second and first image elements. Figure 11 and 12 To the same target image. More specifically, Figure 11 and 12 The images are reconstructions of the same target image. The first hologram and the second hologram are calculated independently from the same target image. However, according to this disclosure, the allocation of processing resources for calculating the first hologram differs from the allocation of processing resources for calculating the second hologram. More processing resources are allocated to the foveal visual region of the retina to improve the image quality of the holographic reconstruction aligned with the foveal visual region. This is not as straightforward as it seems, because each part of the hologram contributes to every part of the image. However, the configuration of the hologram and pupil dilator described herein provides multiple distinct light propagation paths, each corresponding to a corresponding contiguous region of the image. The inventors recognize that their hologram calculation method effectively allows different processing resources to be allocated to different light propagation paths, such that the reconstruction quality can be a function of location within the image. This is not achievable with other holographic methods and offers significant advantages, especially when the observation system has a non-uniform response to light.

[0194] Figure 12 An optional further improvement is shown, in which a sub-region of the image allocated enhanced processing resources extends beyond the foveal visual region of the retina. Figure 12 This illustrates how the sub-region extends to include the entire image features, i.e., the third image element, partially contained within the foveal visual region of the retina. This effectively extends the area of ​​high image quality. In contrast to... Figure 11 In other consistent embodiments, the sub-region extends no more than the corresponding foveal visual region.

[0195] The hologram computation method disclosed herein efficiently computes multiple sub-holograms for each image and combines these sub-holograms to form a hologram for display. In some embodiments, allocating more data processing resources relative to a first image component than to a second image component includes allocating more data processing resources relative to the computation of a first sub-hologram corresponding to a first sub-region than to the computation of a second sub-hologram corresponding to a second sub-region.

[0196] In some embodiments where the hologram is a point cloud hologram, allocating more data processing resources relative to a first sub-region than a second sub-region includes using a higher density of point cloud data points relative to the first sub-region than the second sub-region.

[0197] In other embodiments that use iterative algorithms to compute holograms, allocating more data processing resources relative to the first sub-region includes performing more algorithm iterations relative to the second sub-region of the image.

[0198] In another embodiment, a first sub-hologram corresponding to the first sub-region is calculated using a point cloud method, and a second sub-hologram corresponding to the second sub-region is calculated using an iterative algorithm.

[0199] The overview above outlines optional additional features that provide further advantages. These features will not be repeated here, as they are self-evident and therefore do not require specific examples or diagrams for full understanding.

[0200] Reference Figure 11 and 12 In the described embodiments, an image may be said to include a first image component and a second image component, wherein each image component is a different sub-region or sub-area of ​​the image. That is, an image component is a spatial component of the image, such as a continuous and / or persistent block of image pixels, which together constitute the complete image. However, according to this disclosure, an image can be decomposed differently. That is, an "image component" can be a different aspect or constituent element of the image.

[0201] Second set of embodiments: Monochromatic components

[0202] In some embodiments, the image used for reconstruction is a color image comprising multiple monochrome image components (e.g., red, green, and blue components), which together form the color image, for example, by being superimposed on a display plane. For example, the color image may include multiple image pixels, where each image pixel includes red pixel values, green pixel values, and blue pixel values. This concept will be very familiar to the reader. In these embodiments, the first image component is a first monochrome image component of the image. In these embodiments, the second image component is a second monochrome image component of the image. A third monochrome image component of the image may exist, where the first image component corresponds to red, the second image component corresponds to green, and the third image component corresponds to blue. Each monochrome image component comprises multiple pixels.

[0203] In these embodiments, each monochrome image component is processed independently. Each monochrome image component can be considered an independent "image". A hologram for each monochrome image component is calculated. Thus, each image can generate at least a first hologram and a second hologram. The first hologram can be appropriately illuminated (e.g., with a laser diode providing red light) to form a first (red) image component. The second hologram can be appropriately illuminated (e.g., with a laser diode providing green light) to form a second (green) image component. Spatially modulated light of each color is propagated to the observation system through a pupil dilator. By way of example only, schemes such as frame-order color and spatially separated color, referred to herein, can be used to reconstruct the monochrome image components to form a panchromatic image.

[0204] The reader will understand that it may be beneficial to allocate more holographic computational resources to one monochrome component of the image than to another monochrome component.

[0205] For example, in some embodiments, the observation system or observer is more sensitive to light corresponding to a first monochromatic image component than to a second monochromatic image component. That is, the observation system is more sensitive (e.g., provides a larger output) to light having a wavelength corresponding to the first monochromatic image component compared to light having a wavelength corresponding to the second monochromatic image component. Prioritizing the computation of the hologram may be advantageous, as this will elicit a maximum response from the observation system. For example, when the observation system is a human observer, it is advantageous to spend more time or process more data points when computing a hologram that will produce a green image instead of a red or blue image. These embodiments achieve technological advancements in improving the quality of images perceived by the observation system.

[0206] In other embodiments, a first monochrome image component is determined to be visually more dominant than a second monochrome image component in the image. Due to this determination, it is advantageous to preferentially compute the hologram of the first monochrome image component compared to the second monochrome image component. These embodiments also achieve technological advancements in improving the image quality perceived by the observation system.

[0207] In another embodiment, background regions of the image are identified (e.g., through known image processing / analysis techniques), and parameters of these background regions are identified, such as color or a typical dominant color, like the most common or average color. In these embodiments, it is determined that one monochrome image component is more likely to be visible (e.g., against the background) than another monochrome image component. This determination may be based on the identification parameters of the image background. This determination may include a measurement of the contrast between background parameters (e.g., color) and corresponding parameters of the monochrome image components. According to this disclosure, more data processing resources are allocated to the computation of holograms corresponding to the monochrome image components that are considered or determined to be more visible against the background among a plurality of monochrome image components.

[0208] In other embodiments, the color balance of the reconstructed multicolor image varies in image regions corresponding to non-retinal foveal visual areas identified by, for example, an eye, head, or gaze-tracking observer. For example, this could include shifting the color balance toward 500 nm and / or reducing the intensity of any image light with wavelengths greater than 600 nm.

[0209] The improved methods and apparatus described herein can be implemented in a variety of different applications and observation systems. For example, they can be implemented in head-up displays (HUDs). Although virtual images, which require the eye to convert received modulated light to form a perceived image, have been generally described herein, the improved methods and apparatus described herein can be applied to real images.

[0210] Additional features

[0211] The embodiments relate to electrically activated LCOS spatial light modulators by way of example only. The teachings of this disclosure can be equivalently implemented on any spatial light modulator capable of displaying computer-generated holograms according to this disclosure, such as any electrically activated SLM, optically activated SLM, digital micromirror device, or microelectromechanical device.

[0212] In some embodiments, the light source is a laser, such as a laser diode. The holographic projection system disclosed herein can be used to provide an improved head-up display. In some embodiments, a vehicle is provided that includes a holographic projection system mounted in the vehicle to provide a HUD. The vehicle can be a motor vehicle, such as a car, truck, van, lorry, motorcycle, train, airplane, boat, or ship.

[0213] In the second set of disclosed embodiments, the holographic reconstruction is in color. In some embodiments, a method known as Spatial Separated Color (SSC) is used to provide color holographic reconstruction. In other embodiments, a method known as Frame Sequential Color (FSC) is used.

[0214] The SSC method uses three spatially separated arrays of light-modulated pixels for three monochrome holograms. An advantage of the SSC method is that the images can be very bright because all three holographic reconstructions can be formed simultaneously. However, if three spatially separated arrays of light-modulated pixels are provided on a common SLM due to spatial constraints, the quality of each monochrome image is suboptimal because only a subset of the available light-modulated pixels is used for each color. Therefore, relatively low-resolution color images are provided.

[0215] The FSC method can use all pixels of a common spatial light modulator to sequentially display three monochrome holograms. Cyclic monochrome reconstruction (e.g., red, green, blue, red, green, blue, etc.) is fast enough that a human observer can perceive a multicolor image from the synthesis of the three monochrome images. The advantage of FSC is that the entire SLM is used for each color. This means the quality of the resulting three color images is optimal because all pixels of the SLM are used for each color image. However, the disadvantage of the FSC method is that the brightness of the synthesized color image is approximately three times lower than that of the SSC method because each monochrome illumination event can only occur within one-third of the frame time. This disadvantage can be addressed by over-exciting the laser or by using a higher-power laser, but this requires greater power, leading to higher cost and increased system size.

[0216] The example describes illuminating an SLM with visible light, but those skilled in the art will understand that, for example, the light source and the SLM can also be used to guide infrared or ultraviolet light, as disclosed herein. For example, to provide information to a user, those skilled in the art will know the techniques used to convert infrared and ultraviolet light into visible light. For example, this disclosure extends to the use of phosphors and / or quantum dot techniques for this purpose.

[0217] Some embodiments describe 2D holographic reconstruction by way of example only. In other embodiments, the holographic reconstruction is 3D holographic reconstruction. That is, in some embodiments, each computer-generated hologram forms a 3D holographic reconstruction.

[0218] The methods and processes described herein can be embodied on a computer-readable medium. The term "computer-readable medium" includes media arranged for temporary or permanent storage of data, such as random access memory (RAM), read-only memory (ROM), buffer memory, flash memory, and cache memory. The term "computer-readable medium" should also be considered to include any medium or combination of media capable of storing instructions for machine execution, such that when the instructions are executed by one or more processors, the machine performs, wholly or partially, any or all of the methods described herein.

[0219] The term "computer-readable medium" also covers cloud-based storage systems. The term "computer-readable medium" includes, but is not limited to, one or more tangible and non-transitory data repositories (e.g., data volumes) in the example forms of solid-state storage chips, optical discs, disks, or any suitable combinations thereof. In some example embodiments, instructions for execution may be transmitted by a carrier medium. Examples of such carrier media include transient media (e.g., propagation signals for transmitting instructions).

[0220] It will be apparent to those skilled in the art that various modifications and variations can be made without departing from the scope of the appended claims. This disclosure covers all modifications and variations within the scope of the appended claims and their equivalents.

Claims

1. A method for reconstructing an image from a hologram, the method comprising: Receive an image for display within a display area of ​​a display system, wherein the display area is visible from an observation area spatially separated from it; Determine the first image component of the image; Calculate a hologram of an image, wherein the hologram is configured to distribute light at an angle according to a position within the image, such that the angular channels of the angled light correspond to corresponding continuous regions of the image; Display a hologram on a display device and spatially modulate light according to the displayed hologram; Spatially modulated light is propagated through a pupil dilator, which is arranged such that the spatially modulated light provides multiple different light propagation paths from the display device to the viewing area, wherein each light propagation path corresponds to a corresponding consecutive region of the image due to the angular distribution of the light from the hologram. The method includes allocating more data processing resources to compute the hologram relative to the first image component of the image compared to the second image component. Wherein, the first image component corresponds to the first sub-region of the image, and the second image component corresponds to the second sub-region of the image. The hologram computation includes computing multiple sub-holograms, each sub-hologram corresponding to a different one in a corresponding continuous region of the image, and allocating more data processing resources relative to the first image component than to the second image component includes allocating more data processing resources relative to the computation of the first sub-hologram corresponding to the first sub-region than to the computation of the second sub-hologram corresponding to the second sub-region.

2. The method of claim 1, wherein a first sub-hologram corresponding to a first sub-region is calculated using a point cloud method / algorithm, and a second sub-hologram corresponding to a second sub-region is calculated using an iterative method / algorithm.

3. The method as described in claim 1 or 2, wherein, A hologram is a point cloud hologram, and more data processing resources are allocated to the first sub-region than to the second sub-region, including the use of point cloud data points with a higher density relative to the first sub-region than to the second sub-region.

4. The method as described in claim 1 or 2, wherein, Computing a hologram involves executing iterative algorithms, and allocating more data processing resources relative to the first sub-region includes performing more algorithm iterations relative to the first sub-region of the image than to the second sub-region.

5. The method of claim 4, wherein, An image region is observed from the observation area using an observation system that includes an entrance pupil, a lens, and a light sensor, and the iterative algorithm includes: The first stage includes determining a first composite light field at the entrance pupil of the observation system, wherein the first composite light field is generated by the propagation of light from the display device along at least one light propagation path of the pupil dilator and by clipping according to the entrance pupil of the observation system. The second stage includes determining a second composite light field at the light sensor of the observation system, wherein the second composite light field is generated by the propagation of light from the first composite light field through the lens of the observation system from the entrance pupil and by modification of the amplitude component according to the image. The third stage includes determining the third composite light field at the entrance pupil, wherein the third composite light field is generated by the light of the second composite light field propagating backward from the sensor through the lens and according to the clipping of the entrance pupil; The fourth stage includes determining a fourth composite light field at the display plane, wherein the fourth composite light field is generated by the back propagation of light from the third composite light field along at least one light propagation path of the pupil dilator and by clipping according to the display device; and Holograms are extracted from the fourth composite light field.

6. The method of claim 5, wherein, The at least one light propagation path is only one of a plurality of light propagation paths provided by the pupil dilator, and the first to fourth stages are performed for each of the plurality of light propagation paths to extract a sub-hologram for each light propagation path, wherein the plurality of sub-holograms corresponding to the plurality of light propagation paths are combined to form a hologram for display on a display device, and / or wherein the first to fourth stages are iteratively repeated for each light propagation path prior to the step of extracting a sub-hologram from the final iteration.

7. The method of claim 1 or 2, further comprising: If the changes in size and / or position within the image for the first sub-region are determined, then repeat the method, and / or If changes in size and / or position within the image with respect to the first sub-region are determined, the hologram is recalculated relative to the first image component but not the second image component, and / or A first sub-region of the image is determined by determining a corresponding first sub-region of the display area, wherein the first sub-region of the display area corresponds to the foveal visual region of the retina of the observation system in the observation area, and wherein a second sub-region of the image is determined by determining a corresponding second sub-region of the display area, wherein the second sub-region of the display area corresponds to the peripheral visual region of the observation system.

8. The method of claim 7, wherein the first sub-region of the image extends beyond the corresponding foveal visual region, or wherein the first sub-region of the image extends beyond the corresponding foveal visual region to include the entire image features of the image partially contained within the foveal visual region.

9. The method of claim 7, further comprising eye, head, or gaze tracking of the observation system to determine a first sub-region of the display area.

10. The method of claim 9, further comprising determining an image region corresponding to the observer's blind spot based on eye, head, or gaze tracking, and processing the image prior to computing the hologram to remove image content corresponding to the observer's blind spot.

11. The method of claim 9, further comprising: If the rate of change in eye or head position or gaze direction is greater than the stored value, then determine the rate of change in eye or head position or gaze direction, and reduce the data processing resources allocated for calculating the hologram, and / or Predict future eye or head position or gaze direction based on stored and associated data, so that more data processing resources can be allocated to compute holograms.

12. The method of claim 11, wherein the display device has a variable display rate, and the method further includes updating the display device more quickly if the rate of change of eye or head position or gaze direction is greater than a stored value.

13. The method of claim 1 or 2, further comprising increasing the intensity of the first sub-region of the image relative to the second sub-region before calculating the hologram of the image.

14. The method of claim 1, wherein, The image is a multicolor image, the first image component is the first monochrome image component of the image, the second image component is the second monochrome image component of the image, wherein the calculation, display and propagation steps are performed independently for each monochrome image component of the image.

15. The method of claim 14, wherein: The light sensor of the observation system, arranged within the observation area to receive spatially modulated light, is more sensitive to light corresponding to the first monochromatic image component than to light corresponding to the second monochromatic image component, and / or The method further includes: Determine that the first monochrome image component is visually more dominant than the second monochrome image component in the image, or The parameters of the image background are determined, and based on the determined image parameters, it is determined that the first monochrome image component is more likely to be visible on the background than the second monochrome image component.

16. The method of claim 15, wherein the parameter of the image background is color.

17. The method of claim 14 or 15, further comprising eye, head, or gaze tracking of the observation system to determine the foveal visual region of the display area and altering the color balance of the reconstructed polychromatic image in an image region corresponding to the non-foveal visual region of the display area.

18. The method of claim 17, wherein altering the color balance of the reconstructed polychromatic image in a non-retinal foveal visual region comprises shifting the color balance toward 500 nm and / or reducing the intensity of any image light with a wavelength greater than 600 nm.

19. A light engine arranged to reconstruct an image from a hologram, wherein the light engine comprises: An image processor is arranged to receive an image for display within a display area and to determine a first image component of the image, wherein the display area is visible from an observation area spatially separated from it; A holographic engine arranged to compute a hologram of an image, wherein the hologram is configured to distribute light at an angle according to a position within the image, such that the angular channels of the angled light correspond to corresponding continuous regions of the image; Display devices, which are arranged to display holograms and spatially modulate light according to the displayed holograms; A pupil dilator is arranged to receive and propagate spatially modulated light from a display device, thereby providing multiple distinct light propagation paths for the spatially modulated light from the display device to the viewing area. Each light propagation path corresponds to a specific continuous region of the image due to the angular distribution of the light from the hologram. In this configuration, the optical engine is arranged to allocate more data processing resources to the first image component than the second image component for computing the hologram. Wherein, the first image component corresponds to the first sub-region of the image, and the second image component corresponds to the second sub-region of the image. The hologram computation includes computing multiple sub-holograms, each sub-hologram corresponding to a different one in a corresponding continuous region of the image, and allocating more data processing resources relative to the first image component than to the second image component includes allocating more data processing resources relative to the computation of the first sub-hologram corresponding to the first sub-region than to the computation of the second sub-hologram corresponding to the second sub-region.

Images