Holographic processing

Abstract

An optical engine is provided, comprising a display device. The display device comprises an array of pixels. Each pixel is for applying a phase delay to incident light. The optical engine is configured to receive a hologram of an image. The hologram contains an array of pixel values. The hologram comprises a first region containing diffracted components and a second region containing non-diffracted components. The first region is adjacent to the second region. The optical engine is further configured to apply a hologram blur function to the hologram in the boundary region between the first and second regions and to output a corrected hologram. The hologram blur function is configured to modify the gradient of the pixel values ​​in the boundary region.

Description

Technical Field

[0001] The present disclosure relates to an optical engine, a method, and a (processed) diffractive structure. More specifically, the present disclosure relates to an optical engine configured to process a hologram to prevent stray light and unwanted light (e.g., scattered light) from reaching an observation window of the optical engine. Some embodiments relate to a holographic projector, an image generation unit, or a head-up display.

Background Art

[0002] Light scattered from an object contains information on both amplitude and phase. This amplitude and phase information can be captured on a photosensitive plate by, for example, well-known interference techniques to form a holographic record, i.e., a "hologram", including interference fringes. The hologram is reconstructed by irradiating it with appropriate light to form a two-dimensional or three-dimensional holographic reproduction image, i.e., a reproduced image, representing the original object.

[0003] Computer-generated holography can numerically simulate the interference process. Computer-generated holograms can be calculated by methods based on mathematical transformations such as Fresnel transformation and Fourier transformation. These types of holograms are sometimes referred to as Fresnel / Fourier transform holograms or simply Fresnel / Fourier holograms. Fourier holograms can be regarded as the Fourier domain / plane representation of an object or the frequency domain / plane representation of an object. Computer-generated holograms can be calculated by, for example, coherent ray tracing or point cloud methods.

[0004] Computer-generated holograms may be encoded on a spatial light modulator arranged to modulate the amplitude and / or phase of incident light. The optical modulation can be realized using, for example, electrically addressable liquid crystals, optically addressable liquid crystals, or micromirrors.

[0005] A spatial light modulator typically consists of multiple individually addressable pixels, also called cells or elements. The optical modulation scheme can be binary, multilevel, or continuous. Alternatively, the device may be continuous (i.e., not composed of pixels), and therefore the optical modulation may be continuous throughout the device. A spatial light modulator may be reflective, meaning that the modulated light is reflected and output. Similarly, a spatial light modulator may be transmissive, meaning that the modulated light is transmitted and output.

[0006] A holographic projector can be provided using the system described herein. Such projectors are used in head-up displays (HUDs). [Overview of the Initiative]

[0007] The aspects of this disclosure are defined in the attached independent claims.

[0008] Generally, an optical engine is provided for receiving a hologram of an image, and the hologram comprises a first region and a second region. In embodiments, the hologram comprises an array of pixel values. Each pixel value may be a phase delay or represent a phase delay. The optical engine is configured to apply a hologram blur function to the boundary region between the first and second regions of the hologram. The hologram blur function is configured to change the gradient (i.e., rate of change) of the pixel values ​​of the hologram within (e.g., across the boundary region) the boundary region. In some embodiments, the hologram blur function is configured to reduce the magnitude / size of one or more phase differences between adjacent pixel values ​​of the hologram within the boundary region. In some embodiments, the hologram blur function is configured to reduce the size of the stepwise change in pixel values ​​at the boundary / boundary between the first and second regions of the hologram. In some embodiments, the hologram blur function reduces the maximum pixel value difference between adjacent pixels within or across the boundary. In some embodiments, the function increases the number of pixels associated with the boundary by extending or spreading the change in pixel values ​​(associated with the boundary) to more pixels. In the absence of a hologram blur function, relatively large / abrupt (stepwise) changes in pixel values ​​may occur at the boundary between the first and second regions. For example, in some embodiments, the first region may be a diffractive region and the second region may be a non-diffractive region. In embodiments, the diffractive region is the pixel region where the hologram content is calculated (and / or displayed), and the non-diffractive region is the pixel region where the hologram content is not calculated (and / or displayed). In embodiments, there is no overlap between the diffractive and non-diffractive regions, and together they constitute the entire display region. However, to avoid doubt, the boundary region according to this disclosure may overlap with one or both of the diffractive and non-diffractive regions. The pixel values ​​of the hologram in the diffractive region may change substantially (per pixel value) throughout the entire diffractive region. This is so that light incident on the diffractive region is diffracted as a result of the difference between adjacent pixel values.The pixel values ​​of a hologram in the non-diffractive region are substantially constant, and each pixel value within the non-diffractive region is substantially the same. This prevents light incident on the non-diffractive region from being diffracted. Therefore, at the boundary between the first (diffractive) region and the second (non-diffractive) region, the pixel values ​​of adjacent or nearby pixels crossing the boundary between the first and second regions may change stepwise (if there is no hologram blurring). For example, in the second region, each pixel value may be 0. In the first region, each pixel value may range from 0 to 2π. As will be apparent to those skilled in the art, if the pixel value of a pixel at the boundary between the first and second regions in the first region is 2π, the (step) change in the difference between that pixel and its adjacent pixels in the second region is 2π. Applying a hologram blurring function, as described in more detail herein, reduces the size / magnitude of these changes by blurring or smoothing the edges of the hologram at the boundary (i.e., by blurring or smoothing the boundary between the diffractive and non-diffractive regions of the hologram). The inventors have found that by blurring the edges of a hologram using a hologram blur function, the so-called "halo effect" (noise) that appears in the holographic reproduction of a hologram can be significantly reduced. This halo effect is described in more detail herein.

[0009] The use of the hologram blur function in this disclosure is counterintuitive. The hologram may be an image hologram, and when displayed on a display device, the light incident on the hologram may be spatially modulated according to the hologram and arranged to form a hologram wavefront. The hologram wavefront can then form a hologram reconstruction of the image. The quality of the hologram reconstruction of the image depends on the amount and quality of (phase) information contained in the hologram. In other words, the quality of the hologram reconstruction may depend on the number of pixel values ​​of the hologram used to form the reconstruction, and the precision of the pixel values ​​themselves. Therefore, in holographic projection, it is common to ensure a high-quality hologram reconstruction by maximizing the number of pixel values ​​of the hologram used to form the hologram reconstruction and ensuring that those pixel values ​​are as accurate as possible. Furthermore, the process of forming a hologram reconstruction is fundamentally based on diffraction and depends on well-defined diffraction structures such as changes in phase delay. However, the hologram blur function in this disclosure changes the pixel values ​​of the hologram (in the boundary region), thus reducing the number and precision of pixel values ​​that can contribute to the hologram reconstruction. This is equivalent to blurring the diffraction structure. Therefore, there has been a significant prejudice in the field to avoid the use of the hologram blur function described herein. However, the inventors have found that the benefits of applying the hologram blur function to reduce the halo effect far outweigh the loss / reduction in the quality / quantity of phase information available in the hologram reconstructed image. Furthermore, the inventors have found that the amount of pixel values ​​that need to be blurred to significantly reduce the halo does not have a significant adverse effect on the appearance (to the naked eye) of the hologram reconstructed image.

[0010] As will be explained in more detail below, the inventors have found that the so-called halo effect can be particularly pronounced in special configurations where the hologram is positioned relatively close to the display device to form a holographic image (for example, when the distance between the display device and the holographic image is less than 50 millimeters, and arbitrarily less than 20 millimeters). Therefore, reducing the amount of change in pixel (phase) phase using a hologram blur function, thereby significantly reducing the halo effect, may be particularly advantageous in such special configurations.

[0011] In some embodiments, the optical engine includes an observation window. The optical engine is configured to display a hologram of an image on a display device and to spatially modulate light according to the hologram. The optical engine is further configured so that the spatially modulated light forms a holographic reconstructed image on the reconstructed surface. The image includes one or more regions containing (image) content (diffractive regions) and one or more regions not containing (image) content (non-diffractive regions). In such cases, the holographic reconstructed image of the image ideally includes the corresponding diffractive and non-diffractive regions. The diffractive regions of the hologram may form the diffractive regions of the holographic reconstructed image.

[0012] In some embodiments, the optical engine may comprise one or more waveguide pupil dilators. Spatially modulated light may be relayed to and coupled to a first waveguide. The first waveguide may replicate its light to dilate the exit pupil of the optical system in a first direction. It may also comprise a second waveguide that receives the light output from the first waveguide and is arranged to dilate the exit pupil of the optical system in a second direction. Conceptually, this result can be thought of as the creation of an array of holograms or display device replicas displayed on a display device. The array of replicas can be said to reside on a "virtual surface." This virtual surface may be arranged in an alternating pattern, as described in UK Patent Application GB2118911.3 filed December 23, 2021. In particular, each replica may have a different vertical distance from the display device because the optical path lengths in the waveguides corresponding to each replica are different. Thus, a portion of the virtual surface associated with each replica (e.g., x, y dimensions) is offset perpendicularly from the display device (e.g., z dimension). The virtual image of the hologram reconstructed image is visible when the observation system (e.g., the user's eyes) is positioned in an observation window downstream of the waveguide. The virtual image may be formed at a virtual image distance upstream of the display device. Typically, the virtual image distance can be between approximately 1 meter and approximately 10 or 20 meters. One advantage of the above optical system (including one or more waveguides) is that it extends the observation window (i.e., the user's eye box) to one or two dimensions, thus allowing the eye (or other observation system) to move to some extent, while simultaneously allowing the observation system to view the hologram reconstructed image even with a very small display device / hologram. As described in UK Patent Application GB2302916.8 filed February 28, 2023, such an arrangement allows for a high-quality virtual image of the hologram reconstructed image itself (when focused on the virtual image distance). However, in order to observe the virtual image of the hologram reconstructed image, the viewer needs to see through or pass through the image of the hologram / display device, which can negatively impact the viewing experience.

[0013] In GB2302916.8 (filed February 28, 2023), the applicant disclosed an optical system to address the above problem. This optical system comprises a display device configured to spatially modulate light in response to a displayed hologram to form a holographic wavefront. This holographic wavefront forms a holographic reconstructed image of the image downstream of the display device. The optical system further comprises optical components (e.g., lenses) configured to form a) a virtual image of the holographic reconstructed image and b) an image of the displayed hologram / display device. These optical components are arranged such that the magnitude of separation between the virtual image of the holographic reconstructed image and the image of the displayed hologram / display device is greater than the virtual image distance of the virtual image of the holographic reconstructed image. Thus, the image of the displayed hologram / display device is far removed from the virtual image of the holographic reconstructed image. In some examples, the virtual image of the display device is formed at infinity, and the effective distance between the virtual image of the display device and the virtual image of the holographic reconstructed image (which is at a finite image distance) is infinite. In other examples, the real image of the display device is formed downstream of the optical components, and this is also formed far away from the virtual image of the hologram reconstruction (which is upstream of the optical components). This arrangement of optical components has the advantage of significantly reducing the impact of virtual planes on the observer in the observation window, as they can be distracting or irritating. In particular, the optical components can be positioned to form the image of the hologram / display device at infinity (far beyond the virtual image of the hologram reconstruction) or downstream of the optical components (usually behind the observation system). In either example, when the optical components are positioned in this way, the observer does not need to see through the image of the hologram / display device or pass through it to observe the virtual image of the hologram reconstruction.

[0014] In the optical system described above, the holographic reconstructed image (or relay-type holographic reconstructed image) needs to be formed within the focal length of the lens of the optical system positioned between the holographic reconstructed image and the waveguide. Meanwhile, the display device (or relay-type display device) needs to be positioned at or (slightly) beyond the focal length of the lens, so that a virtual image is formed at infinity or a real image is formed downstream of the waveguide, respectively. Conventionally, the distance between the display device and the holographic reconstructed image formed by the display device can be relatively large (e.g., several meters). Therefore, in order to achieve the above arrangement and ensure that the virtual image of the holographic reconstructed image is formed at the desired virtual image distance (e.g., about 10 meters), the optical system may need to have a lens with a relatively very long optical axis and / or a relatively very long focal length. This contradicts the general requirement for compactness. For example, this optical system may be used for vehicle head-up displays, where package size requirements are generally stringent. Furthermore, arrangements with a very long optical axis and / or lenses with a relatively very long focal length can result in a relatively high percentage of light being lost from the system. These problems can be solved by providing an optical system positioned to display a hologram that forms a holographic reconstructed image relatively close to the display device (downstream of the display device). This enables a compact optical system / head-up display including the optical system. As described in UK Patent Application GB2304312.8 filed on 24 March 2023, the applicant previously proposed using a relatively small group of pixels of the display device to form each image point of the holographic reconstructed image (in a point-cloud hologram) as one method of forming a holographic reconstructed image relatively close to the display device.For example, a holographic image very close to that of a display device can be formed using a sequence of pixels including less than 100,000 pixels, optionally less than 25,000 pixels, optionally less than 5,000 pixels, optionally less than 1,000 pixels, optionally less than 500 pixels, optionally less than 200 pixels, and optionally less than 100 pixels.

[0015] Through thorough simulations and experiments, the inventors discovered that by using the above optical system, it is possible to miniaturize the optical system (by forming a holographic reconstructed image relatively close to the display device using a relatively small number of pixels) while maintaining good holographic reconstructed image quality, and that viewers do not need to see through or transmit the image of the hologram / display device to observe the virtual image of the holographic reconstructed image. However, the inventors also discovered that when the holographic reconstructed image is formed relatively close to the display device, unexpected artifacts / noise may be visible to the viewer compared to conventional optical systems (for example, when the holographic reconstructed image is formed at least 1 meter away from the display device, and when most / substantially all pixels of the display device contribute to each image point of the holographic reconstructed image). These artifacts / noise appear as unexpected bright areas adjacent to the area containing the content of the holographic reconstructed image. These artifacts have a shape corresponding to the shape of the adjacent diffraction regions of the holographic reconstructed image. The inventors discovered that these artifacts have particularly high intensity along the boundary of the artifact furthest from each / adjacent diffraction region. As a result, artifacts / noise may appear as a “halo” around or adjacent to the diffracted region of the holographic projection. Therefore, this noise will be referred to as the “halo” throughout this disclosure and contributes to the “halo effect.” We have found that this halo negatively impacts the viewer’s experience when observing the virtual image of the holographic projection. This is because the holographic projection contains unintended (halo) artifacts (these artifacts are not present in the holographic encoded image and are not apparent, at least to the naked eye, in other more conventional holographic projection configurations).

[0016] The inventors discovered that the halo effect is caused by the scattering of light when a display device that displays a hologram is illuminated. In some examples, the display device is a pixelated display device such as a spatial light modulator (liquid crystal on silicon). The inventors discovered that light can be scattered from at least some of the pixels of the display device, potentially forming the halo effect. For example, in a point cloud hologram, the hologram is calculated so that a group of pixels on the display device forms each image point (the image point is within the diffraction region of the hologram-reconstructed image) of the hologram. Pixels on the display device may contribute to multiple image points. Ideally, the light illuminating the display device is diffracted so as to contribute only to the intended image points of the hologram-reconstructed image. However, in reality, some light may be scattered. This scattering can be caused, for example, by the birefringence of the display device, imperfections in the hologram calculation, or imperfections in the display device's ability to perfectly and accurately display the hologram / spatially modulate light according to the hologram. Therefore, some of the light illuminating the pixels of the display device may be scattered and not contribute to the intended image point of the holographic image, and instead may form artifacts (things that do not exist in the image encoded in the displayed hologram) (e.g., halos) in the holographic image of the image.

[0017] The inventors discovered that the proportion of scattered light can be maximized when the phase delay applied by adjacent pixels of a display device changes abruptly. When a hologram is calculated such that a relatively small number of display device pixels contribute to each image point of the holographic image, the regions of the display device encoded in the hologram can contribute to the diffracted regions of the holographic image. These regions may be referred to herein as diffracted regions or diffracted components. There may also be “empty” regions in the display device (i.e., regions that do not contain diffracted components and have a constant phase value, such as 0 or 2π). These “empty” regions may be referred to herein as non-diffracted regions or non-diffracted components. At the boundary between diffracted and non-diffracted components, the phase delay can change very abruptly. Therefore, if there are non-diffracted component regions on the hologram or display device (pixels near the boundary between the diffracted and non-diffracted regions of the display device or hologram account for the majority of the scattered light), scattering can be particularly pronounced. Furthermore, scattering from pixels near the boundary between the regions encoded in the hologram and the empty regions of the display device scatters most of the light incident on those pixels. This is because these pixels contribute to the image pixels only at a small angle, rather than the complete diffraction of the display device. This also means that pixels near the boundary scatter most of the light incident on them. Both of these effects mean that the halo effect is particularly pronounced or obvious in unconventional or unusual optical systems where a very small number of pixels contribute to each image point. In conventional systems where a very large number of pixels contribute to each image point, the halo effect may be far less noticeable (sometimes invisible to the naked eye) because the ratio of scattered light to the light used to form the image point is much lower. Furthermore, such conventional systems generally do not contain empty or non-diffracted regions on the display device / hologram. Additionally, through simulations and experiments, the inventors discovered that the size of the "halo" depends on the diffraction angle and the distance between the display device and the holographic image. The greater the distance, the larger the halo and the wider the scattered light diffuses.Therefore, in conventional holographic optics, the halo is diffused over a relatively wide area, its intensity is substantially negligible, and it is not noticeable. Furthermore, the halo associated with the diffraction region of the holographic projection image is likely to overlap with and be obscured by another diffraction region of the holographic projection image. However, when the holographic projection image is formed relatively close to the display device, the halo may be formed relatively close to the periphery of the diffraction region of the holographic projection image and may be relatively strong. Furthermore, the halo is much more likely to overlap with other diffraction regions. In summary, we have identified a halo effect that negatively impacts the viewing experience when the optical system is positioned to form a holographic projection image relatively close to the display device. This halo effect is usually not a problem (or even invisible) in conventional holographic projection optics, and has been largely unproblematic until now.

[0018] The inventors identified the problem of the halo effect and recognized the need to eliminate or reduce it. One option is a hardware solution that blocks or deflects scattered light (which forms the halo effect) from reaching the observation window of the optical system / optical engine. However, hardware solutions increase complexity and cost. Therefore, the inventors developed a software solution to this problem (which does not increase the complexity or cost of the overall optical system).

[0019] As described above, the inventors discovered that the halo effect occurs primarily when there is a sudden, relatively large stepwise change in the phase delay applied by adjacent pixels in a display device (for example, when a hologram may exist at the boundary between diffractive and non-diffractive regions when displayed in a display device). Therefore, the inventors recognized that the halo effect can be (significantly) reduced by reducing the magnitude of these changes in phase delay. In particular, the inventors recognized that by applying the above-described hologram blur function to the boundary between the diffractive and non-diffractive regions of a hologram, the pixel values ​​at that boundary can be blurred or smoothed (at the cost of losing some (accurate) phase information from the hologram). Through thorough simulations and experiments, the inventors found that this blurring or smoothing can be applied without significantly degrading the apparent quality of the hologram-reconstructed image formed using the hologram (despite the loss of phase information).

[0020] In a first embodiment, an optical engine is provided. The optical engine comprises a display device having an array of pixels. Each pixel of the display device provides a phase delay to incident light. For example, each pixel of the display device may be controllable (individually) to provide a desired phase delay. Each pixel of the display device is controllable to provide one of a plurality of discrete phase delays based on the phase value of a hologram pixel, thereby enabling the representation or display of a hologram.

[0021] The optical engine is configured to receive a hologram of an image. The hologram contains an array of pixel values. Each pixel value may be a phase delay or represent a phase delay. For example, each pixel value of the hologram may have a phase delay of a pixel in a display device or correspond to a phase delay. The display device may be configured to display the hologram. In this case, the array of pixels in the display device can be controlled to apply a phase delay quantified by the respective phase delay values ​​of the corresponding hologram pixels. The hologram contains a first region and a second region. The first region contains or consists of a diffraction component. The second region contains or consists of a non-diffraction component. The first region is adjacent to the second region.

[0022] The optical engine is further configured to apply a hologram blur function to the hologram. The optical engine is configured to apply the hologram blur function to the boundary region between the first and second regions. The boundary region may include the boundary between the first and second regions. The boundary region may overlap with at least one, or optionally both, a portion of the first region and a portion of the second region. The optical engine may be configured to output a corrected hologram (the corrected hologram is the hologram after the hologram blur function has been applied).

[0023] The hologram blur function is configured to modify the gradient of pixel values ​​within the boundary region of the modified hologram (for example, the entire boundary region) (compared to the unmodified hologram initially received by the optical engine).

[0024] The first region may consist of a set of consecutive pixel values. When a hologram is displayed on a display device, a set of consecutive pixels on the display device may represent the first region. The first region (which includes or consists of diffraction components) may be called the (first) diffraction component region or the (first) diffraction region. In this specification, the first region including "diffraction components" means that the pixel (phase) values ​​within the first region are spatially varied. In other words, adjacent pixel values ​​or sets of adjacent pixel values ​​within the first region may be different. Therefore, when a hologram is displayed on a display device, light incident on a region of the display device encoded by diffraction components (e.g., the first region) may diffract the incident light. For example, the incident light may be spatially modulated according to the diffraction components, forming a holographic wavefront that forms the holographic reconstructed image of the image.

[0025] The second region may consist of a set of consecutive pixel values. When a hologram is displayed on a display device, a set of consecutive pixels on the display device may represent the second region. The second region may be referred to as the (first) non-diffraction component region or the (first) non-diffraction region. In this specification, the second region containing the "non-diffraction component" means that the pixel (phase) values ​​within the second region do not change spatially (within the second region). In other words, each pixel value within the second region of the hologram may be substantially the same or constant. In one example, each pixel value may be zero. In another example, each pixel value may be 2π. Therefore, when a hologram is displayed on a display device, light incident on a region of the display device encoded with the non-diffraction component (e.g., the second region) may not diffract the incident light.

[0026] In this specification, “hologram blur function” is a function suitable for application to a hologram to alter the gradient of pixel values. For example, a hologram blur function may be applied to one or more pixels of a hologram. A hologram blur function applied to a single pixel of a hologram may alter that pixel based on multiple or a group of pixels. The alteration of the gradient of pixel values ​​applied by the hologram blur function may smooth or blur the pixel values. The hologram blur function may alter the gradient of pixel values ​​substantially continuously and gradually (e.g., gradually increasing or gradually decreasing) in the boundary region. This can reduce the magnitude of individual changes in the values ​​(phase) of adjacent pixels (in the boundary region). Therefore, the application of a hologram blur function can significantly reduce the halo effect caused by the pixel to which the hologram blur function is applied. In other words, the changes in pixel values ​​associated with the boundary of the modified hologram are greater than the changes in the corresponding boundary of the original hologram.

[0027] In some embodiments, the hologram includes multiple pairs of adjacent pixel values. Each of these multiple pairs of adjacent pixel values ​​may be located at or along the boundary between a first region and a second region. Each pair of pixel values ​​may include a first pixel value and a second pixel value. The first pixel value may be a value in the first region. The second pixel value may be a value in the second region. The optical engine may be configured to apply a hologram blur function to the hologram for at least one pixel value of each pair of pixels. In some embodiments, for each pair of pixels, the hologram blur function may be configured to reduce the difference between the first and second pixel values ​​of each pair of pixels. This reduction may be made such that the difference in pixel values ​​in the modified hologram is reduced compared to the difference between the first and second pixel values ​​in the absence of the hologram blur function (i.e., the hologram as initially received).

[0028] In some embodiments, substantially all of the boundary region may overlap with the first region or the second region. In other words, any portion of the boundary region may overlap with at least one of the first region or the second region. As described above, the boundary region is the region of the hologram to which the hologram blur function is applied. That is, the boundary of the hologram is the region where the pixel value (gradient) changes. Therefore, the pixel values of the portion of the first region that overlaps with the boundary region may be changed by the hologram blur function. Similarly, the pixel values of the portion of the second region that overlaps with the boundary region may also be changed by the hologram blur function. The inventors have found that it may be advantageous to change the values of both the first region and the second region in the hologram blur function (since it may reduce the number of phase values changed in either region and thus reduce the amount of information lost in either region). In some embodiments, at least 25% of the boundary region may overlap with the first region. In some embodiments, at least 25% of the boundary region may overlap with the second region. In some embodiments, the boundary region may be substantially centered on the boundary between the first region and the second region (in the received hologram). Therefore, substantially 50% of the boundary region may overlap with the first region and substantially 50% of the boundary region may overlap with the second region.

[0029] In some embodiments, the boundary region includes a first end. The first end can be disposed, contained, or located within the first region. The boundary region can include a second end. Therefore, at least a portion of the boundary region can overlap with the first region (e.g., a portion of the boundary region that extends between the first end and the boundary between the first region and the second region). The second end can be located on the opposite side of the first end. The second end can be disposed, contained, or located within the second region. Therefore, at least a portion of the boundary region can overlap with the second region (e.g., a portion of the boundary region that extends between the second end and the boundary between the first region and the second region).

[0030] In some embodiments, the hologram blur function is arranged such that pixel values in the boundary region in the modified hologram gradually change according to the distance from the second end (after application of the hologram blur function). For example, the hologram blur function may be arranged such that pixel values in the boundary region in the modified hologram gradually change according to the distance from the second end. It gradually increases or decreases according to the distance from the second end.

[0031] In some embodiments, the hologram blur function is arranged such that the change in pixel values in the boundary region within the modified hologram has a substantially Gaussian shape from the second side to the first side (or vice versa).

[0032] The inventors have discovered that the greater the number of pixel values to which the hologram blur function is applied, the greater the reduction effect of the halo effect. However, increasing the number of pixel values to which the hologram blur function is applied increases the number of pixel values to be changed and also increases the amount of phase information lost in the modified hologram (compared to the unmodified / untreated hologram). The inventors recognized that the size of the boundary region needs to be selected to balance these two effects (so that the halo effect is sufficiently reduced without significant degradation of the image quality in the hologram reproduction image due to loss of phase information). In some embodiments, the thickness of the boundary region is defined as the distance between the first end and the second end. In some embodiments, the thickness of the boundary region is 20% or less, optionally 10% or less, optionally 5% or less of the width of the display area of the display device. In some embodiments, the thickness of the boundary region is 1% or more, optionally 5% or more of the width of the display area of the display device. In some embodiments, the thickness of the boundary region is 5 pixels or more, optionally 10 pixels or more, optionally 50 pixels or more. In some embodiments, the thickness of the boundary region is 1000 pixels or less, optionally 500 pixels or less, optionally 200 pixels or less. The inventors have found that such a thickness provides a good balance of the halo reduction effect while maintaining the image quality.

[0033] In some embodiments, the optical engine configured to apply a hologram blur function includes an optical engine configured to detect the boundary between a first region and a second region. In some embodiments, the optical engine is configured to detect the boundary using algorithmic means, as will be apparent to those skilled in the art. In some embodiments, the optical engine is configured to detect the boundary based on the pixel values ​​of the hologram.

[0034] In some embodiments, the optical engine may be configured to apply a binary or binary filter to the hologram. The optical engine may be configured to apply a binary or binary filter to the hologram as part of a boundary detection process. The optical engine may be configured to detect boundaries based on the pixel values ​​after the binary or binary filter has been applied to the hologram (i.e., based on the pixel values ​​of the binary hologram). In some embodiments, the optical engine may be configured to output a binary hologram in which each pixel value of the (original) hologram is replaced with a first or second value based on the respective (original) pixel value. Thus, the binary hologram may consist of pixel values ​​having only the first or second value.

[0035] In some embodiments, the optical engine may be configured to apply a binary filter to each pixel value of the hologram. The binary filter is configured to take the pixel values ​​of the hologram as input and compare those pixel values ​​to a predetermined value. The predetermined value may be the (expected) value of each pixel value in the non-diffraction component region of the hologram. For example, if each pixel value in the non-diffraction component region of the hologram is zero, the predetermined value of the binary filter may also be zero. The binary filter may be configured to output a first value if each pixel value is not equal to the predetermined value, and a second value if each pixel value is equal to the predetermined value. In some embodiments, the first value is 1 and the second value is zero. However, this is merely an example. It will be apparent to those skilled in the art that the first and second values ​​can take any value, as long as they are different from each other.

[0036] In some embodiments, the optical engine is configured to detect boundaries based on the point in time when the change or derivative of adjacent pixel values ​​in a binary hologram exceeds a (predetermined) threshold. Generally, in a binary hologram, each pixel value in the first region corresponding to the first region of the original hologram (including the diffraction component) has either a first or second value, and each pixel value in the second region corresponding to the second region of the original hologram (including the non-diffraction component) has either a first or second value. Therefore, the change or derivative of adjacent pixel values ​​in a binary hologram can be substantially zero in either the first or second region. Only at the boundary between the first and second regions (the region where adjacent pixel values ​​change from a first value to a second value, or vice versa) is the change or derivative of adjacent pixel values ​​greater than zero. Therefore, the change or derivative of adjacent pixel values ​​in a binary hologram can be non-zero only at the boundary. Thus, the optical engine has the advantage of being able to detect and determine the location of the boundary based on whether the change or derivative of adjacent pixel values ​​exceeds a threshold (the threshold is zero).

[0037] In particular, after receiving a hologram (including a first and second region), the optical engine may be configured to detect boundaries within the received hologram. The optical engine may be configured to detect boundaries in order to determine the region to which the hologram blur function should be applied. The optical engine being configured to detect boundaries (rather than relying on, for example, a predetermined boundary position) can be advantageous because it means that the optical engine can apply the hologram blur function to any shape or form of the first / second region of the hologram. For example, the optical engine may be configured to display a sequence of different holograms on a display device. Each hologram in the sequence may contain different first and second regions. For example, the positions and shapes of the first and second regions may differ from hologram to hologram. Therefore, the boundary between the first and second regions may vary from hologram to hologram in the sequence. The fact that the optical engine is configured to detect boundaries means that the optical engine can actively determine the boundary (and associated boundary region) of each hologram without needing to receive boundary position information.

[0038] In some embodiments, the hologram blur function is configured to output the average of the adjacent pixel values ​​for each pixel value to which the hologram blur function is applied. For example, the hologram blur function may be applied to the first pixel value in a boundary region. Thus, the hologram blur function may be configured to take the first pixel value as input. The hologram blur function may be configured to refer to or determine the adjacent pixel values ​​of the first pixel value, for example, a two-dimensional array of pixel values ​​adjacent to the first pixel. The hologram blur function may be configured to determine the average of the pixel values ​​(including both the first pixel value and the adjacent pixel values). The hologram blur function may further be configured to output the average pixel value. The output of the hologram blur function can replace the first pixel value of the modified hologram. This process may be repeated for each pixel in the boundary region.

[0039] In some embodiments, the hologram blur function is a spatial filter, such as a box blur filter. In some embodiments, the hologram blur function includes a kernel, such as a two-dimensional kernel. In some embodiments, the width and / or depth of the kernel is substantially equal to the thickness (in pixels) of the boundary region. The width and / or depth of the kernel may refer to the width or depth of the kernel along the first or second dimension of the kernel. The thickness may refer to the number of elements or pixels of the kernel extending into the first or second dimension. The statement that the width or depth of the kernel is substantially equal to the thickness of the boundary region means that the number of elements or pixels in each dimension of the kernel is equal to the thickness (in pixels) of the boundary region.

[0040] In some embodiments, an optical engine configured to apply a hologram blur function may apply that function to each pixel value at the boundary. In some embodiments, the boundary region of the modified hologram includes average pixel values ​​calculated using the hologram blur function.

[0041] In some embodiments, the hologram is positioned such that the distance between the display device and the holographic reconstructed image (the one closest to the display device) is relatively short. This distance may be defined as being free from physical optical components between the display device and the holographic reconstructed image. For example, the distance may be 50 millimeters or less, optionally 20 millimeters or less, or optionally 10 millimeters or less. Optionally, the distance may be between 1 millimeter and 20 millimeters. As described above, the inventors have found that the halo effect can be particularly pronounced when the distance between the display device and the holographic reconstructed image is this relatively short (and positioning the hologram to form a reconstructed image at such a short distance is highly unconventional).

[0042] In some embodiments, the optical engine may be configured to form a relayed hologram reconstructed image. This relayed hologram reconstructed image may be formed further downstream of the display device. The relayed hologram reconstructed image may substantially be an image of a hologram reconstructed image formed relatively close to the display device. For example, some embodiments of the optical engine further include an optical relay. The optical relay may be located downstream of the display device. The optical relay may include two lenses arranged to cooperate in forming the relayed hologram reconstructed image. The optical relay may further be configured to form a relayed display device (which may be an image of the display device). In such cases, the distance between the relayed display device and the (relayed) first hologram reconstructed image may be 50 millimeters or less, optionally 20 millimeters or less, or optionally 10 millimeters or less. Optionally, the distance may be between 1 millimeter and 20 millimeters.

[0043] In some embodiments, the holographic reconstructed image includes multiple image points. In some embodiments, the hologram is arranged such that each image point of the holographic image is formed using a contiguous group of pixels on the display device. Each contiguous group of pixels may constitute less than 10% of the total number of pixels on the display device. Each contiguous group of pixels may constitute less than 5% of the total number of pixels on the display device. Each contiguous group of pixels may consist of less than 100,000 pixels, optionally less than 25,000 pixels, optionally less than 1,000 pixels, or optionally less than 100 pixels. This allows the holographic image to be formed at a relatively short distance (short distance as defined above) from the display device / relay display device.

[0044] As described above, the hologram displayed on the display device is an image hologram. In some embodiments, the image includes image content in a first diffraction region. Therefore, the reconstructed hologram may include image content corresponding to the first diffraction region. The first region of the hologram (including the diffraction component) can be arranged to form image content in the first diffraction region.

[0045] In some embodiments, the image / hologram reconstruction consists of multiple diffraction regions. Each diffraction region may contain image content.

[0046] In some embodiments, a hologram may include multiple distinct first regions and multiple distinct second regions. Multiple boundaries may exist between the multiple first and second regions. One or more boundary regions may exist. Each of the multiple boundaries may be contained within one or more boundary regions. The optical engine may be configured to apply a hologram blur function to the hologram in each of the one or more boundary regions. The hologram blur function can thus change the gradient of the pixel values ​​in the boundary region or in each boundary region. This smooths or blurs the pixel (phase) values ​​of the hologram at each boundary / boundary between the first and second regions, reducing the magnitude of the difference between adjacent pixel values ​​at the boundary and thus reducing the so-called halo effect. In other words, in more complex holograms with multiple boundaries, there may be relatively large changes in pixel (phase) values ​​at the boundaries where multiple first and second regions are detected, and this may contribute to the halo effect. By applying a hologram blur function, the halo effect can be significantly reduced even in more complex holograms.

[0047] In some embodiments, the hologram is arranged such that each image point of the holographic image is formed using a contiguous group of pixels on the display device. Each contiguous group of pixels may constitute less than 10% of the total number of pixels on the display device. Each contiguous group of pixels may constitute less than 5% of the total number of pixels on the display device. Each contiguous group of pixels may consist of less than 100,000 pixels, optionally less than 25,000 pixels, optionally less than 1,000 pixels, or optionally less than 100 pixels. This allows the holographic image to be formed at a relatively short distance (short distance as defined above) from the display device / relay display device.

[0048] In some embodiments, the optical engine may further include a waveguide. The waveguide is arranged to guide the received holographic wavefront between a pair of reflective surfaces, one of which is partially transparent, from which multiple replicas of the holographic wavefront are emitted.

[0049] In some embodiments, the optical engine may further include optical components. These optical components may be positioned between the holographic reconstructed image and the waveguide. The optical components may also be positioned upstream of the display device to form a virtual image of the holographic reconstructed image.

[0050] In some embodiments, the optical engine further comprises an optical relay between the display device and the waveguide, the optical relay comprising two lenses arranged to cooperate in forming a relayed hologram reconstructed image. The relayed hologram reconstructed image may be an image of the hologram reconstructed image formed by the hologram displayed on the display device. The distance between the relayed hologram reconstructed image and the optical component is less than the focal length of the optical component, and the virtual image of the hologram reconstructed image formed by the optical component may be a virtual image of the relayed hologram reconstructed image.

[0051] In some embodiments, the optical components are arranged such that the holographic wavefront coupled to the waveguide is a holographic wavefront transform that encodes an image, and optionally a hologram or a holographic wavefront Fourier transform.

[0052] In a second embodiment, a vehicle head-up display is provided that includes the optical engine of the preceding embodiment.

[0053] A third embodiment provides a method for processing a hologram. In some embodiments, the method is a method for processing the boundary region of a hologram. In some embodiments, the method is a method for processing a hologram such that the hologram is blurred or smoothed in the boundary region.

[0054] This method involves receiving a hologram of an image. The hologram includes an array of pixel values. In some embodiments, each pixel value is or corresponds to a phase delay value. This phase delay value can be used to control the phase delay of pixels in a display device such as a spatial light modulator. The hologram includes a first region that includes or consists of a diffraction component. The hologram includes a second region that includes or consists of a non-diffraction component. The first region is adjacent to the second region.

[0055] This method further includes applying a hologram blur function to the hologram in the boundary region between the first and second regions to output a modified hologram. In embodiments, the boundary region includes the boundary between the hologram region and the empty region. In embodiments, the boundary region overlaps with at least one, or optionally both, a portion of the first region and a portion of the second region. The hologram blur function is configured to vary the gradient of pixel values ​​in the boundary region.

[0056] In a fourth embodiment, a diffraction structure or hologram is provided. This diffraction structure or hologram may also be referred to as a “blurred” or “processed” diffraction structure or hologram. While the term "diffraction structure" is used herein, it is clear that the term “diffraction structure” may be replaced with “hologram.”

[0057] The diffraction structure comprises an array of pixel values. The diffraction structure is arranged to spatially modulate light to form a holographic wavefront. The holographic wavefront is arranged to form a holographic reconstruction of the image encoded in the diffraction structure. The diffraction structure comprises a first region containing diffracted components and a second region containing non-diffracted components. The diffraction structure further comprises a boundary region between the first and second regions. In embodiments, the boundary region of the hologram is a region where the hologram is blurred. In particular, the boundary region of the hologram may be a region to which a holographic blur function is applied to the hologram. This reduces the difference in pixel values ​​between adjacent pixels. The boundary region comprises a first end adjacent to the first region and a second end located on the opposite side of the first end and adjacent to the second region. The diffraction structure is arranged such that the pixel values ​​in the boundary region gradually change (gradually increase or gradually decrease) depending on the distance from the second end.

[0058] In this disclosure, the term “replica” is used solely to indicate that spatially modulated light is split and the composite light field is directed along multiple different optical paths. The term “replica” is used to refer to each occurrence or instance of the composite light field after a replication event (e.g., partial reflection and transmission by a pupil dilator). Each replica travels along a different optical path. Some embodiments of this disclosure relate to the propagation of light encoded in a hologram rather than an image; that is, to the propagation of light spatially modulated in a hologram of an image, rather than the image itself. Thus, it can be said that multiple replicas of the hologram are formed. Those skilled in the art of holography will understand that the composite light field associated with the propagation of hologram-encoded light changes with respect to propagation distance. The use of the term “replica” in this specification is independent of propagation distance, and therefore, two light branches or paths associated with a replication event are called “replicas” of each other, even if the lengths of the branches are different. In other words, the composite light field evolves differently along each path. That is, two composite light fields are considered “replicas” according to this disclosure, even if their propagation distances are different. However, this is conditional on them originating from the same replication event or a series of replication events.

[0059] In this disclosure, “diffractive field” or “diffractive optical field” refers to an optical field formed by diffraction. A diffractive optical field can be formed by irradiating with a corresponding diffraction pattern. According to this disclosure, an example of a diffraction pattern is a hologram, and an example of a diffractive optical field is a holographic optical field, or an optical field that forms a holographic reconstruction of an image. A holographic optical field forms a (holographic) reconstruction of the image on the reconstruction plane. A holographic optical field propagating from a hologram to a reconstruction plane can be said to contain light encoded by the hologram, or light within the holographic region. A diffractive optical field is characterized by a diffraction angle determined by the minimum feature size of the diffractive structure and the wavelength of the light (of the diffractive optical field). According to this disclosure, a “diffractive optical field” can also be said to be an optical field that forms a reconstructed image on a plane spatially separated from the corresponding diffractive structure. This specification discloses an optical system for propagating a diffractive optical field from a diffractive structure to an observer. A diffractive optical field can form an image.

[0060] The term “hologram” is used to refer to a record containing amplitude information, phase information, or a combination thereof relating to an object. The term “holographic reconstructed image” is used to refer to the optical reconstructed image of an object formed by illuminating a hologram. The systems disclosed herein are described as “holographic projectors” because the holographic reconstructed image is a real image and is spatially separated from the hologram. The term “reconstruction field” is used to refer to the two-dimensional region in which the holographic reconstructed image is formed and fully focused. When a hologram is displayed on a spatial light modulator containing pixels, the reconstruction field is repeated in the form of multiple diffraction orders, each diffraction order being a copy of the zeroth-order reconstruction field. The zeroth-order reconstruction field is the brightest reconstruction field and therefore generally corresponds to the preferred or primary reconstruction field. Unless otherwise specified, the term “reconstruction field” is interpreted to refer to the zeroth-order reconstruction field. The term “reconstruction plane” is used to refer to a plane in space that contains all reconstruction fields. The terms “image,” “reconstructed image,” and “image region” refer to the region of the reconstruction field illuminated by the light of the holographic reconstructed image. In some embodiments, the “image” may consist of discrete spots called “image spots” or, for convenience, “image pixels.”

[0061] The terms "encoding," "writing," or "addressing" are used to describe the process of providing multiple control values ​​to multiple pixels of an SLM, each determining the modulation level of that pixel. It can be said that the pixels of the SLM are configured to "display" an optical modulation distribution in response to the reception of these control values. Therefore, it can be said that the SLM "displays" a hologram, which can be thought of as an array of optical modulation values ​​or levels.

[0062] It has been discovered that a holographic reconstruction image of acceptable quality can be formed from a “hologram” that contains only phase information related to the Fourier transform of the original object. Such holographic recordings are sometimes called phase-only holograms. Although the embodiments relate to phase-only holograms, this disclosure also applies to amplitude-only holography.

[0063] This disclosure is also applicable to forming a holographic reconstruction image using amplitude and phase information associated with the Fourier transform of the original object. In some embodiments, this is achieved by complex modulation using a so-called fully complex hologram, which includes both amplitude and phase information associated with the original object. Such a hologram is sometimes called a fully complex hologram because the value (gray level) assigned to each pixel of the hologram has both amplitude and phase components. The value (gray level) assigned to each pixel can be represented as a complex number having both amplitude and phase components. In some embodiments, a fully complex computer-generated hologram is computed.

[0064] The terms phase value, phase component, phase information, or simply phase may be used as an abbreviation for “phase delay” when referring to the phase of a pixel in a computer-generated hologram or spatial light modulator. That is, the phase value is actually a numerical value (e.g., in the range of 0 to 2π) that represents the amount of phase delay provided by that pixel. For example, a pixel in a spatial light modulator described as having a phase value of π / 2 delays the phase of the received light by π / 2 radians. In some embodiments, each pixel in a spatial light modulator can operate at any of several possible modulation values ​​(e.g., phase delay values). The term “gray level” may be used to refer to multiple available modulation levels. For example, the term “gray level” may be used for convenience to refer to multiple available phase levels in a phase-only modulator, although different phase levels may not provide different shades of gray. The term “gray level” may also be used for convenience to refer to multiple available complex modulation levels in a complex modulator.

[0065] Therefore, a hologram consists of a grayscale array, i.e., an array of optical modulation values ​​such as phase delay values ​​or complex modulation values. A hologram can also be considered a diffraction pattern, as it is a pattern that is displayed on a spatial light modulator and causes diffraction when illuminated with light of a wavelength equivalent to (usually shorter than) the pixel pitch of the spatial light modulator. This specification refers to combining holograms with other diffraction patterns, such as diffraction patterns that function as lenses or gratings. For example, a diffraction pattern that functions as a grating can be combined with a hologram to translate the reconstructed field on the reconstructed surface, or a diffraction pattern that functions as a lens can be combined with a hologram to focus the holographic reconstructed image on the reconstructed surface in the near field.

[0066] In the following detailed description, different embodiments and groups of embodiments may be disclosed separately, but any feature of any embodiment or group of embodiments can be combined with other features or combinations of features of any embodiment or group of embodiments. In other words, all possible combinations and permutations of the features disclosed herein are assumed. [Brief explanation of the drawing]

[0067] Specific embodiments are described only as examples, with reference to the following diagram.

[0068] [Figure 1] Figure 1 is a schematic diagram showing a reflective SLM that generates a holographic image on a screen. [Figure 2] Figure 2 shows a projection image containing eight image regions / components V1 to V8, and cross-sections of the corresponding hologram channels H1 to H8. [Figure 3] Figure 3 shows a hologram displayed on an LCOS that directs light to multiple separate regions. [Figure 4] Figure 4 shows a system including a display device that displays the calculated hologram as shown in Figures 2 and 3. [Figure 5A] Figure 5A shows a perspective view of a two-dimensional pupil dilator of the first embodiment, which comprises two replicators, each containing a pair of stacked surfaces. [Figure 5B] Figure 5B shows a perspective view of a two-dimensional pupil dilator of the first embodiment, which comprises two replicators, each having the shape of a solid waveguide. [Figure 6] Figure 6 is a schematic cross-sectional view of the optical components of the optical system. [Figure 7] Figure 7 is a schematic cross-sectional view of the optical components of another optical system, which includes optical relays arranged to form a virtual image of a hologram relayed to infinity. [Figure 8] Figure 8 is a schematic cross-sectional ray diagram showing the characteristics of the optical system in Figure 7. [Figure 9] Figure 9 is a schematic cross-sectional view of an optical component of yet another optical system, which includes an optical relay and is configured to form a real image of the relayed hologram. [Figure 10] Figure 10 is a schematic cross-sectional ray diagram showing the characteristics of the optical system in Figure 9. [Figure 11] Figure 11 is a schematic cross-sectional view showing a part of a pixelation display device that forms a holographic reconstructed image containing multiple image points. [Figure 12A] Figure 12A shows a schematic diagram of the holographic reconstruction of the first image consisting of the first diffraction region. [Figure 12B] Figure 12B shows a schematic diagram of the holographic reconstruction of the second image, which includes multiple diffracted and non-diffracted regions. [Figure 13] Figure 13 schematically shows the arrangement of phase values ​​of the hologram. [Figure 14] Figure 14 schematically shows the array of binary hologram values, including the boundary, detected by applying a binary filter to the hologram in Figure 13. [Figure 15] Figure 15 shows the hologram from Figure 13 with the detected boundary from Figure 14 applied, and the boundary region containing the detected boundary. Figure 15 also illustrates the process of applying the hologram blur function to the phase values ​​within the boundary region of the hologram.

[0069] Throughout the drawing, the same reference number is used to refer to the same or similar parts. [Modes for carrying out the invention]

[0070] The present invention is not limited to the embodiments described below, but extends to the entire scope of the appended claims. That is, the present invention can be carried out in different forms and should not be construed as being limited to the embodiments described for illustrative purposes.

[0071] Unless otherwise specified, singular terms may include their plural forms.

[0072] Structures described as being formed on or below other structures, or above or below other structures, are interpreted to include cases where structures are in contact with each other, and even cases where a third structure is positioned between them.

[0073] When describing temporal relationships, for example, if the temporal order of events is described as "later," "successive," "next," or "previous," unless otherwise specified, this disclosure should be interpreted as including both consecutive and non-consecutive events. For example, unless words such as "just," "immediately," or "directly" are used, the description should be interpreted as including cases that are not consecutive.

[0074] In this specification, terms such as “first,” “second,” etc., may be used to describe various elements, but these elements are not limited by these terms. These terms are used solely to distinguish one element from another. For example, without departing from the scope of the appended claims, the first element may be called the second element, and similarly, the second element may be called the first element.

[0075] Features of different embodiments can be combined or combined with each other, either partially or entirely, and can interoperate with each other in various ways. Depending on the embodiment, they can be performed independently or together in an interdependent manner.

[0076] In this disclosure, the term “substantially” when applied to a structural unit of an apparatus may be interpreted as the technical features of the structural unit being generated within the technical tolerances of the methods used to manufacture it.

[0077] Conventional optical configurations of holographic projection Figure 1 shows an embodiment in which a computer-generated hologram is encoded into a single spatial light modulator. The computer-generated hologram is the Fourier transform of the object for reconstruction. Therefore, the hologram can be said to be the Fourier domain, frequency domain, or spectral domain representation of the object. In this embodiment, the spatial light modulator is a reflective liquid crystal (LCOS) device on silicon. The hologram is encoded into the spatial light modulator, and a reconstructed holographic image is formed at a reconstructed field, such as a light receiving surface like a screen or diffuser.

[0078] A light source 110, such as a laser or laser diode, is positioned to illuminate the SLM 140 via a collimating lens 111. The collimating lens causes a nearly planar wavefront of light to be incident on the SLM. In Figure 1, the direction of the wavefront is not perpendicular (for example, 2 or 3 degrees away from true orthogonality with respect to the plane of the transparent layer). However, in other embodiments, a nearly planar wavefront is provided with perpendicular incidence, and a beam splitter arrangement is used to separate the input and output optical paths. In the embodiment shown in Figure 1, light from the light source is reflected off the back of the mirror on the SLM and is positioned to interact with the optical modulation layer to form an output wavefront 112. The output wavefront 112 is applied to an optical system including a Fourier transform lens 120 focused on the screen 125. More specifically, the Fourier transform lens 120 receives the modulated beam of light from the SLM 140 and performs a frequency-space transformation to generate a holographic reconstructed image on the screen 125.

[0079] In particular, in this type of holography, each pixel of the hologram contributes to the entire reconstructed image. There is no one-to-one correlation between a specific point (or image pixel) on the reconstruction field and a specific light modulation element (or hologram pixel). In other words, the modulated light emanating from the light modulation layer is distributed throughout the entire reconstruction field.

[0080] In these embodiments, the position of the holographic image in space is determined by the refractive power (focusing power) of the Fourier transform lens. In the embodiment shown in Figure 1, the Fourier transform lens is a physical lens; that is, the Fourier transform lens is an optical Fourier transform lens, and the Fourier transform is performed optically. Any lens can function as a Fourier transform lens, but the performance of the lens limits the accuracy of the Fourier transform performed. Those skilled in the art understand how to perform an optical Fourier transform using a lens. In some embodiments of this disclosure, the lens of the observer's eye performs the hologram-to-image transformation.

[0081] Hologram calculation In some embodiments, the computer-generated hologram is a Fourier transform hologram, or simply a Fourier hologram, or a Fourier-based hologram, in which the image is reconstructed in the far field using the Fourier transform properties of a positive lens. A Fourier hologram is computed by a Fourier transform that returns the desired light field of the reconstruction plane to the lens plane. Computer-generated Fourier holograms can be computed using the Fourier transform. Embodiments relate only, as examples, to Fourier holography and Gerchberg-Saxton type algorithms. This disclosure is equally applicable to Fresnel holography and Fresnel holograms, which can be computed in a similar manner. In some embodiments, the hologram is a phase or phase-only hologram. However, this disclosure is also applicable to holograms computed by other techniques, such as those based on the point cloud method.

[0082] In some embodiments, the hologram engine is configured to exclude the contribution of light blocked by the limiting aperture of the display system from the hologram calculation. UK Patent Application 2101666.2, filed 5 February 2021 and incorporated herein by reference, discloses a first hologram calculation method that uses eye-tracking and ray tracing to identify sub-areas of a display device for calculating a point cloud hologram that eliminates ghost images. The sub-areas of the display device correspond to the aperture in this disclosure and are used to exclude the light path from the hologram calculation. UK Patent Application 2112213.0, filed 26 August 2021 and incorporated herein by reference, discloses a second method based on a modified Gercberg-Saxton algorithm, which includes the step of cropping the light field according to the pupil of the optical system during the hologram calculation. The cropping of the light field corresponds to the determination of the limiting aperture in this disclosure. Filing on 23 December 2021 and incorporated herein by reference, UK Patent Application 2118911.3 discloses a third method for calculating a hologram, comprising the step of determining a region of a so-called expansion modulator formed by a hologram replicator. According to this disclosure, the region of the expansion modulator is also an opening.

[0083] In some embodiments, a real-time engine is provided that is configured to receive image data and compute holograms in real time using an algorithm. In some embodiments, the image data is a video containing a series of image frames. In other embodiments, the holograms are pre-computed, stored in computer memory, and called upon as needed to be displayed on the SLM. In other words, in some embodiments, a repository of predetermined holograms is provided.

[0084] Wide field of view using a small display device Broadly speaking, this disclosure relates to image projection. This disclosure relates to an image projector comprising an image projection method and a display device. This disclosure also relates to a projection system comprising an image projector and a display system. In this projection system, the image projector projects or relays light from a display device to a display system. This disclosure is equally applicable to monocular and binocular display systems. The display system may comprise one or more eyes of the viewer. The display system comprises optical elements having optical power (e.g., the lens of a human eye) and a display surface (e.g., the retina of a human eye). The projector is sometimes referred to as an “optical engine”. The display device and the image formed (or perceived) using the display device are spatially separated from each other. The image is formed on a display surface or perceived by the viewer. In some embodiments, the image is a virtual image, and the display surface may be referred to as a virtual image surface. In other examples, the image is a real image formed by a holographic reconstructed image, and the image is projected or relayed to the display surface. In these other examples, spatially modulated light of an intermediate holographic reconstructed image formed in free space or on a screen or other light-receiving surface between the display device and the viewer is propagated to the viewer. In both cases, the image is formed by irradiating the display device with a diffraction pattern (such as a hologram or kinoform).

[0085] A display device is composed of pixels. The pixels of a display can show diffraction patterns or structures that diffract light. The diffracted light forms an image on a plane spatially distant from the display device. According to well-known optics, the magnitude of the maximum diffraction angle is determined by other factors such as the size of the pixel and the wavelength of light.

[0086] In some embodiments, the display device is a spatial light modulator, such as a liquid crystal on silicon ("LCOS") spatial light modulator (SLM). Light propagates from the LCOS towards a display entity / system, such as a camera or eye, across a range of diffraction angles (e.g., from zero to the maximum diffraction angle). In some embodiments, magnification techniques can be used to extend the range of available diffraction angles beyond the conventional maximum diffraction angle of the LCOS.

[0087] In some embodiments, the hologram (or its light) itself is transmitted to the eye. For example, the spatially modulated light of the hologram (the holographic reconstructed image, i.e., not yet fully converted into an image), which could informally be said to be "encoded" by the hologram, is transmitted directly to the viewer's eye. The viewer can perceive a real or virtual image. In these embodiments, no intermediate holographic reconstructed image / image is formed between the display device and the viewer. In these embodiments, it may be said that the lens of the eye performs the conversion or transformation from hologram to image. The projection system or optical engine can be configured so that the viewer effectively looks directly at the display device.

[0088] The specification refers to a “light field,” but this is a “complex light field.” The term “light field” simply refers to a pattern of light with a finite size in at least two orthogonal spatial directions, e.g., x and y. The word “complex” is used herein solely to indicate that the light at each point in the light field may be defined by amplitude and phase values ​​and therefore representable by complex numbers or pairs of values. For the purposes of holographic calculations, a complex light field is a two-dimensional array of complex numbers, where the complex numbers define the intensity and phase of light at multiple discrete locations within the light field.

[0089] According to well-known optical principles, the range of angles of light propagating from a display device that can be observed by the eye or other object / system of observation depends on the distance between the display device and the object of observation. For example, at an observation distance of 1 meter, only a small portion of the angle from the LCOS (Low-Core Optical System) can pass through the pupil of the eye and form an image on the retina at a particular eye position. The range of angles of light rays propagating from the display device determines the range that can pass through the pupil of the eye and form an image on the retina, and thus determines the portion of the image that the observer "sees." In other words, not all parts of the image are visible from any single point on the observation plane (for example, any single eye position within an observation window such as an eyebox).

[0090] In some embodiments, the image perceived by the viewer is a virtual image displayed upstream of the display device. That is, the viewer perceives the image as being farther away from the display device. Conceptually, one can think of the viewer as seeing a virtual image through a very small "display device-sized window," such as a 1 cm diameter window, at a relatively large distance, e.g., 1 m. Furthermore, the user is viewing this display device-sized window through a very small pupil of the eye. Consequently, the field of view is narrowed, and the specific angular range that can be seen depends heavily on the position of the eye at any given time.

[0091] Pupil dilators address the problem of how to extend the angular range of light rays propagated from a display device and pass through the pupil of the eye effectively to form an image. Display devices are generally (relatively) small, and the projection distance is (relatively) large. In some embodiments, the projection distance is at least one order of magnitude, for example, at least two orders of magnitude, larger than the diameter or width (i.e., the size of the pixel array) of the entrance pupil and / or opening of the display device.

[0092] Using a pupil dilator expands the field of view (i.e., the user's eye box) laterally, allowing eye movement while the user can still see the image. As those skilled in the art will understand, in an image system, the field of view (the user's eye box) is the area in which the observer's eye can perceive an image. This disclosure deals with non-infinite virtual image distances, i.e., near-field virtual images.

[0093] Conventionally, two-dimensional pupil dilators consist of one or more one-dimensional optical waveguides, each formed using a pair of opposing reflective surfaces, where the output light from the surfaces forms an observation window or eyebox. Light received from a display device (e.g., spatially modulated light from an LCOS) is replicated by the waveguides or each waveguide so that the field of view (or display area) is expanded in at least one dimension. In particular, the waveguides expand the observation window by generating additional rays or "replicas" through the division of the amplitude of the incident wavefront.

[0094] The display device may have an active area or display area that is less than 10 cm, for example, less than 5 cm or less than 2 cm. The propagation distance between the display device and the display system may be more than 1 m, for example, more than 1.5 m or more than 2 m. The optical propagation distance in the waveguide may be up to 2 m, for example, up to 1.5 m or up to 1 m. This method can receive an image and determine a corresponding hologram of sufficient quality in less than 20 ms, for example, less than 15 ms or less than 10 ms.

[0095] In some embodiments, although described only as examples of diffracted or holographic light fields according to this disclosure, a hologram is configured to route light into multiple channels, each corresponding to a different part (i.e., sub-area) of the image. The channels formed by the diffracted structure are referred to here simply as “holographic channels” to reflect that they are channels of light encoded by the hologram with image information. The light in each channel can be said to be in the holographic region, not the image or spatial region. In some embodiments, the hologram is a Fourier or Fourier transform hologram, and therefore the holographic region is the Fourier or frequency domain. The hologram may also be a Fresnel or Fresnel transform hologram. The hologram may also be a point cloud hologram. Hereinafter, a hologram is described as routing light into multiple holographic channels to reflect that the image that can be reconstructed from the hologram has a finite size and can be arbitrarily divided into multiple image sub-regions, each holographic channel corresponding to each image sub-region. Importantly, the hologram in this example is characterized by how it distributes its image content when illuminated. Specifically, the hologram divides its image content by angle. In other words, each point on the image is associated with an intrinsic ray angle in the spatially modulated light formed by the hologram when illuminated. At the very least, since the hologram is two-dimensional, it is a pair of intrinsic angles. To avoid any doubt, the behavior of this hologram is different from conventional ones. The spatially modulated light formed by this special type of hologram, when illuminated, is divided into multiple holographic channels, each holographic channel defined by a range of ray angles (two-dimensional). From the above, it is understood that the holographic channels (i.e., subranges of ray angles) that may be considered in spatially modulated light are associated with each part or subregion of the image. That is, all the information necessary to reconstruct that part or subregion of the image is contained within the subrange of angles of the spatially modulated light formed from the hologram of the image.When spatially modulated light is observed as a whole, evidence of multiple individual optical channels is not necessarily present.

[0096] Nevertheless, holograms are identifiable. For example, if only a continuum or subregion of the spatially modulated light formed by the hologram is reconstructed, only a subregion of the image should be visible. If different continuums or subregions of the spatially modulated light are reconstructed, different subregions of the image should be visible. A further identifying feature of this type of hologram is that the shape of the cross-sectional area of ​​the hologram channel substantially corresponds to (i.e., is substantially the same as) the shape of the entrance pupil, although their size may differ, at least in the correct plane from which the hologram was calculated. Each light / hologram channel propagates from the hologram at different angles or angular ranges. These are exemplary ways of characterizing or identifying this type of hologram, but other methods may also be used. In summary, the holograms disclosed herein are characterized and identifiable by how the image content is dispersed within the light encoded by the hologram. Again, to avoid any doubt, references herein to holograms configured to guide light or to angularly divide an image into multiple holographic channels are made for illustrative purposes only, and this disclosure is equally applicable to any type of holographic light field, and even to any type of diffracted light field or pupil dilation of a diffracted light field.

[0097] This system can be delivered in a compact and streamlined physical form. This allows for the creation of systems suitable for a variety of real-world applications, such as those where space is limited and real estate value is high. For example, it can be implemented in head-up displays (HUDs) such as those in vehicles and automobiles.

[0098] According to this disclosure, pupil dilation is provided for diffracted light or diffracted light containing a divergent beam. The diffracted light field is defined by a “light cone”. Thus, the size of the diffracted light field (defined on a two-dimensional plane) increases with the propagation distance from the corresponding diffracting structure (i.e., the display device). It can be said that the pupil dilator replicates a hologram or forms a replica of at least one hologram, conveying that the light delivered to the viewer is spatially modulated according to the hologram.

[0099] In some embodiments, two one-dimensional waveguide pupil dilators are provided, each one-dimensional waveguide pupil dilator positioned to effectively expand the size of the system's exit pupil by forming multiple replicas or copies of the exit pupil (or light from the exit pupil) of the spatial light modulator. The exit pupil can be understood as the physical region from which light is emitted by the system. It can also be said that each waveguide pupil dilator is positioned to expand the size of the system's exit pupil. Furthermore, it can also be said that each waveguide pupil dilator is positioned to expand / increase the size of the eyebox in which the observer's eye can be positioned to see / receive the light emitted by the system.

[0100] Light Channeling Holograms formed according to some embodiments provide multiple hologram channels that can have cross-sectional shapes defined by the apertures of an optical system by angularly dividing the image content. The hologram is computed to provide this channeling of the diffracted light field. In some embodiments, this is achieved during hologram computation by considering the apertures (virtual or real) of the optical system, as described above.

[0101] Figures 2 and 3 show examples of this type of hologram that can be used in combination with the pupil dilation devices disclosed herein. However, these examples are not intended to limit the invention.

[0102] Figure 2 shows a projection image 252 containing eight image regions / components V1 to V8. While Figure 2 shows eight image components as an example, image 252 can be divided into any number of components. Figure 2 also shows an encoded optical pattern 254 (i.e., a hologram) that can reconstruct image 252, such as when transformed by the lens of a suitable display system. The encoded optical pattern 254 consists of first to eighth sub-holograms or components H1 to H8, corresponding to the first to eighth image components / regions V1 to V8. Figure 2 further illustrates how a hologram decomposes its image content by angle. Thus, a hologram is characterized by the channeling of light it performs, as shown in Figure 3. Specifically, the hologram in this example directs light into multiple distinct regions. These distinct regions are disks in the example shown, but other shapes are also conceivable. The optimal disk size and shape may depend on the size and shape of the aperture of the optical system, such as the entrance pupil of the observation system, after propagation through the waveguide.

[0103] Figure 4 shows a system 400 that includes a display device for displaying the calculated hologram as shown in Figures 2 and 3.

[0104] System 400 includes a display device, in this configuration, an LCOS 402. The LCOS 402 displays a modulation pattern (or "diffraction pattern") containing a hologram and is positioned to project the holographically encoded light toward an eye 405, which includes a pupil, a lens 409, and a retina (not shown), which serves as an aperture 404. There is a light source (not shown) positioned to illuminate the LCOS 402. The lens 409 of the eye 405 performs the conversion from hologram to image. The light source may be of any suitable type; for example, it may be a laser light source.

[0105] The visual system 400 further includes a waveguide 408 positioned between the LCOS 402 and the eye 405. The presence of waveguide 408 ensures that all angular content from the LCOS 402 is received by the eye, even at the relatively long projection distance shown in the figure. This is because waveguide 508 acts as a pupil dilator. This method is well known and will be described only briefly here.

[0106] Simply put, the waveguide 408 shown in Figure 4 consists of a substantially elongated structure. In this example, waveguide 408 is made of an optical slab of refractive material, but other types of waveguides are also known and may be used. Waveguide 408 is positioned, for example, at an oblique angle so as to intersect the light cone (i.e., the diffracted light field) projected from LCOS 402. In this example, the size, location, and position of waveguide 408 are configured so that light from each of the eight ray beams in the light cone enters waveguide 408. Light from the light cone enters waveguide 408 via a first planar surface of waveguide 408 (closest to LCOS 402), is at least partially guided along the length of waveguide 408, and then emitted via a second planar surface (closest to the eye) substantially opposite the first surface. As is well understood, the second plane is partially reflective and partially transmissive. In other words, as each ray travels from the first plane through the waveguide 408 and strikes the second plane, some of the light is transmitted through the waveguide 408, and some is reflected by the second plane and returns to the first plane. The first plane is reflective, and all light that strikes the first plane from within the waveguide 408 is reflected by the second plane. Therefore, some of the light is refracted between the two planes of the waveguide 408 and then transmitted, while other light is reflected and undergoes one or more reflections (or "bounces") between the planes of the waveguide 408 before being transmitted.

[0107] Figure 4 shows a total of nine “reflection” points B0 to B8 along the length of waveguide 408. As shown in Figure 2, the light associated with all points in the image (V1-V8) is transmitted from the waveguide at each “reflection” from the second plane of waveguide 408, but only the light from one angular portion of the image (for example, any light from V1 to V8) has a trajectory that allows it to reach eye 405 from each “reflection” point B0 to B8. Furthermore, the light from different angular portions of the image V1 to V8 also reaches eye 405 from each “reflection” point. Thus, in the example in Figure 4, each angular channel of encoded light reaches the eye only once from waveguide 408.

[0108] Waveguide 408 forms multiple replicas of the hologram along its length at respective “bounce” points B1-B8 corresponding to the direction of pupil dilation. As shown in Figure 5, the multiple replicas are extrapolated linearly to the corresponding multiple replicas or virtual display devices 402'. This process corresponds to the step of “unfolding” the optical path within the waveguide so that the rays of the replicas are extrapolated to the “virtual surface” without internal reflection within the waveguide. Thus, the light from the dilated exit pupil can be considered to originate from the virtual surface (also referred to herein as the “diminished modulator”) that constitutes the display device 402 and the replica display device 402'.

[0109] While this specification has generally described virtual images, which require the eye to transform received modulated light to form a perceived image, the methods and configurations described herein can also be applied to real images.

[0110] 2D pupil dilation The configuration shown in Figure 4 includes a single waveguide that provides pupil dilation in one dimension, but pupil dilation can be provided in multiple dimensions, such as two dimensions. Furthermore, the example in Figure 4 uses a computed hologram to create light channels corresponding to different parts of the image, but the systems described in this disclosure and below are not limited to such hologram types.

[0111] Figure 5A shows a perspective view of system 500, which includes two replicators 504 and 506 arranged to extend the ray 502 into two dimensions.

[0112] In the system 500 of Figure 5A, the first replicator 504 consists of a first pair of surfaces stacked parallel to each other and arranged to provide replication (or pupil dilation) similar to the waveguide 408 of Figure 4. The first pair of surfaces are similar (and possibly identical) in size and shape to each other and are substantially elongated in one direction. The collimated ray 502 is directed towards the input of the first replicator 504. As is well known to those skilled in the art, due to an internal reflection process between the two surfaces and partial transmission of light from each of several output points on one of the surfaces (the top surface as shown in Figure 5A), the light of the ray 502 is replicated in a first direction along the length of the first replicator 504. Thus, a first plurality of replica rays 508 are emitted from the first replicator 504 toward the second replicator 506.

[0113] The second replicator 506 comprises a second pair of surfaces stacked parallel to each other and is positioned to receive each of the collimated rays of the first plurality of rays 508, and further positioned to replicate, i.e., provide pupil dilation, by extending each of those rays in a second direction substantially perpendicular to the first direction. The first pair of surfaces are similar (and possibly identical) in size and shape to each other and are substantially rectangular. The second replicator is implemented in a rectangular shape so that it has a length along the first direction to receive the first plurality of rays 508, a length along the second orthogonal direction, and provides replication in the second direction. Through the process of internal reflection between the two surfaces, and the partial transmission of light from each of the multiple output points on one of the surfaces (the top surface as shown in Figure 5A), the light of each ray in the first plurality of rays 508 is replicated in the second direction. Therefore, a second plurality of rays 510 are emitted from the second replication device 506, and the second plurality of rays 510 include replicas of the input rays 502 along the first and second directions, respectively. Thus, the second plurality of rays 510 can be considered to include a two-dimensional grid or array of replica rays.

[0114] Therefore, it can be said that the combination of the first and second replicators 504 and 505 in Figure 5A provides a two-dimensional replicator (or "two-dimensional pupil dilator"). Thus, the replica rays 510 may be emitted along the optical path to an extended eye box in a display system such as a head-up display.

[0115] In the system shown in Figure 5A, the first replicator 504 is a waveguide containing a pair of elongated straight reflectors stacked parallel to each other, and similarly, the second replicator 504 is a waveguide containing a pair of rectangular reflectors stacked parallel to each other. In other systems, the first replicator is a solid elongated straight waveguide, and the second replicator is a solid planar rectangular waveguide, with each waveguide containing an optically transparent solid material such as glass. In this case, the pair of parallel reflectors are formed by a pair of opposing main sidewalls, each optionally containing reflective and reflective-transmitting surface coatings, as is well known to those skilled in the art.

[0116] Figure 5B shows a perspective view of system 500, which includes two replicators 520 and 540 arranged to replicate a ray 522 in two dimensions, the first replicator being a solid elongated waveguide 520 and the second replicator being a solid planar waveguide 540.

[0117] In the system shown in Figure 5B, the first replicator / waveguide 520 is positioned such that its pair of elongated parallel reflecting surfaces 524a, 524b are perpendicular to the plane of the second replicator / waveguide 540. Thus, the system includes an optical coupler positioned to couple light from the output port of the first replicator 520 to the input port of the second replicator 540. In the illustrated configuration, the optical coupler is a planar / bending mirror 530 positioned to bend or rotate the optical path of light to achieve the required optical coupling from the first replicator to the second replicator. As shown in Figure 5B, the mirror 530 is positioned to receive light from the output port / reflecting surface 524a of the first replicator / waveguide 520, containing a one-dimensional array of replicas extending to the first dimension. Mirror 530 is tilted at an angle that provides guidance and replica formation along the length of the second dimension, redirecting the received light into the optical path to the input port of the (perfect) reflective surface of the second replicator 540. Mirror 530 is an example of an optical element that can redirect light in the manner illustrated, and it will be understood that one or more other elements could be used instead to perform this task.

[0118] In the illustrated configuration, the (partially) reflective and transmitted surface 524a of the first replicator 520 is adjacent to the input port of the first replicator / waveguide 520, receiving the input beam 522 at a certain angle to perform guiding and replica formation along the length of the first dimension. Thus, the input port of the first replicator / waveguide 520 is located at the input end of the same surface as the reflective and transmitted surface 524a. Those skilled in the art will understand that the input port of the first replicator / waveguide 520 may be located in other suitable locations.

[0119] Therefore, the arrangement in Figure 5B allows the first replicator 520 and the mirror 530 to be provided as part of a relatively thin first layer in the planes of the first and third dimensions (illustrated as the xz plane). In particular, the size or "height" of the first planar layer in which the first replicator 520 is located is reduced in the second dimension (illustrated as the y dimension). The mirror 530 is configured to direct light away from the first layer / plane in which the first replicator 520 is located (i.e., the "first planar layer") and to direct light towards the second layer / plane in which the second replicator 540 is located (i.e., the "second planar layer"), which is above the first layer / plane and substantially parallel to the first layer / plane. Therefore, the overall size or "height" of the system, including the first and second replicators 520, 540 and the mirrors 530 arranged in stacked first and second planar layers in the first and third dimensions (illustrated as the xz plane), is compact in the second dimension (illustrated as the y dimension). Those skilled in the art will understand that many variations of the arrangement in Figure 5B for implementing this disclosure are possible and have been considered.

[0120] The image projector may be positioned to project a divergent or diffracted light field. In some embodiments, the diffracted light field is encoded as a hologram. In some embodiments, the diffracted light field consists of a beam of divergent rays. In some embodiments, the image formed by the diffracted light field is a virtual image.

[0121] In some embodiments, the first pair of parallel / complementary surfaces are elongated or slender surfaces, relatively long along the first dimension and relatively short along the second dimension, for example, relatively short along each of the two other dimensions, with each dimension substantially orthogonal to the other. The process of reflection / transmission of light from between the first pair of parallel surfaces is arranged so that the light propagates within the first waveguide pupil dilator, and the general direction of light propagation is in the direction in which the first waveguide pupil dilator is relatively long (i.e., its “slender” direction).

[0122] This specification discloses a system that uses diffracted light to form an image and provide an eyebox size and field of view suitable for real-world applications, such as applications in the automotive industry using head-up displays. Diffracted light is light that forms a holographic reconstruction of an image from a diffracted structure, such as a Fourier hologram or a Fresnel hologram. The use of diffraction and diffracted structures requires a display device with a high density of very small pixels (e.g., 1 micrometer), which in practice means a small display device (e.g., 1 centimeter). The inventors have addressed the problem of a method for providing 2D pupil dilation with a diffracted light field, for example, diffracted light consisting of a diverging (uncollimated) beam of light rays.

[0123] In some embodiments, the display system comprises a display device, such as a spatial light modulator (SLM) or a pixelated display device, such as a Liquid Crystal on Silicon (LCoS) SLM, arranged to supply or form diffracted or divergent light. In such embodiments, the aperture of the spatial light modulator (SLM) is the limiting aperture of the system. That is, the size of the aperture of the spatial light modulator, more specifically, the size of the region that demarcates the array of light-modulated pixels configured within the SLM, determines the size (e.g., spatial spread) of the beam of light that can exit the system. In accordance with this disclosure, the exit pupil of the system (limited by the small display device having a pixel size for light diffraction) is expanded by using at least one pupil expander to reflect the increase or decrease in spatial spread.

[0124] A diffracted or divergent light field can be said to have a "magnitude of the light field" defined in a direction substantially perpendicular to the direction of propagation of the light field. Because light diffracts / diverges, the magnitude of the light field increases with propagation distance.

[0125] In some embodiments, the diffracted light field is spatially modulated according to the hologram. In other words, in such embodiments, the diffracted light field constitutes a “holographic light field.” The hologram may be displayed on a pixelated display device. The hologram may be a computer-generated hologram (CGH). The hologram may be a Fourier hologram, a Fresnel hologram, a point cloud hologram, or any other suitable type of hologram. The hologram may optionally be computed to form channels of holographic light, each corresponding to a different part of the image that the viewer is intended to see (or perceive, if it is a virtual image). The pixelated display device may be configured to display multiple different holograms sequentially or in sequence. Each embodiment and example disclosed herein can be applied to the display of multiple holograms.

[0126] The output port of the first waveguide pupil dilator may be coupled to the input port of the second waveguide pupil dilator. The second waveguide pupil dilator may be positioned to guide the diffracted light field—including a portion, preferably a large portion, preferably all, of the replica of the light field output by the first waveguide pupil dilator—from its input port to its respective output port by internal reflection between a third pair of parallel planes of the second waveguide pupil dilator.

[0127] A first waveguide pupil dilator may be positioned to provide pupil dilation or replication in a first direction, and a second waveguide pupil dilator may be positioned to provide pupil dilation or replication in a second different direction. The second direction may be substantially orthogonal to the first direction. The second waveguide pupil dilator may be positioned to maintain the pupil dilation provided by the first waveguide pupil dilator in the first direction and to dilate (or replicate) a portion, preferably most, preferably all, of the replicas it receives from the first waveguide pupil dilator in the second different direction. The second waveguide pupil dilator may be positioned to receive the light field directly or indirectly from the first waveguide pupil dilator. One or more other elements may be provided along the light field propagation path between the first waveguide pupil dilator and the second waveguide pupil dilator.

[0128] The first waveguide pupil dilator may be substantially elongated, and the second waveguide pupil dilator may be substantially planar. The elongated shape of the first waveguide pupil dilator may be defined by a length along a first dimension. The planar, i.e., rectangular shape of the second waveguide pupil dilator may be defined by a length along a first dimension and a width along a second dimension substantially perpendicular to the first dimension. The length along the first dimension of the first waveguide pupil dilator corresponds to the length or width along the first or second dimension of the second waveguide pupil dilator, respectively. The first face of the pair of parallel faces of the second waveguide pupil dilator constituting its input port may be shaped, sized, and / or positioned to correspond to an area defined by the output ports on the first pair of parallel faces of the first waveguide pupil dilator, such that the second waveguide pupil dilator is positioned to receive each of the replicas output by the first waveguide pupil dilator.

[0129] The first and second waveguide pupil dilators may collectively provide pupil dilation in a first direction and a second direction perpendicular to the first direction, and optionally, the plane containing the first and second directions is substantially parallel to the plane of the second waveguide pupil dilator. In other words, the first and second dimensions defining the length and width of the second waveguide pupil dilator, respectively, may be parallel to the first and second directions in which the waveguide pupil dilator provides pupil dilation (or they may be parallel to the second and first directions, respectively). The combination of the first waveguide pupil dilator and the second waveguide pupil dilator is sometimes commonly referred to as a “pupil dilator”.

[0130] The expansion / replication provided by the first and second waveguide expanders can be said to have the effect of expanding the exit pupil of the display system in each of the two directions. The area defined by the expanded exit pupil may define an expanded eyebox area from which the viewer can receive light from the input diffracted or divergent light field. The eyebox area can be said to be located on or define the field of view plane.

[0131] The two directions in which the exit pupil expands may be coplanar or parallel to the first and second directions in which the first and second waveguide pupil dilators provide replication / expansion. Alternatively, in arrangements that include other elements such as an optical combiner, e.g., a vehicle's windshield (or windshield), the exit pupil may be considered an exit pupil from the other element such as the windshield. In such arrangements, the exit pupil may be non-parallel to the first and second directions in which the first and second waveguide pupil dilators provide replication / expansion. For example, the exit pupil may be substantially perpendicular to the first and second directions in which the first and second waveguide pupil dilators provide replication / expansion.

[0132] The field of view plane and / or eyebox region may be non-planar or non-parallel to the first and second directions from which the first and second waveguide pupil dilators provide replication / dilation. For example, the field of view plane may be substantially perpendicular to the first and second directions from which the first and second waveguide pupil dilators provide replication / dilation.

[0133] To provide suitable emission conditions for achieving internal reflection within the first and second waveguide pupil dilators, the elongated dimensions of the first waveguide pupil dilator may be inclined relative to the first and second dimensions of the second waveguide pupil dilator.

[0134] Combiner Shape Correction The advantage of projecting a hologram onto an eyebox is that optical compensation can be encoded in the hologram (see, for example, European Patent No. 2936252, incorporated herein). This disclosure is compatible with holograms that compensate for the complex curvature of an optical combiner used as part of a projection system. In some embodiments, the optical combiner is a vehicle windshield. Details of this approach are described in European Patent No. 2936252, and the detailed features of those systems and methods are not essential to the novel teachings of this disclosure and are merely illustrative of configurations that may benefit from the teachings of this disclosure, so they are not repeated here.

[0135] control device This disclosure is also compatible with optical configurations including a control device (e.g., an optical shutter device) for controlling the delivery of light from an optical channeling hologram to a viewer. The holographic projector may further include a control device positioned to control the delivery of an angular channel to an eyebox position. UK Patent Application 2108456.1, filed on 14 June 2021 and incorporated herein by reference, discloses at least one waveguide pupil dilator and control device. Readers will understand from at least this prior disclosure that the optical configuration of the control device is fundamentally based on the user's eyebox position and is compatible with any hologram calculation method that realizes optical channeling as described herein. The control device can be said to be an optical shutter or aperture device. The optical shutter device comprises a 1D array of apertures or windows, each aperture or window being independently switchable between a light-transmitting state and a light-blocking state to control the delivery of the holographic optical channel and its replica to the eyebox. Each aperture or window may comprise a plurality of liquid crystal cells or pixels.

[0136] Image Formation Figure 6 is a schematic cross-sectional view of the optical components of the optical system 600, in which the relayed hologram is coupled to the waveguide 611 and replicated by the waveguide 611.

[0137] The optical axis of the optical system 600 is shown by the dotted line 602 in Figure 6. The optical system 600 includes a display device 604, which in this example is a liquid crystal on silicon space light modulator. The display device 604 is configured to display a hologram of an image. Downstream of the display device 604 is an optical relay 606. The optical relay 606 includes a first lens 608 and a second lens 610. The optical system 600 further includes a waveguide 611 downstream of the second lens 610 of the optical relay 606. The waveguide 611 includes a pair of opposing surfaces 622, 624 arranged to guide light between them, as in the example described above.

[0138] The first lens 608 of the optical relay 606 has a front focal plane 612 and a rear focal plane 614. The front focal plane 612 is upstream of the first lens 608, and the rear focal plane 614 is downstream of the first lens 608. The second lens 608 of the optical relay 606 has a front focal plane 616 and a rear focal plane 618. The front focal plane 616 is upstream of the second lens 610, and the rear focal plane 618 is downstream of the second lens 610. The normals to the front and rear focal planes of the first lens 608 and the second lens 610, respectively, are parallel to the optical axis 602, and the distance from the front and rear focal planes to the first or second lens is equal to the focal length f of the respective lens. In this example, the display device 604 is positioned approximately at the front focal plane 612 of the first lens 608. In this example, the front focal plane 616 of the second lens 610 is substantially coplanar with the back focal plane 614 of the first lens 608. In this example, the waveguide 611 is positioned such that the back focal plane 618 of the second lens 610 is located between the first and second surfaces 624 of the waveguide 622. In the example shown in Figure 6, the focal lengths f of the first lens 608 and the second lens 610 are the same. Therefore, the optical relay forms a 4f system (i.e., the length of the optical relay is equal to four times the focal length of either the first lens 608 or the second lens 610). However, in other embodiments, the focal length of the first lens 608 may differ from the focal length of the second lens 610. In such cases, the optical relay forms a magnifying (or reducing) telescope.

[0139] The optical system 600 further includes a coherent light source, such as a laser, which is not shown in Figure 6. During operation of the optical system 600, the coherent light is positioned to illuminate the display device 604. Thus, this light can be spatially modulated according to the hologram of the image displayed on the display device. The spatially modulated light is received by the first lens 608 and relayed to the second lens 610. The holographic reconstruction image 626 of the image is formed at the back focal plane 612 of the first lens 608, i.e., between the first lens 608 and the second lens 610. The second lens 610 relays the spatially modulated light to the waveguide 611. As described in relation to the aforementioned figures, the waveguide 611 replicates the light received from the display device, forming multiple replicas or copies of the hologram displayed on the display device 604. Each replica contains spatially modulated light according to the hologram on the display device. In the embodiment, the optical system further includes a second waveguide (not shown) that provides guidance and replication in a second direction, thereby outputting a two-dimensional array of replicas from the second waveguide. The spatially modulated light is relayed from the output of the second waveguide to an eyebox / viewing plane (extended by replication via the waveguide). When a viewing system (e.g., the user's eye) is placed on the eyebox / viewing plane, the viewing system receives the spatially modulated light and forms a virtual image of the hologram displayed on the display device at the virtual image distance encoded in the hologram.

[0140] The optical system 600 can provide a good virtual image of the hologram when the observation system is positioned on the observation surface / eyebox. However, artifacts may form / appear on the observation surface (i.e., the surface containing multiple replicas). Artifacts may result from dark bands caused by the display device being illuminated with light of uneven intensity, and / or from physical characteristics of the display device (e.g., scattering due to characteristics of the display device). In either case, the artifacts may be replicated by the waveguide, forming a repeating pattern of artifacts on the observation surface. Therefore, while the virtual image of the image / hologram reconstruction itself may be of good quality, the view of the virtual image of the image on the observation surface may appear to be obstructed by the repeating pattern of artifacts. The observation system may have to effectively "see through" the repeating pattern of artifacts in order to observe the virtual image.

[0141] Separation of holographic image and holographic reconstructed image Figure 7 is a schematic cross-sectional view of the optical components of the first optical system 700. This optical system is arranged so that the image of the hologram / display device is far away from the virtual image of the hologram's holographic reproduction, thereby reducing or eliminating the aforementioned artifacts. The first optical system 700 is arranged so that the Fourier transform of the hologram displayed on the display device 704 is coupled to the waveguide 711 (not the hologram itself). As will be described later, this is because a lens 750 is present between the relayed hologram and the waveguide 711.

[0142] The optical system 700 has an optical axis indicated by the dotted line 702 in Figure 7. In this example, the optical system 700 includes a display device 704, which is a liquid crystal on silicon spatial light modulator. The display device 704 is configured to display a hologram of an image. Downstream of the display device 704 is an optical relay 706. The optical relay 706 includes a first lens 708 and a second lens 710.

[0143] The display device 704 and optical relay 706 of optical system 700 are very similar to the display device 604 and optical relay 706 of optical system 600. For example, the first lens 708 of optical relay 706 has a front focal plane 712 and a rear focal plane 714. The front focal plane 712 is upstream of the first lens 708, and the rear focal plane 714 is downstream of the first lens 708. The second lens 708 of optical relay 706 has a front focal plane 716 and a rear focal plane 718. The front focal plane 716 is upstream of the second lens 710, and the rear focal plane 718 is downstream of the second lens 710. The normals to the front and rear focal planes of the first and second lenses 708 and 710, respectively, are parallel to the optical axis 702, and the distance from each front focal plane to the rear focal plane of each first or second lens is equal to the focal length f of each lens. In this example, the display device 704 is positioned approximately at the front focal plane 712 of the first lens 708. In this example, the front focal plane 716 of the second lens 710 is approximately coplanar with the rear focal plane 714 of the first lens 708. In the example shown in Figure 7, the focal lengths f of the first lens 708 and the second lens 710 are the same. Therefore, the optical relay forms a 4f system (i.e., the length of the optical relay is equal to four times the focal length f of either the first lens 708 or the second lens 710). However, in other embodiments, the focal length of the first lens 708 may be different from the focal length of the second lens 710. In such cases, the optical relay may form a magnifying (or reducing) telescope.

[0144] Unlike optical system 600, optical system 700 further includes an optical component 750 between the second lens 710 and the waveguide 711. In this example, the optical component 750 is a (third) lens. In this example, the third lens 750 is a Fourier lens. The front focal plane 754 of the third lens 750 is upstream of the third lens 750 and is substantially coplanar with the back focal plane 718 of the second lens 710. The back focal plane 757 of the third lens 750 is located between the first surface 722 and the second surface 724 of the waveguide 711.

[0145] In this example, the focal length f of the third lens 750 is the same as the focal lengths f of the first lens 708 and the second lens 710. Thus, the optical relay 706 and the third lens 750 together define a 6f system (where the distance between the front focal plane 712 of the first lens 708 and the back focal plane 757 of the third lens 750 is equal to six times the focal length of the first lens 708, the second lens 710, or the third lens 752). However, in other examples, the focal length of the third lens 750 may be different from the focal lengths of the first lens 708 and / or the second lens 710.

[0146] Therefore, a key difference between the optical system 600 and the optical system 700 according to this disclosure is that the optical system 700 according to this disclosure includes an additional lens 750 between the display device 704 and the waveguide 711.

[0147] Another important difference between optical system 600 and optical system 700 is that in optical system 700, the hologram displayed on the display device 704 is positioned such that a holographic reconstructed image 756 of the hologram is formed downstream of the display device when the display device 704 is illuminated with coherent light from a coherent light source such as a laser. This is a holographic reconstructed image 756 formed without the use of a physical lens between the display device 704 and the holographic reconstructed image 756. Instead, the hologram is calculated to form the holographic reconstructed image 756 at this position. In particular, the hologram is calculated / positioned such that the distance between the holographic reconstructed image 756 and the first lens 708 is less than the focal length f of the first lens 708, and the distance between the display device 704 and the first lens 708 is equal to the focal length f of the first lens 708.

[0148] The optical relay 706 is arranged to relay the hologram on the display device to form a relayed hologram 760 downstream of the second lens 710, and to form a relayed hologram reconstructed image 758 downstream of the relayed hologram 760. The relayed hologram 760 corresponds to the display device (including the hologram of the displayed image). The relayed hologram reconstructed image 758 corresponds to the hologram reconstructed image 756.

[0149] In this example, the relayed hologram reconstructed image 758 is formed such that the distance between the relayed hologram reconstructed image 758 and the third lens 750 is shorter than the focal length of the third lens 750, and the distance between the relayed hologram 760 and the third lens 750 is equal to the focal length of the third lens 750. By positioning the relayed hologram 760 and the relayed hologram reconstructed image 758 with respect to the third lens 750 in this way, the third lens 750 can form separated images of the relayed hologram and the relayed hologram reconstructed image. This will be explained in more detail with reference to Figure 8.

[0150] Figure 8 shows a schematic cross-sectional view of the third lens 750 and waveguide 711 (and relayed hologram 760 and relayed hologram reconstructed image 758) of Figure 7. These components are shown separately from other optical components of the optical system 700 (such as the display device 704 and optical relay 706). Figure 8 is a schematic ray diagram showing the rays from the relayed hologram 760 and the relayed hologram reconstructed image 758.

[0151] As those skilled in the art will understand, a (convex) lens (e.g., the third lens 750) forms a virtual image of an object at infinity when the object to be imaged is located at the focal length of the lens. As described above, the relayed hologram 760 is formed (by the optical relay 706) at the focal length f of the third lens 750 (specifically, the front focal plane 752 of the third lens 750). Thus, the third lens 750 is positioned to form a virtual image of the relayed hologram 760 at infinity. The virtual image at infinity is located upstream of the display device 704 / third lens 750. The formation of this virtual image is represented by rays of light that enter the third lens 750 from the relayed hologram 760 and then extend parallel to it. These rays are shown in Figure 8 as dashed lines alternating between dots and dashed lines.

[0152] Those skilled in the art will also understand that a (convex) lens (e.g., a third lens 750) can also form a virtual image of an object at a finite image distance upstream of the lens if the object to be imaged is positioned such that the distance between the lenses is shorter than the focal length of the lens. As described above, the relayed hologram-reproduced image 758 is formed (by the optical relay 706) such that the distance between the relayed hologram-reproduced image 758 and the third lens 750 is shorter than the focal length of the third lens 750. In other words, the relayed hologram-reproduced image 758 is positioned between the front focal plane 752 of the third lens 750 and the third lens 750 itself. By forming the relayed hologram-reproduced image 758, the third lens 750 is positioned to form a virtual image 800 of the relayed hologram-reproduced image 758 at a finite image distance upstream of the third lens 750. The formation of this virtual image 800 is represented by rays arriving at the third lens 750 from the relayed holographic image 758 and then converging upstream of the third lens 750. In Figure 8, these rays are shown as dashed lines containing only dots.

[0153] The virtual image of the relayed hologram 760 and the virtual image 800 of the relayed hologram reconstructed image 758 are both located upstream of the third lens 750. However, the virtual image distance of the virtual image of the relayed hologram 760 is infinite, while the virtual image distance of the virtual image 800 of the relayed hologram reconstructed image 758 is finite. Therefore, the two virtual images are far apart from each other (in fact, the distance between the two virtual images is virtually infinite). The artifact (described above) may be a feature of the virtual image of the relayed hologram 760. The appearance of the artifact may not be present or noticeable in the virtual image 800 of the relayed hologram reconstructed image 758. The inventors have found that by separating the two virtual images as described above, the prominence of the artifact in the field of view of the observation system can be significantly reduced, and in some cases completely eliminated. While not bound by theory, this is likely because the virtual image (including artifacts) of the relayed hologram 760 is far removed from the virtual image of the relayed hologram reconstructed image 758, and in this case, it is projected beyond the virtual image of the relayed hologram reconstructed image 758 to infinity. Therefore, the display system does not need to "see through" the virtual image of the relayed hologram 760 in order to display the virtual image of the relayed hologram reconstructed image 758.

[0154] Figure 9 is a schematic cross-sectional view of the optical components of a second optical system 900, in which the image of the hologram / display device is positioned far away from the virtual image of the hologram reconstruction, thereby reducing or eliminating the aforementioned artifact. The second optical system 900 is similar to the first optical system 700 in that the relayed hologram and the relayed hologram reconstruction image are positioned far apart from each other with respect to the third lens. However, in the second optical system 900, the image of the relayed hologram is a real image formed downstream of the waveguide (e.g., behind the observation system), rather than at infinity and upstream of the third lens. This point will be explained in more detail below.

[0155] Similar to the first optical system 700, the second optical system 900 is arranged such that the Fourier transform of the hologram displayed on the display device 904 is coupled to the waveguide 911 (not the hologram itself).

[0156] The optical system 900 includes an optical axis indicated by the dotted line 902 in Figure 9. In this example, the optical system 900 includes a display device 904, which is a liquid crystal on silicon spatial light modulator. The display device 904 is configured to display a hologram of an image. Downstream of the display device 904 is an optical relay 906. The optical relay 906 includes a first lens 908 and a second lens 910. The optical system 900 further includes a third lens 950.

[0157] The display device 904, optical relay 906, and third lens 950 of optical system 900 are very similar to the display device 704, optical relay 706, and third lens of first optical system 700. For example, the first lens 908 of optical relay 906 has a front focal plane 912 and a rear focal plane 914. The front focal plane 912 is upstream of the first lens 908, and the rear focal plane 914 is downstream of the first lens 908. The second lens 910 of optical relay 906 has a front focal plane 916 and a rear focal plane 918. The front focal plane 916 is upstream of the second lens 910, and the rear focal plane 918 is downstream of the second lens 910. The normals of the front and rear focal planes of the first lens 908 and the second lens 910, respectively, are parallel to the optical axis. 902 is such that the distance from the front and rear focal planes to the first or second lens is equal to the focal length f of the respective lens. In this example, the front focal plane 916 of the second lens 910 is substantially coplanar with the rear focal plane 914 of the first lens 908. In the example shown in Figure 9, the focal lengths f of the first lens 908 and the second lens 910 are the same. Thus, the optical relay forms a 4f system (i.e., the length of the optical relay is equal to four times the focal length f of either the first lens 908 or the second lens 910). However, in other embodiments, the focal length of the first lens 908 may be different from the focal length of the second lens 910. In such cases, the optical relay can form a magnifying (or reducing) telescope. Similar to the first optical system 700, the third lens 950 is a Fourier lens. The front focal plane 954 of the third lens 950 is located upstream of the third lens and is substantially coplanar with the back focal plane 918 of the second lens 910. The back focal plane 957 of the third lens 950 is located between the first and second surfaces of the waveguide 911. In this example, the focal length f of the third lens 950 is the same as the focal lengths of the first lens 908 and the second lens 910. Thus, the optical relay 906 and the third lens 950 together define a 6f system (the distance between the front focal plane 912 of the first lens 908 and the back focal plane 957 of the third lens 950 is equal to six times the focal length of the first / second or third lens 908, 910, or 952).However, in other examples, the focal length of the third lens 950 may be different from the focal length of the first lens 908 and / or the second lens 910.

[0158] The main difference between the first optical system 700 and the second optical system 900 is that in the second optical system 900, the display device 904 is not substantially positioned at the front focal plane 912 of the first lens 908 (as in the first optical system 700). Instead, the distance between the display device 904 and the first lens 908 is greater than the focal length f of the first lens 908. However, as with the first optical system 700, in the second optical system 900, the hologram displayed on the display device 904 is positioned such that a holographic reconstruction image 956 of the hologram image is formed downstream of the display device, and the distance between the holographic reconstruction image 956 and the first lens 908 is less than the focal length f of the first lens 908. Therefore, the distance between the display device 904 and the holographic reconstruction image 956 in the second optical system 900 is greater than the distance between the display device 704 and the holographic reconstruction image 756 in the second optical system 700.

[0159] The optical relay 906 is arranged to relay the hologram on the display device to form a relayed hologram 960 downstream of the second lens 910, and to form a relayed hologram reconstructed image 958 downstream of the relayed hologram 960. The relayed hologram 960 corresponds to the display device (including the hologram of the displayed image). The relayed hologram reconstructed image 958 corresponds to the hologram reconstructed image 956.

[0160] In this example, the relayed hologram reconstructed image 958 is formed such that the distance between the relayed hologram reconstructed image 958 and the third lens 950 is shorter than the focal length of the third lens 950, and the distance between the relayed hologram 960 and the third lens 950 is longer than the focal length of the third lens 950. By positioning the relayed hologram 960 and the relayed hologram reconstructed image 986 relative to the third lens 950 in this way, the third lens 950 can form separated images of the relayed hologram and the relayed hologram reconstructed image. This will be explained in more detail with reference to Figure 10.

[0161] Figure 10 shows a schematic cross-sectional view of the third lens 950 and waveguide 911 (and relayed hologram 960 and relayed hologram reconstructed image 958) of Figure 9. These components are shown separately from other optical components of the optical system 900 (such as the display device 904 and optical relay 906). Figure 10 is a schematic ray diagram showing the rays from the relayed hologram 960 and relayed hologram reconstructed image 958.

[0162] As those skilled in the art will understand, a (convex) lens (e.g., a third lens 958) forms a real image of an object when the object to be imaged is located beyond the focal length of the lens. This real image is formed at a finite image distance downstream of the lens. As described above, the relayed hologram 960 is formed (by the optical relay 906) beyond the focal length f of the third lens 950. In particular, the distance between the relayed hologram 960 and the third lens 950 is greater than the focal length f of the third lens. Therefore, the third lens 950 is positioned to form a real image 1002 of the relayed hologram 960 downstream of the third lens 950. The formation of this real image 1002 is represented by rays that enter the third lens 950 from the relayed hologram 960 and converge to a point downstream of the third lens 950 (and waveguide 911). These rays are shown in Figure 10 as dashed lines alternating between dots and dashed lines.

[0163] In both the first optical system 700 and the second optical system 900, the relayed hologram reconstructed image is formed (by optical relay), and the distance between the relayed hologram reconstructed image and the third lens is shorter than the focal plane of the third lens. Thus, as in the first optical system 700, in the second optical system 900, the third lens 950 is positioned upstream of the third lens 950 to form a virtual image 1000 of the relayed hologram reconstructed image 958 at a finite image distance. The formation of this virtual image 100 is represented by rays entering the third lens 950 from the relayed hologram reconstructed image 958 and converging to a point upstream of the third lens 950. These rays are shown in Figure 10 as dashed lines containing only points.

[0164] Therefore, the third lens 950 (and more generally the optical system 900) is positioned to form a virtual image of the hologram reconstructed image 958 relayed upstream of the third lens and a real image of the hologram 960 relayed downstream of the waveguide 911. In this way, the two images (virtual and real) are far apart from each other.

[0165] In this example, the real image of the relayed hologram 960 is located downstream of the observation window / eyebox (not shown in the figure, but located between the waveguide 911 and the real image of the relayed hologram 960). Therefore, as described above, the artifact is visible / obvious in the image of the relayed hologram 960 but not in the relayed hologram reconstructed image 958, and thus the prominence of the artifact in the field of view of the observation system can be greatly reduced or completely eliminated. In particular, since the image of the relayed hologram reconstructed image 958 is located in front of the observation system and the relayed hologram 960 is located behind the observation system, the observation system does not need to "see through" the image of the relayed hologram (including the artifact) when observing the virtual image of the hologram reconstructed image 958.

[0166] The first and second optical systems 700, 900 (as disclosed herein) described above each comprise optical relays 702, 902, respectively. In each example, the optical relays form the relayed holograms 760, 960 and the relayed holographic reconstructed images 758, 958 of the hologram. In each example, the third lenses 750, 950 form images of the relayed hologram and the relayed holographic reconstructed image. Some examples in this disclosure do not include optical relays. These examples comprise a (single) lens that forms images of the hologram / display device itself and the holographic reconstructed image itself, rather than relayed versions of the hologram and the holographic reconstructed image. However, the principle is substantially the same as described above, in that the hologram / display device and the holographic reconstructed image are positioned relative to a (single) lens, and the image of the hologram / display device is far removed from the image of the holographic reconstructed image, thereby reducing / eliminating the appearance / effect of the aforementioned artifacts.

[0167] Hello Artifact The optical systems 700 and 900 are arranged such that the distance between the display devices 704 and 904 and the holographic reconstructed images 756 and 956 is relatively short. This distance is indicated by feature 2000 in the figure. For example, this distance may be less than 20 millimeters. In particular, the holographic reconstructed images 756 and 956 can be formed in close proximity to the display devices 704 and 904 (if there are no optical components / lenses between the display device and the reconstruction). This is achieved by forming each image point of the holographic reconstructed image using a relatively small number of consecutive pixels in the display device. As a result, each sub-hologram uses a relatively small number of pixels. For example, a consecutive group of pixels accounting for less than 5% of the total number of pixels in the display device can form each sub-hologram / consecutive group of pixels. For example, each continuum group consists of pixels less than or equal to 100,000 pixels, optionally less than 25,000 pixels, optionally less than 5,000 pixels, optionally less than 1,000 pixels, optionally less than 500 pixels, optionally less than 200 pixels, and optionally less than 100 pixels.

[0168] The inventors found that, although a high-quality holographic image can be formed using a small, continuous group of pixels (as described above), forming the holographic image relatively close to the display device can result in noticeable additional artifacts / noise for the observer, compared to conventional optical systems (for example, forming the holographic image at least 1 meter away from the display device, where most / substantially all pixels of the display device contribute to each image point of the holographic image). The inventors revealed that these additional artifacts / noise are formed by light scattered by the display device. The inventors discovered that these additional artifacts / noise appear as a halo, which is referred to herein as the "halo effect." Scattering and the halo effect are shown in Figures 11, 12A, and 12B, respectively.

[0169] Figure 11 is a schematic cross-sectional view showing a portion of a pixelated display device 1104 (in this example, a liquid crystal on silicon spatial light modulator) arranged to form image points 1106 of a holographic reconstructed image 1156. The display device 1104 has a two-dimensional array of pixels 1102. However, since Figure 11 is a schematic cross-section along the normal of the display device, in Figure 11 the pixels 1102 extend along one dimension. The holographic reconstructed image 1156 in Figure 11 is formed in relatively close proximity to the display device 1103. The holographic reconstructed image 1156 comprises multiple image points 1106. The pixels 1102 in Figure 11 are illuminated by (coherent) light. Illumination of the pixels 1102 is not shown in Figure 11. However, the optical paths from some pixels 1102 (forming some image points 1106) are shown in Figure 11. In particular, Figure 11 shows how each image point 1106 of the hologram reconstructed image 1156 is formed by a continuous group of pixels 1102 of the display device 1104. Each pixel 1102 of the display device contributes to multiple image points 1106. For example, the first group 1110 of pixels 1102 contributes to the first image point 1111 of the reconstruction, and the second group 1112 of pixels 1102 contributes to the second image point 1113 of the reconstruction. However, as shown in Figure 11, most of the pixels 1102 of the first group 1110 are also present in the second group 1112.

[0170] Ideally, the light illuminating the display device is diffracted to contribute only to the intended image point 1106. However, the inventors found that in reality, some light is scattered by the pixels 1102 / display device 1104. This scattered light is indicated by the dashed arrow 1120 in Figure 11. The scattered light is not induced to form a specific image point 1106. Instead, the scattered light 1120 forms artifacts / noise in the holographic reconstructed image 1156. As described above with respect to the optical systems 700, 900 (for example), the holographic reconstructed image 1156 is replicated, imaged, and observable from the observation window. Therefore, artifacts / noise in the holographic reconstructed image 1156 are visible to the observation system using the optical system. Artifacts / noise can negatively impact the observation experience.

[0171] Figures 12A and 12B show holographic reconstructions that include artifacts / noise in the form of a halo (formed by scattered light 1120).

[0172] Figure 12A shows a first hologram reconstruction image 1256 formed by a display device displaying a first hologram of a first image (the display device / hologram is not shown in Figure 12A). The first image consists of a first diffraction region. Therefore, the hologram reconstruction image also includes a first diffraction region 1202. The first diffraction region 1202 is the hologram reconstruction image of the image content contained within the first diffraction region of the first hologram. In Figure 12A, the image content is schematically represented by a patterned filled region. If the hologram reconstruction image 1256 is a complete reconstruction image of the first image (consisting only of the first diffraction region) displayed on the display device, then the hologram reconstruction image 1256 consists only of the first diffraction region 1202. However, due to light scattering (shown in Figure 11), a noise region 1204 is formed around the first image content within the first diffraction region 1202. The noise region 1204 comprises a boundary 1206 having a shape corresponding to the shape of the first content within the first diffraction region 1202. In this example, since the first content within the first diffraction region 1202 is approximately square, the boundary of the noise region 1204 also has a corresponding square shape. The inventors found that the intensity of scattered light forming the noise is maximum at or near the boundary 1206 of the noise region 1204. Thus, the noise region 1204 appears as a halo 1208 (a shape corresponding to the shape of the first diffraction region 1202). The rest of the noise region 1204 (between the first diffraction region 1202 and the halo 1208) is filled with low-intensity scattered light. The inventors revealed that the noise takes on this shape as a result of scattered light by the pixels, the diffraction angle of the pixels, and the distance between the display device and the hologram-reproduced image. It is clear that the noise region 1204 is schematically depicted. In particular, the halo 1208 is exaggerated in Figure 12A.

[0173] Figure 12B shows a second hologram reconstructed image 1258 formed by a display device that displays a second hologram of the second image (the display device / hologram is not shown in Figure 12B). The second image is more complex than the first image. Specifically, the second image consists of multiple diffracted regions 1222 separated by non-diffracted regions 1224, each containing the second image content. The diffracted regions 1222 and non-diffracted regions 1224 form a checkerboard pattern. Each diffracted region 1222 forms a "halo," which are added together to form an inner halo 1225 and an outer halo 1226.

[0174] Hologram blurring to reduce the halo effect The inventors have developed a software solution for processing holograms to significantly reduce the halo effect. Details are described below. This software solution includes a process to smooth / blur the hologram by correcting / modifying the phase value of the hologram. More specifically, this solution includes a process to blur / soften the diffraction features / structure of the hologram, which is highly counterintuitive. A hologram processing method according to this disclosure will be described with reference to Figures 13-15.

[0175] Figure 13 schematically represents the arrangement of phase values ​​1302 of the hologram 1300. When the hologram is displayed on a display device (such as display devices 704, 904, etc.), the pixels of the display device are driven according to the phase values ​​of the hologram, i.e., the pixel values ​​1302. The hologram 1300 consists of a first region 1304 and a second region 1306. The first region 1304 contains (actually consists of) diffracted components. In other words, the phase values ​​1302 of the first region 1304 are spatially variable, and light incident on the first region 1304 (when the hologram is displayed on a display device) is diffracted. The spatially variable nature of the phase values ​​in the first region is represented by the seemingly random distribution of phase values ​​in the first region 1304. The second region 1306 of the hologram contains (actually consists of) non-diffracted components. In other words, the phase values ​​1302 in the second region 1304 do not change spatially, and the phase values ​​are substantially constant. In this example, each phase value 1302 in the second region 1304 is zero. The hologram 1300 is a hologram of an image calculated to form a holographic reconstruction of an image located relatively close to the display device when the hologram is displayed on the display device and properly illuminated as described above. The diffraction component of the first region 1304 spatially modulates the light incident thereon, forming the holographic reconstruction. The non-diffractive content of the second region 1306 is substantially an artifact resulting from the calculation that the hologram is designed to form a holographic reconstruction at a location very close to the device, as described above. Also, as described above, the presence of a first region (containing diffractive content) and a second region (containing non-diffractive content) within the hologram means that a relatively abrupt and large phase change occurs at the boundary between the first and second regions. The method relating to this disclosure reduces abrupt and large phase changes by smoothing or blurring the hologram in the boundary region including this boundary.

[0176] In the example, this method first involves detecting the boundary between the first region 1304 and the second region 1306. In the example, the boundary detection step first involves a) applying a binary filter to the hologram to output a binary hologram, b) determining / detecting the boundary in the binary hologram if the gradient or derivative of the phase value change in the binary hologram is non-zero, and c) applying the detected boundary to the (original) hologram 1300. The above steps a) and b) for detecting the boundary are shown in Figure 14.

[0177] Figure 14 shows a binary hologram 1400 calculated using the (original) hologram 1300 from Figure 13. The binary hologram 1400 in Figure 14 is calculated by applying a binary (or 2-value) filter to each phase value 1302 of the hologram 1300. The binary filter is configured to take each value 1302 of the hologram as input. The binary filter is configured to compare the input value to a predetermined value. In this example, the predetermined value is zero. If the input value is not equal to the predetermined value (i.e., not equal to zero), the binary function is configured to output a first value. If the input value is equal to the predetermined value (i.e., equal to zero), the binary function is configured to output a second value. In this example, the first value is 1 and the second value is 0. Therefore, the phase value of the first region 1404 of the binary hologram 1400 (corresponding to the first region 1304 of the first hologram 1300) is equal to 1, and the phase value of the second region 1406 of the binary hologram 1400 (corresponding to the second region 1306 of the first hologram 1300) is equal to 0.

[0178] The boundary 1410 between the first region 1404 and the second region 1406 is detected based on the change in adjacent phase values ​​1402 within the binary hologram 1400. For (adjacent) pixel pairs within the first region 1404, the change is zero (because both pixels in the pair are equal to 1). Similarly, for (adjacent) pixel pairs within the second region 1406, the change is zero (because both pixels in the pair are equal to 0). Only at the boundary between the first region 1404 and the second region 1406 (i.e., when a pixel pair consists of one pixel from the first region 1404 and one pixel from the second region 1406) is the change non-zero. Therefore, using the non-zero change, the position of the boundary 1410 between the first region 1404 and the second region 1406 within the binary hologram 1400 (represented by the thick black line in Figure 14) can be detected. The boundary 1410 has a corresponding position within the (original) hologram 1300. Therefore, the detected boundary 1410 (detected on the binary hologram 1400) can be applied to the original hologram 1300. This is shown in Figure 15, which illustrates how the detected boundary 1510 is superimposed / applied to the hologram 1300.

[0179] Figure 15 also shows a method of applying a hologram blur function to the boundary region 1502 of the hologram to blur / smooth the phase values ​​of the hologram within that boundary region 1502 and reduce the magnitude of the change in phase values ​​at the boundary 1510 between the first region 1304 and the second region 1306. In this example, the boundary region 1502 has a first end 1504 and a second end 1506 located opposite the first end 1504. The first end 1504 overlaps with the first region 1304 of the hologram. The second end 1506 overlaps with the second region 1306 of the hologram. The thickness of the boundary region 1502 is defined between the first end 1504 and the second end 1506. In this example, the thickness is 3 times the phase value 1302 / pixel value.

[0180] In this example, the hologram blur function is a box blur filter with a 3x3 kernel. In other words, when the hologram blur function is applied to the phase values ​​(of boundary region 1502), it is configured to calculate the average of the phase values ​​in a 3x3 array. The 3x3 array of phase values ​​includes the phase value to which the hologram blur function is applied, and the eight adjacent phase values. In this example, the 3x3 array of phase values ​​is centered on the phase value to which the hologram blur function is applied (i.e., the input phase value). Therefore, the remaining eight phase values ​​surround the input phase value. The hologram blur function is configured to output the average phase value, and in the modified hologram, the input phase value is replaced by the average phase value. Figure 15 shows an example of applying a box blur filter to three rows of phase values ​​in boundary region 1502. The three rows of phase values ​​consist of the values ​​π, 2π, and 0. Boundary 1510 is between the phase values ​​2π and 0. Therefore, in the original, i.e., unprocessed hologram, this row contains a sudden phase jump of 2π at boundary 1510. The box blur filter is applied sequentially to each phase value in the row. This is represented by 3x3 arrays 1512-1516, respectively, where array 1512 is a 3x3 array of phase values ​​when the box blur filter is applied to phase value 1522(π), array 1514 is a 3x3 array of phase values ​​when the box blur filter is applied to phase value 1524(2π), and array 1516 is a 3x3 array of phase values ​​when the box blur filter is applied to phase value 1526(0). In all cases, the box blur filter outputs the average phase value. Roughly speaking, these output values ​​are 4 / 3π, π, and 0.5π, respectively. These output values ​​replace the respective input values ​​in the modified hologram. The phase changes between adjacent output phase values ​​in the three rows are smaller than in the case of three input values. This process is repeated for each pixel within boundary region 1502. The output is a modified hologram in which each pixel within the boundary region 1502 is replaced with an average value calculated using the box blur function.

[0181] The inventors have advantageously discovered that processing holograms according to the described method significantly reduces the aforementioned so-called halo effect.

[0182] It is clear that the above examples are for illustrative purposes only and are not limiting. In particular, Figures 13-15 show 64-pixel holograms. However, it is clear that this method can be similarly applied to holograms of any size (holograms are usually much larger, e.g., 512 x 512 pixels). Similarly, Figure 15 shows a hologram blur function including a boundary region 1502 with a thickness of only 3 pixels and a 3 x 3 kernel, but in most actual examples, the boundary region 1502 and kernel will be much larger. For example, the inventors have found that combining a boundary region with a thickness of about 50 pixels and a 50 x 50 kernel provides a good balance, significantly reducing the halo effect without excessively altering the phase value of the hologram and without significantly negatively impacting the image quality of the reconstructed image. Furthermore, the examples shown in Figures 13-15 are simple examples, with a single linear boundary between a single first region 1304 and a second region 1306. However, it is clear to those skilled in the art that the same method can be applied to more complex hologram patterns. For example, the hologram used in the holographic reconstruction image of Figure 12B consists of multiple distinct "first" regions, each containing a different diffraction component, and each region is tangent to one or more "second" regions containing non-diffraction components. The boundaries between these regions are complex (i.e., not a single straight line), and multiple boundaries may exist. However, these complex boundaries can be identified as in Figure 15, and a hologram blur function can be applied accordingly. Furthermore, in the above example, boundary detection involves forming a binary hologram using a binary filter. However, those skilled in the art will understand that other methods for detecting or determining boundaries are also available. For example, boundaries within the (original) hologram 1300 can be detected using algorithms such as those used in image processing for edge detection.

[0183] The above describes a method for hologram processing, and it will be apparent to those skilled in the art that we can provide an optical engine configured to perform this method.

[0184] Additional features The methods and processes described herein can be incorporated into a computer-readable medium. The term “computer-readable medium” includes media configured to store data temporarily or permanently, such as random access memory (RAM), read-only memory (ROM), buffer memory, flash memory, and cache memory. The term “computer-readable medium” is also interpreted to include any medium, or combination of media, on which instructions for execution by a machine can be stored. These instructions, when executed by one or more processors, cause the machine to execute one or more of the methods described herein, either in whole or in part.

[0185] The term “computer-readable medium” also includes cloud-based storage systems. The term “computer-readable medium” includes, but is not limited to, one or more tangible, non-temporary data repositories (such as data volumes), including solid-state memory chips, optical discs, magnetic discs, or any suitable combination thereof. In some embodiments, execution instructions may be transmitted via carrier media. Examples of such carrier media include temporary media (such as propagating signals that transmit instructions).

[0186] It will be apparent to those skilled in the art that various modifications and changes can be made without departing from the scope of the appended claims. This disclosure covers all modifications and changes within the scope of the appended claims and their equivalents.

Claims

1. An optical engine comprising a display device having an array of pixels, wherein each pixel imparts a phase delay to the light incident thereon, and the optical engine, It is configured to receive an image hologram, wherein the hologram includes an array of pixel values, and the hologram includes a first region including a diffraction component and a second region including a non-diffraction component, and the first region is adjacent to the second region. The system is configured to apply a hologram blur function to the hologram in the boundary region between the first region and the second region to output a corrected hologram. An optical engine in which the hologram blur function is configured to modify the gradient of the pixel values ​​across the boundary region.

2. The optical engine according to claim 1, wherein the hologram includes a plurality of pairs of adjacent pixel values, each pair of pixel values ​​includes a first pixel value in the first region and a second pixel in the second region, and for each pair of pixels, the hologram blur function is configured to reduce the difference between the first pixel value and the second pixel value of the corresponding pair of pixels.

3. The optical engine according to claim 1 or 2, wherein the boundary region includes a first end located within the first region and a second end located within the second region opposite to the first end, and the hologram blur function is configured such that the change in pixel values ​​associated with the boundary of the modified hologram is greater than the change in pixel values ​​associated with the corresponding boundary of the original hologram.

4. The optical engine according to claim 3, wherein the thickness of the boundary region is defined as the distance between the first end and the second end, and the thickness of the boundary region is 20% or less of the width of the display device.

5. The optical engine according to claim 4, wherein the thickness of the boundary region is 5 pixels or more, arbitrarily 10 pixels or more, and arbitrarily 50 pixels or more.

6. The optical engine according to claim 4 or 5, wherein the thickness of the boundary region is 1,000 pixels or less, arbitrarily 500 pixels or less, and arbitrarily 200 pixels or less.

7. An optical engine according to any one of the preceding claims, configured to detect the boundary between the first region and the second region.

8. The optical engine according to claim 7, configured to detect the boundary based on the pixel values ​​of the hologram.

9. The optical engine according to claim 7 or 8, configured to apply a binary filter to the hologram before detecting the boundary and output a binary hologram.

10. The optical engine according to claim 9, configured to detect the boundary when the change or derivative of the pixel value of the binary hologram exceeds a threshold.

11. The optical engine according to any one of the prior claims, wherein the application of the hologram blur function includes applying the function to each pixel value in the boundary region.

12. The optical engine according to any one of the preceding claims, wherein the hologram blur function is configured to output the average value of adjacent pixel values ​​for each pixel value to which the hologram blur function is applied.

13. The optical engine according to claim 12, wherein the boundary region of the modified hologram includes an average pixel value calculated using the hologram blur function.

14. The optical engine according to any one of the preceding claims, wherein the hologram blur function includes a kernel such as a two-dimensional kernel.

15. The optical engine according to claim 14, wherein the width and / or depth of the kernel is substantially equal to the thickness of the boundary region.

16. The optical engine according to any one of the prior claims, wherein the hologram is configured to generate a holographic wavefront such that the display device spatially modulates light according to the hologram to generate a holographic reconstructed image including a plurality of image points.

17. The optical engine according to claim 16, wherein the hologram is configured such that each image point of the hologram reconstructed image is generated using a continuous group of pixels of the display device.

18. The optical engine according to claim 17, wherein each consecutive group of pixels includes less than 100,000 pixels, and optionally less than 25,000 pixels.

19. The optical engine according to claim 17 or 18, wherein each consecutive group of pixels includes a number of pixels less than 5% of the total number of pixels of the display device.

20. A method for processing holograms, A step of receiving an image hologram, wherein the hologram includes an array of pixel values ​​and includes a first region including a diffraction component and a second region including a non-diffraction component, and the first region is adjacent to the second region. The step includes applying a hologram blur function to the hologram in the boundary region between the first region and the second region to output a corrected hologram, A method wherein the hologram blur function is configured to change the gradient of the pixel values ​​in the boundary region.

21. A diffraction structure comprising an array of pixel values, wherein the diffraction structure is configured to spatially modulate light incident thereon to generate a holographic wavefront, and the holographic wavefront is configured to generate a holographic reconstruction of the image encoded in the diffraction structure. The diffraction structure includes a first region containing a diffraction component, a second region containing a non-diffraction component, and a blurred region between the first and second regions. The blurred region includes a first end portion located in the first region and a second end portion located in the second region, opposite to the first end portion. The diffraction structure is configured such that the pixel values ​​within the blurred region gradually change with distance from the second end.

Images