Hologram calculation method

Abstract

A hologram engine is provided for calculating a hologram of a target image displayed on a display device of an optical system. The target picture comprises a plurality of image points. Each image point is defined by a ray path from it to an entrance pupil of an observer. Each light path forms an angle with the display device. The hologram engine is arranged to determine a first sub-hologram of a first subset of image points of the target picture by: for each image point of the first subset: a) propagating a wave along a propagation path forming an angle with the display device that is reduced relative to an angle formed by the respective light path; and b) determining the contribution of the wave at the display device for each image point of the first subset.

Description

Technical Field

[0001] The present disclosure relates to a hologram engine and corresponding method for computing a hologram. More specifically, the present disclosure relates to a hologram engine for computing a hologram using a modified point cloud method. Even more specifically, the present disclosure relates to a hologram engine for computing a hologram by reducing the angle of a propagation path along which a wave propagates. Some embodiments relate to a holographic projector, a picture generation unit, or a head-up display. Background Art

[0002] Light scattered from an object contains both amplitude and phase information. This amplitude and phase information can be captured by well-known interference techniques, for example on a photosensitive plate, to form a holographic recording or "hologram" comprising interference fringes. The hologram can be reconstructed by illumination with appropriate light to form a two-dimensional or three-dimensional holographic reconstruction or replay image representing the original object.

[0003] Computer generated holography can numerically simulate interference processes. Computer generated holograms can be calculated by techniques based on mathematical transformations such as Fresnel or Fourier transforms. These types of holograms can be called Fresnel / Fourier transform holograms or simply Fresnel / Fourier holograms. Fourier holograms can be considered as Fourier domain / plane representations of objects or frequency domain / plane representations of objects. Computer generated holograms can also be calculated, for example, by coherent ray tracing or point cloud techniques.

[0004] The computer generated hologram may be encoded on a spatial light modulator arranged to modulate the amplitude and / or phase of the incident light. For example, the light modulation may be achieved using electrically addressable liquid crystals, optically addressable liquid crystals or micromirrors.

[0005] A spatial light modulator typically includes a plurality of individually addressable pixels, which may also be referred to as cells or elements. The light modulation scheme may be binary, multi-level, or continuous. Alternatively, the device may be continuous (i.e., not including pixels), so that the light modulation may be continuous across the device. A spatial light modulator may be reflective, meaning that the modulated light is output in a reflected manner. A spatial light modulator may also be transmissive, meaning that the modulated light is output in a transmissive manner.

[0006] The system described herein can be used to provide a holographic projector. Such projectors have been used in head-up displays (HUDs). Summary of the invention

[0007] Aspects of the present disclosure are defined in the accompanying independent claims.

[0008] In general, a hologram engine is provided for computing a hologram of a target picture in a counter-intuitive but computationally efficient manner.

[0009] A hologram engine according to the present disclosure is arranged to calculate a hologram of a target picture including a plurality of image points. Thus, the term "hologram engine" refers to a system arranged to determine a hologram of an image, for example, a computing system or a hologram calculator or a data processor or an image processor. The system may include a memory arranged to store a hologram and a device for retrieving a hologram based on an image. The system may include a computing chip or a processor arranged to calculate a hologram from an image, for example, using an algorithm and / or a logic circuit. The input of the hologram engine is an image, and the output is a hologram of the image. The "hologram engine" may be embodied as a field programmable gate array or an application specific integrated circuit. The hologram engine (more specifically, its algorithm or logic circuit) is arranged to virtually / computationally propagate a wave (also referred to herein as a "wavelet") associated with each image point along a propagation path and then determine the contribution of (one or more) propagated waves at a display device to do so. Based on the contribution determined / simulated for each image point at the display device, a hologram may be calculated. For example, an array of complex amplitude values ​​of a wave at the intersection of the wave and the display device can be determined. From this, a hologram can be calculated which, when properly illuminated, spatially modulates the light incident thereon to form a holographic wavefront that can reconstruct the or each image point. When calculating a pure phase hologram, an array of phase values ​​of the pixels of the display device can be determined. The hologram engine according to the present disclosure can be described as a point cloud method for hologram calculation. Point cloud hologram calculation methods are known. However, the hologram engine according to the present disclosure is arranged to calculate holograms in a quite unconventional and counterintuitive manner. In particular, the inventors have developed a hologram calculation method in which the corresponding propagation path of each image point in at least a subset of the image points of the target image is modified (e.g., reduced).

[0010] Conventionally, point cloud hologram computation may include virtually propagating a wave from each image point to the center of the entrance pupil of the observer (in the viewing window of the optical system including the display device). In other words, typically, the wave may propagate along a ray path defined between the image point and the center of the entrance pupil. The hologram computation is then based on the characteristics (contributions) of the wave propagated along this ray path at the display device. The propagated wave will intersect the display device at an angle that depends on the position of the center of the entrance pupil and the corresponding image point. When the computed hologram is properly illuminated, a holographic wavefront will be formed that propagates in a manner that reconstructs the simulated / propagated wave so that the corresponding image point is holographically reconstructed to appear at the expected position (defined in the original target picture). In other words, conventionally, if the propagated wave intersects the display device at a specific angle, assuming that the angle falls within the field of view of the display device, the holographically reconstructed point will appear at a position in the field of view of the system including the display device that depends on the angle. The field of view is typically determined by (at least) a) the wavelength of the light used to illuminate the display device; and b) the pixel pitch of the display device (i.e., the distance between the centers of adjacent pixels). The smaller the pitch, the larger the field of view. The display device may only be able to holographically reconstruct image points within the angular range of the field of view.

[0011] After thorough simulation and experimentation, the inventors have found that significant advantages, particularly computational efficiency, can be achieved by modifying the hologram calculation method so that each wave propagates along a propagation path that has been modified or selected to form a reduced angle with the display device. In other words, the angle formed by the propagation path and the display device can be smaller than the angle formed by the light path and the display device. For example, each propagation path can be selected to pass through the display device substantially orthogonally (i.e., substantially perpendicularly or substantially parallel to the normal of the display device). The modification can effectively cause the waves to be propagated as if their associated image points have been translated or otherwise displaced or moved from one or more positions originally defined in the target image. For example, the propagation path may have been modified as if one or more image points have been translated or otherwise displaced or moved closer to the center of the target image (or closer to the field of view of the system). In some examples, the propagation path may have been modified as if one or more image points have been translated or otherwise displaced or moved to substantially at the center of the target image (or substantially at the field of view of the system). This can be the case when each propagation axis has been modified to be substantially orthogonal to the display device. Another way to describe this effect is to say that the starting point of the (modified) propagation path is no longer the corresponding image point in the target picture (as is conventionally the case). Without further processing steps, a hologram calculated using the described modified propagation path may result in a holographic reconstruction of the target picture that appears as if the image points were located essentially in the center of the field of view, possibly overlapping each other. This may not be an accurate reconstruction of the target picture, which may include image points distributed over an angular range of the field of view (not simply clustered together). In other words, in the modified calculation method according to the present disclosure, a large amount of position information (which is usually encoded in the hologram calculation at the wave propagation stage) is lost. In view of this, this approach is very counterintuitive.

[0012] However, the inventors have recognized that it is particularly advantageous for one or more waves of the hologram calculation method to propagate along a propagation path that forms a reduced angle with the display device (e.g., substantially orthogonal to the display device). The inventors have found that when the propagation path is modified in this manner, the hologram calculation can be made significantly more efficient. For example, the inventors have recognized that the spatial frequency of the hologram of an image point toward the edge of the field of view of a system including the display device is higher than the spatial frequency of the hologram of the image point toward the center of the field of view. Typically, the spatial frequency of the hologram of an image point decreases as the image point approaches the center of the field of view (so that the angle formed by the path from the display device to the image point is relatively small, approaching zero). Spatial frequency can refer to the frequency of the complex amplitude at the display device. In a pure phase hologram, a hologram of a point with a relatively high spatial frequency can have a rapidly changing phase. For example, the phase of adjacent pixels of a display device displaying the hologram can change close to pi. A hologram of a point with a relatively low spatial frequency can have a relatively slowly changing phase. For example, the phase of adjacent pixels of a display device displaying the hologram can change a relatively small portion of pi. The inventors have recognized that by computing holograms using a modified propagation path for image points as described above, the spatial frequency of the holograms for those image points can be significantly reduced. Importantly, the inventors have recognized that this enables the initial hologram calculation to be performed at a lower resolution. For example, the complex amplitude / phase at a display device may be sampled for a portion of the pixels of the display device. The inventors have recognized that when the spatial frequency is relatively low, interpolation may be used to fill in the pixel values ​​between the sampled pixels. The inventors have recognized that when the spatial frequency is relatively high, undersampling and interpolation are not possible because the phase / complex amplitude values ​​at the display device may change so quickly that interpolation may be (very) inaccurate and result in (significant) aliasing problems. The modified propagation path described above implements an undersampling and interpolation approach.

[0013] The inventors have found that interpolation is computationally cheaper than the initial hologram calculation, so that the undersampling significantly increases the computational speed and efficiency of the hologram calculation. For example, the inventors have achieved a five-fold increase in computational efficiency compared to conventional point cloud techniques. Memory requirements are also significantly reduced.

[0014] As will be appreciated by those skilled in the art, it is often desirable to reduce the computational cost (or improve computational efficiency) of hologram computation. This is particularly important in order to achieve real-time hologram computation with good quality image resolution. In the context of holographic projectors, such as heads-up displays used in vehicles, it is particularly advantageous to be able to achieve such real-time hologram computation with as little computational power as possible. The hologram computation method according to the present disclosure facilitates the achievement of such real-time holography.

[0015] As described above, the modified propagation path has the effect of computing the hologram as if the image points had been translated. Therefore, further processing may be required after the initial hologram calculation in order to return the image points to their correct positions (i.e., further processing after the propagation and contribution determination steps). The inventors have recognized that such further processing can be quickly and simply achieved using an appropriate phase ramp / gradient function that is arranged to compensate for the position changes of the image points caused by the modified propagation path. The use of a phase ramp or gradient function to translate image points (by applying an angular "kick" to light incident on a display device displaying the function) is familiar to those skilled in the art. In some embodiments, two phase ramps may be used. For example, a first phase ramp function may be used to compensate for propagation path modifications in a first dimension (e.g., the x dimension), and a second phase ramp function may be used to compensate for propagation path modifications in a second dimension (e.g., the y direction). In addition, the inventors have recognized that a subset of image points may be treated as a group in the hologram calculation. The propagation paths of the image points within the subset may be modified by the same angle (e.g., in both x and y). Each of the modified propagation paths can be used to calculate a sub-hologram for the image points within the subset. One or more phase ramp functions can then be applied to the sub-hologram. Thus, the one or more phase ramp functions can have the effect of compensating for the modified propagation paths for all image points within the subset at once. (Rather than applying a different phase ramp function to each image point individually). The hologram calculation method can iterate through multiple subsets of sub-holograms and repeat the process using the appropriate phase ramp functions so that the reconstruction is effectively stitched together using the phase ramp functions. In other words, different phase ramp functions can be applied to different subsets of image points, but within a subset of image points, the same one or more phase ramp functions can be applied.

[0016] While removing the position / angle information of a point in the hologram calculation and compensating for it using a phase ramp function (pushing the point back to its original position) may seem unconventional and complex, the inventors have discovered that the increased computational efficiency of the hologram calculation achieved when modifying the propagation path to form a reduced angle with the display device significantly outweighs the obvious increase in complexity.

[0017] According to a first aspect, a hologram engine is provided. The hologram engine is used to calculate a hologram of a target picture for display on a display device of an optical system. The target picture includes a plurality of image points. Each image point is defined or definable by a first path. The first path or light path of each image point extends from the image point to an entrance pupil of an observer (e.g., to the center of the entrance pupil of the observer). Each first path or light path can be described as being ray traceable between the corresponding image point and the entrance pupil. Each first path or light path can be described as connecting or linking the corresponding image point and the entrance pupil. Each first path or light path (of each image point) is at an angle to the display device of the optical system. In other words, the first path or light path can correspond to the propagation path (or second path) of the conventional point cloud hologram calculation method described above. However, the hologram engine does not propagate waves along the light path. In contrast, the hologram engine is arranged to determine a first sub-hologram of a first subset of image points of the target picture by: for each image point of the first subset: a) propagating a wave along a second path or propagation path, the angle formed by the second path or propagation path with the display device being reduced relative to the angle formed by the corresponding first path or light path; and b) determining, for each image point of the first subset, the contribution of the wave (which characterizes the image point) at the display device. The reduced angle formed by the second path or propagation path with the display device means that the angle formed by the second path or propagation path with the display device is smaller than the angle formed by the first path or light path with the display device. As described above, propagating the wave (for each image point of the first subset) along a propagation path that forms a reduced angle with the display device (rather than a light path to the entrance pupil) allows for a more computationally efficient hologram calculation. This is because the spatial frequency of the calculated hologram may be relatively low. This allows, for example, the hologram to be initially undersampled (e.g., calculated at a resolution lower than the native resolution of the display device). Interpolation may be used to achieve a full-resolution hologram.

[0018] The hologram engine may be arranged to determine or form a hologram of a target picture. The determined or formed hologram may include a first sub-hologram. As described in more detail below, the hologram engine may be arranged to combine additional sub-holograms with the first sub-hologram as part of determining or forming the hologram. The hologram engine may be arranged to output, display or store a hologram of a target picture.

[0019] The hologram engine may be arranged to display the hologram on a display device, such as a spatial light modulator. In some embodiments, the hologram engine may include the display device. In some embodiments, the display device may be external to the hologram engine. The hologram is arranged such that when the hologram is displayed on the display device and is appropriately illuminated, a first subset of image points of the target picture may be holographically reconstructed.

[0020] The hologram engine may be arranged to store the (determined or formed) hologram in a memory. In some embodiments, the hologram engine may include the memory. In some embodiments, the memory may be external to the hologram engine.

[0021] Throughout this disclosure, the first path may be referred to as a light path, and vice versa. These terms may be used interchangeably. Throughout this disclosure, the second path may be referred to as a propagation path, and vice versa. These terms may be used interchangeably.

[0022] As used herein, a description of a first path or light path or propagation path that is "angled" with a display device refers to an angle defined between a normal to the display device and the light path or propagation path. In some embodiments, the display device may extend in a first dimension and a second dimension. The first dimension may be perpendicular to the second dimension. For example, the display device may be substantially quadrilateral (e.g., rectangular or square). A first edge of the display device may extend in the first dimension. A second edge of the display device may extend in the second dimension. In some embodiments, the angle formed between the light path or propagation path and the display device may have a component in at least one of the first dimension or the second dimension. In some embodiments, the angle formed between the light path and the propagation path may have a component in both the first dimension and the second dimension.

[0023] In some embodiments, the hologram engine is arranged to apply one or more grating functions (also referred to herein as phase ramp functions) to the sub-hologram. As will be understood by those skilled in the art, a "grating function" or "phase ramp function" is a function that provides a linear translation of light at the exit pupil of an optical system including a display device that displays the function. That is, a grating or phase ramp function comprises an array of light modulation values ​​that, when added to a hologram, linearly shifts the replay field by a defined amplitude and direction. The displacement may be measured in pixels, millimeters, or degrees. The phase ramp may also be referred to as a phase wedge. The phase values ​​of the phase ramp may be warped (e.g., modulo 2π). The warped phase ramp may be considered a phase grating. However, the present disclosure is not limited to phase modulation only, and the terms "grating function", "software grating", and "blazed grating" may be used as examples of beam steering functions such as a warped modulation ramp. The phase ramp may be characterized by its phase gradient.

[0024] One or more grating functions may be arranged to compensate for the reduction in the angle of the propagation path of the image points of the first subset. One or more grating functions may be arranged to reverse the reduction in the angle of the propagation path of the image points of the first subset. This may cause a plurality of image points of the first subset to be reconstructed in positions within the field of view of the optical system corresponding to corresponding positions within the target image when the sub-hologram combined with the one or more grating functions is properly illuminated. For example, if the angle of the propagation axis decreases by 3 degrees in the first dimension during the calculation of the sub-hologram, a grating function may be selected to increase the angle by 3 degrees in the first dimension to compensate. In other words, one or more grating functions may be selected to return the image points of the first subset to their intended positions (as defined in the target image). In some embodiments, a first grating function may be arranged to compensate for the reduction in angle in the first dimension. A second grating function may be arranged to compensate for the reduction in angle in the second dimension. One or more grating functions may be added to the first sub-hologram.

[0025] In some embodiments, the angle formed by each propagation path (i.e., the propagation path associated with the image points of the first subset) and the display device is reduced by substantially the same amount. In other words, the angle of each propagation path may be reduced by the same amount relative to the corresponding angle formed by the first path or light path of the image point and the display device. Because a different first path or light path may be associated with each image point of the first subset, and because each first path or light path within the first subset may form a (slightly) different angle relative to the display device, the angle of each propagation axis within the subset may also be (slightly) different. By reducing the angle formed by each propagation axis of the first subset by the same amount, a grating function (or multiple grating functions) may be applied to the sub-hologram in a single step to have the effect of moving / translating all image points of the first subset to their intended positions.

[0026] In some embodiments, the propagation paths of at least some (optionally, all) of the image points of the first subset do not pass through the center of the entrance pupil. As described above, the ray paths depend on the positions of the entrance pupil and the image points. However, the propagation paths along which the waves associated with the first subset of image points propagate have a reduced angle. Effectively, this moves the imaginary starting point of the propagation path. It also has the effect of moving the imaginary end point of the propagation path so that it may no longer pass through the center of the entrance pupil. Similarly, each propagation path associated with the first subset of image points can be arranged so that the corresponding wave does not pass through the corresponding image point.

[0027] In some embodiments, each propagation path of the image points of the first subset passes through a first portion of the target picture. In other words, the first portion of the target picture can be defined as a portion containing the image points of the first subset. The distance between the first portion and the center of the target picture can be less than the distance between the image points of the first subset and the center, optionally less than the distance between any image point of the first subset and the center. In other words, the effect of reducing the angle formed by each propagation axis with the display device can be to move the image point closer to the center of the picture. Therefore, in some embodiments, each propagation path is arranged as if the corresponding image point has been moved within the target picture to be closer to the center of the target picture.

[0028] In some embodiments, the reduced angle of at least one propagation path is substantially equal to zero. As described above, the starting angle of each light path may be (slightly) different. Thus, typically not all reduced propagation paths form an angle of exactly zero. It may be only one propagation path (associated with the first point) of the first subset of propagation paths that is exactly equal to zero. For example, the propagation path associated with the image point at the center of the first subset of image points may be reduced to be exactly equal to zero. Other image points of the first subset of points may be relatively close to the first image point, so that the propagation path associated with each first image point may be described as being substantially equal to zero.

[0029] In some embodiments, the hologram engine is arranged to initially undersample the first sub-hologram. Undersampling the first sub-hologram may mean determining the values ​​of the pixels of the hologram at a resolution that is less than the native resolution of the display device. This may mean that the values ​​of the first sub-hologram are initially determined for a subset of the pixels of the display device. There may be one or more undetermined pixel values ​​between each determined pixel value. Such undersampling may be a step in determining the contribution of the wave at the display device. As described above, undersampling the hologram may advantageously be computationally efficient. This may be because determining each pixel value of the hologram at this stage is computationally expensive. By undersampling, the number of pixel values ​​that need to be determined may be significantly reduced, thereby reducing the need for computation. The hologram engine may be arranged to interpolate the undersampled first sub-hologram in at least a first dimension of the display device to determine the values ​​of at least some of the undetermined values ​​of the first sub-hologram. In other words, interpolation may fill in the gaps between the determined values ​​of the first sub-hologram. The inventors have recognized that it may be computationally more efficient to calculate these "missing" values ​​using interpolation than by determining the contribution of the wave for all pixel values ​​(i.e., at full resolution). However, the inventors have also recognized that this interpolation method may only be feasible because the wave at the display device for all image points has a low spatial frequency (because the angle of the propagation path is reduced). Therefore, the values ​​on the display device change relatively slowly. Therefore, it is acceptable to initially undersample the pixel values. If the values ​​change relatively quickly, the undersampling and interpolation methods may cause significant aliasing problems.

[0030] In some embodiments, the hologram engine is arranged to divide the target image into subsets of multiple image points. Each subset of image points may include multiple adjacent image points. Each subset of image points may be defined by a discrete angle range. An angle range may be defined as a discrete range of angles from a display device. A subset of image points may be defined by adjacent discrete angle ranges. Each angle range may be substantially equal to the other angle ranges. In other words, the hologram engine may be arranged to divide the target image into multiple subsets of image points by dividing the target image into multiple equal-sized angle ranges. The hologram engine may be arranged to identify which image points are included in each discrete angle range. Multiple subsets of image points may not overlap. Therefore, each image point may be only part of a single subset. In some embodiments, the hologram engine is arranged to divide the target image into at least n subsets of image points. n may be greater than or equal to 2, optionally greater than or equal to 5, optionally greater than or equal to 10, optionally greater than or equal to 50. The n subsets of image points may include a first subset.

[0031] For each of the n (or more) subsets of image points, the hologram engine may be arranged to perform corresponding calculations as described above with respect to the first subset. For example, the hologram engine may be arranged to determine the nth sub-hologram by determining the contribution of the wave or wavelet at the display device for each image point of the nth (or corresponding) subset. Each wave (for each image point of the corresponding subset) may propagate along a propagation path. Each propagation path may be different from the corresponding light path of the corresponding image point. Each propagation path may form a reduced angle with the normal of the display device. The advantages described with respect to the increased computational efficiency of computing the first sub-hologram apply to the computation of each of the n sub-holograms (having a reduced propagation path having a reduced angle).

[0032] In some embodiments, the angle formed by the propagation path associated with each image point within the nth subset and the display device is reduced by substantially the same amount. This corresponds to what was described above with respect to the first subset. In some embodiments, the reduced angle of at least one propagation path associated with the image points in each of the n subsets is substantially equal to zero. Likewise, the starting angle of each light path may be (slightly) different. Thus, in general, not all reduced angles formed by the propagation paths within a subset may be angles that are exactly zero. It may be that only one of the propagation paths in the nth subset (associated with the first point) is exactly equal to zero. For example, the propagation path associated with the image point at the center of each of the n subsets of image points may be reduced to be exactly equal to zero. However, other image points of the nth subset of image points may be relatively close to the first image point, so that the propagation path associated with each first image point may be described as being substantially equal to zero.

[0033] In some embodiments, the hologram engine is arranged to apply one or more grating (or phase ramp) functions to each of the n sub-holograms, the one or more grating functions being arranged to compensate for the reduced angle of the propagation path of the nth subset. In some embodiments, the amount of angular reduction formed by the propagation path is different for different subsets.

[0034] The hologram engine may be arranged to output a hologram of a target picture by combining each of the n sub-holograms (for each of the n subsets of image points). The step of outputting a hologram may also include combining each grating function (applied to each of the n sub-holograms) with the combined n sub-holograms. The step of combining each of the n sub-holograms and / or each of the grating functions may include superimposing the sub-holograms and / or grating functions, for example by summing the corresponding phase values ​​at each pixel of the hologram.

[0035] As described above, the hologram engine may be arranged to divide the target image into at least n subsets, calculate a sub-hologram for each of the n subsets of image points using a modified propagation path (forming a reduced angle with the display device), and compensate for the modified propagation path by applying one or more gradient functions. The inventors have recognized that the target image may be divided into subsets of an optimized number of image points. In other words, the target image may be divided into an optimized number of angle ranges. For example, grouping the image points into subsets is effective because the same reduction of the angle of the propagation axis can be applied to all image points, and the same one or more grating functions can be applied to compensate. If the subset is too small (so that there are a relatively large number of subsets, each of which includes a relatively small number of image points), the reduced angle and one or more grating functions need to be recalculated frequently. However, if the subset is too large (so that there are a relatively small number of subsets, each of which includes a relatively large number of image points), the memory requirements required to accumulate the contributions of all image points become larger. In addition, the larger the subset, the greater the deviation of the angles from zero produced by some propagation paths (once the angle has been reduced). This may reduce the ability to downsample these points, thereby reducing the computational efficiency gain. The exact number of subsets that may be considered optimal will depend on the particular system (e.g., the size of the field of view, the resolution of the target picture, the size and type of memory used in the system, and the available processing power). However, the inventors have found that the number of subsets of image points (n) can generally be much greater than 2, much greater than 5, much greater than 10, or even much greater than 50. In one example, the inventors partitioned the target picture into an array of at least 10 by at least 10 subsets and achieved significant computational efficiency gains.

[0036] Some of the features described above with respect to extending the hologram calculation method to multiple subsets of image points can be summarized as follows. The hologram engine can be arranged to group the image points of the target image into multiple subsets of at least n subsets. Each subset may include a group of adjacent image points falling within a discrete angle range. In other words, the hologram engine can be arranged to divide the target image / field of view into multiple angle ranges. The hologram engine can be arranged to identify which image points fall within a specific angle range to identify at least n subsets. The hologram engine can be arranged to calculate a sub-hologram for each of the n subsets. For example, the hologram engine can be arranged to iteratively step through the n subsets in sequence and calculate the corresponding sub-holograms. The sub-hologram of the nth subset of image points can be referred to as the nth sub-hologram. For the calculation of the sub-hologram of each subset of the n subsets of image points, the hologram engine can be arranged to propagate waves along a propagation path at an angle to the display device, which angle is reduced relative to the angle formed by the corresponding light path of the image point. The angle formed by the propagation path of each image point within the nth subset may be reduced by the same amount (relative to the corresponding light path of the corresponding image point). Preferably, at least one propagation path associated with the image point within each subset may form an angle equal to zero with the display device. Preferably, the propagation path associated with each image point within each subset may form an angle substantially equal to zero with the display device. The hologram engine is further arranged to determine, for each image point within the nth subset, the contribution of the wave propagating along the (reduced angle) propagation path. Because the propagation path angle has been reduced, the hologram calculation may be computationally very efficient. For example, the sub-holograms of each subset may initially be undersampled (e.g. determined at a resolution lower than the native resolution of the display device). The hologram engine may be arranged to interpolate each undersampled sub-hologram until a higher resolution (e.g. until a resolution corresponding to the resolution of the display device or filling empty hologram pixels). In some embodiments, the hologram engine is arranged to compensate for the reduced angle used during the calculation of the sub-holograms by applying one or more compensation gradient functions to each sub-hologram. For example, two gradient functions may be applied to each sub-hologram, a first gradient function for compensating in a first dimension (e.g., the x-direction) and a second gradient function for compensating in a second direction (e.g., the y-direction). The gradient functions may be used to translate the reconstruction of each subset of image points in a manner that is opposite to the effective translation caused by the reduced angle of the propagation axis. The inventors have found that it is convenient to process subsets of image points into groups, rather than going through the process of determining a sub-hologram and applying one or more grating functions to each image point on a point-by-point basis. This is because one or more grating functions can be applied to a sub-hologram for multiple image points (i.e., all image points within a subset) in a single step.Thus, the applied one or more grating functions may effectively compensate for the reduced propagation angle for all image points within the subset in that single step. The hologram engine may be arranged to apply different gradient functions to different sub-holograms. This may be because different sub-holograms are sub-holograms having image points at different positions (associated with different angular ranges) in the target image. Thus, the angle of the propagation path used in the calculation of different sub-holograms may be reduced by different amounts, and therefore different gradient functions may be required to compensate for the reduction. The applied gradient function may be described as effectively stitching the target image back together (by pushing / translating each subset of image points back to the correct position in the field of view when reconstructing the hologram). While removing the angular information when calculating the sub-holograms may appear (on its surface) counterintuitive and complex, by simply reintroducing this angular information using a gradient function, the inventors have found that the computational efficiencies achieved by reducing the angle are significant and outweigh the significantly increased complexity.

[0037] In a second aspect of the present disclosure, a hologram engine for calculating a hologram for display on a display device of an optical system is provided. The hologram is a hologram of a target picture including a plurality of image points. The hologram engine is arranged to determine a first sub-hologram of a first plurality of image points of the target picture by determining the contribution of a wave (or wavelet) at the display device for each image point. The hologram engine is arranged so that the wave has followed a second propagation path from the corresponding image point to the display device. The second propagation path of each image point defines a second angle with the normal of the display device, which is different from (preferably, less than) the first angle formed by the corresponding first propagation path from the corresponding image point to the display device and the normal of the display device. In other words, the angle of the second propagation has changed / reduced relative to the first propagation path. Similar to what is described with respect to the first aspect, this can advantageously mean that the sub-hologram changes relatively slowly (because the wave is propagated so that it has a low spatial frequency). This advantageously enables undersampling during the initial sub-hologram calculation, thereby significantly improving the computational efficiency of determining the value of the sub-hologram. In practice, the modification of the propagation path may effectively mean that the second propagation path propagates from a different point to the initial position of the image point defined in the target picture. This may be equivalent to the corresponding image point having a modified position relative to the initial position in the target picture.

[0038] In some embodiments, the second angle is less than the first angle. In some embodiments, the second angle is substantially equal to zero (so that the second propagation path is substantially perpendicular to the plane of the display device). In some embodiments, the first propagation path for each image point is arranged so that it passes through an aperture of the observation system in an observation window of the optical system. In some embodiments, the second propagation path for at least some image points is arranged so that it does not pass through an aperture of the observation system in the observation window of the optical system.

[0039] In a third aspect of the present disclosure, a hologram engine for calculating a hologram of a target picture including a plurality of image points is provided. The hologram may be suitable for display on a display device of an optical system. The hologram engine is arranged to determine a first sub-hologram of a first plurality of image points of the target picture. For each image point, the hologram engine achieves this by propagating a wave (or wavelet) from the corresponding image point to the display device along a propagation axis. A main angle associated with each image point is defined between the corresponding propagation axis and the normal of the display device. Determining the first sub-hologram includes changing the main angle of the propagation axis of each image point before propagating the wave. In other words, the hologram engine may be arranged so that before propagation, the main angle may be different from the main angle used during propagation. The initial main angle may be defined by a propagation axis defined by an observation system from the corresponding image point to an observation window of the optical system. The effect of changing the main angle before propagation may cause each image point to move from an initial position (as defined in the target image) before the wave propagates.

[0040] In some embodiments, changing the main angle includes reducing the main angle of the propagation axis. In some embodiments, reducing the main angle includes reducing the main angle substantially to zero. This can cause the propagation axis of each image point to be substantially perpendicular to the display device. In some embodiments, the hologram engine is further arranged to determine one or more grating functions for compensating for the change in the main angle. The change in the main angle of the propagation axis of each image point can result in a change in the position of the image point. One or more diffraction gratings can be arranged to return the image point to its original position (when the sub-hologram and the diffraction grating are appropriately illuminated to form a holographic reconstruction of the target picture). The hologram engine can be arranged to combine (e.g., simultaneously superimpose or display) a first sub-hologram with one or more first diffraction gratings.

[0041] In a fourth aspect according to the present disclosure, a hologram engine for calculating a hologram of a target picture including a plurality of image points is provided. The hologram is used in an optical system, the optical system including a display device arranged to display the hologram and spatially modulate light incident thereon according to the hologram to form a holographic wavefront, at least a portion of which can be received by an entrance pupil of an observer in an observation window (or eye box) of the optical system. The hologram engine is arranged to identify a plurality of first image points in a first portion of the target picture. For each first image point, the hologram engine is arranged to use ray tracing to identify a ray propagating from the display device to the entrance pupil via the corresponding first image point. The hologram engine is also arranged to determine a propagation angle of the ray relative to a normal (at the display device) of the display device. The hologram engine is further arranged to reduce the propagation angle. The hologram engine is also arranged to determine a sub-hologram (e.g., determine an array of complex amplitude or phase values) of the plurality of first image points using the reduced propagation angle of the corresponding ray.

[0042] In some embodiments, the hologram engine may be arranged to reduce the propagation angle to be closer to or substantially equal to zero. This may cause the light rays at the corresponding point to be substantially perpendicular to the display device. In other words, this may cause the angle to be substantially equal to zero at the display device. The propagation angle of each first image point (of the first part of the target image) may be reduced by the same amount. The hologram engine may also be arranged to determine one or more grating functions to compensate for the reduction in one or more propagation angles used in the determination of the sub-hologram. In some embodiments, the hologram engine is further arranged to combine the sub-hologram with one or more grating functions.

[0043] In a fifth aspect according to the present disclosure, a hologram engine for calculating a hologram of a target picture including a plurality of image points is provided. The hologram can be used for display on a display device of an optical system. The hologram engine is arranged to identify a plurality of first image points in a first portion of the target picture. The hologram engine is also arranged to reposition (or translate) each first image point from the first portion to a second portion of the target image. The hologram engine is further arranged to determine a first sub-hologram of the repositioned first image point (i.e., a first sub-hologram of the repositioned first image point when in the second portion). The second portion is positioned closer to the center of the target picture than the first portion. Moving the first image point to reposition it to the second portion closer to the center of the target image advantageously means that the image point has been moved to a portion representing a lower spatial frequency than the first portion. This may mean that the first sub-hologram can be advantageously determined at a lower resolution (i.e., undersampling) relative to the first sub-hologram of the image point in the first portion.

[0044] In a sixth aspect, a hologram engine for calculating a hologram of a target picture including a plurality of image points is provided. The hologram is for display on a display device. The hologram engine is arranged to identify a first image point contained in a first portion of the target picture. The hologram engine is further arranged to reposition the first image point from the first portion to a second portion of the target picture. The hologram engine is further arranged to determine a first sub-hologram of the repositioned first image point. The second portion is positioned closer to a center of the target picture than the first portion.

[0045] The definitions of the fifth and sixth aspects define the invention in terms of moving image points, whereas the definition of the first aspect (and many other aspects) defines the invention in terms of changing the angle of the propagation path associated with each image point. It will be clear to a person skilled in the art that changing the angle of the propagation path has an effect corresponding to moving the image points. Therefore, a person skilled in the art will understand that both definitions relate to corresponding concepts. In particular, both the movement of the image points and the change / reduction in the angle of the propagation path have the effect of causing the sub-hologram to have a relatively low spatial frequency.

[0046] In a seventh aspect, a method for calculating a hologram of a target picture for display on a display device of an optical system is provided. The target picture includes a plurality of image points. Each image point is defined by a light path from it to an entrance pupil of an observer. Each light path is at an angle to the display device. The method includes determining a first sub-hologram of a first subset of image points of the target picture. The step of determining the first sub-hologram includes, for each image point of the first subset: a) propagating a wave along a propagation path, the angle formed by the propagation path and the display device is reduced relative to the angle formed by the corresponding light path; and b) for each image point of the first subset, determining a contribution of the wave at the display device.

[0047] The method may include determining or forming a hologram of the target picture. The determined or formed hologram may include a first sub-hologram. The method may include outputting, displaying or storing the hologram of the target picture.

[0048] The method may comprise displaying the hologram on a display device such as a spatial light modulator. The method may comprise illuminating the hologram, for example illuminating the hologram with light from a light source such as a laser. The hologram is arranged such that when the hologram is displayed on the display device and suitably illuminated, a first subset of image points of the target picture may be holographically reconstructed. Thus, the method may comprise holographically reconstructing the first subset of image points.

[0049] The method may comprise storing the (determined or formed) hologram in a memory.

[0050] In some embodiments the method further comprises applying one or more grating functions to the first sub-hologram, the one or more grating functions being arranged to compensate for a decrease in the angle of the propagation path of the image points of the first subset.

[0051] In some embodiments, the method includes identifying or defining a propagation path for each first image point before propagating the corresponding wave along the propagation path. The step of identifying or defining the propagation path may include taking the corresponding light path to modify the propagation path. The modification of the corresponding light path may include reducing the angle formed by the corresponding light path and the display device. The angle may be reduced to substantially zero. In other words, it can be said that the method includes the step of moving each image point of the first subset. This is because it can be said that moving the light path has the effect of moving the starting point of the light path, and the starting point of the light path can be equivalent to the position of the corresponding image point. The step of moving each image point of the first subset may include moving each image point closer to the center of the target picture. This may be because reducing the angle formed at the display device can be considered to have the effect of moving the image point closer to the center of the field of view of the display.

[0052] In some embodiments, the angle formed by the propagation path associated with each image point of the first subset and the display device is reduced by substantially the same amount.

[0053] In some embodiments, the step of propagating the wave may include propagating the wave along a propagation path for at least some of the image points of the first subset, the propagation path not passing through the entrance pupil. In some embodiments, the step of propagating the wave along a propagation path for the image points of the first subset may include propagating the wave along a propagation path arranged such that the respective wave does not pass through the respective image point. In some embodiments, each propagation path of the image points of the first subset passes through a first portion of the target picture, wherein a distance between the first portion and a center of the target picture is less than a distance between at least one image point of the first subset and the center.

[0054] In some embodiments, determining the first sub-hologram may include initially undersampling the first sub-hologram. This may cause the values ​​of the first sub-hologram to be initially determined for a subset of pixels of the display device, with one or more undetermined pixels between each determined sub-hologram pixel. The method may also include interpolating the undersampled first sub-hologram in at least a first dimension of the display device. The interpolation may cause the values ​​of at least some of the undetermined values ​​of the first sub-hologram to be determined.

[0055] In some embodiments, the method may include dividing the target picture into at least n subsets of image points, where n is greater than or equal to 10. The n subsets may include a first subset. The method may include: for each of the n subsets of image points, determining an nth sub-hologram for the nth subset. This may include determining a contribution of a wave at a display device for each image point of the nth subset. Each wave propagates along a propagation path that forms a reduced angle with a normal to the display device relative to an angle formed by a corresponding light path. In some embodiments, the angle formed by the propagation path and the display device associated with each image point in the nth subset is reduced by substantially the same amount.

[0056] The method may comprise applying one or more grating functions to the sub-holograms in each of the n subsets, the one or more grating functions being arranged to compensate for the reduced angle of the propagation path of the nth subset.

[0057] In some embodiments, the method of the seventh aspect is a computer-implemented method.

[0058] In an eighth aspect according to the present disclosure, there is provided an optical system comprising a display device for displaying a hologram and a processor adapted to perform the steps of the method of the seventh aspect.

[0059] In a ninth aspect according to the present disclosure, there is provided a computer program comprising instructions, which, when executed by a computer, cause the computer to execute the steps of the method of the seventh aspect.

[0060] In a tenth aspect according to the present disclosure, there is provided a computer-readable storage medium comprising instructions, which, when executed by a computer, causes the computer to perform the steps of the method of the seventh aspect.

[0061] In an eleventh aspect according to the present disclosure, an optical system is provided, comprising a display device for displaying a hologram and a processor. The processor is adapted to perform the steps of a method for calculating a hologram of a target picture for display on the display device of the optical system, the target picture comprising a plurality of image points, wherein each image point is defined by a first path from the image point to an entrance pupil of an observer, each first path being at an angle to the display device. The method (which the processor is adapted to perform) comprises:

[0062] Determine a first sub-hologram of a first subset of image points of the target picture;

[0063] Wherein determining the first sub-hologram comprises, for each image point of the first subset:

[0064] propagating the wave along a second path, the second path forming an angle with the display device that is reduced relative to an angle formed by a corresponding first path; and

[0065] A contribution of the wave at the display device is determined for each image point of the first subset.

[0066] According to a ninth aspect of the present disclosure, there is provided a computer program comprising instructions, which, when executed by a computer, causes the computer to perform the steps of a method for calculating a hologram of a target picture for display on a display device of an optical system, wherein the target picture comprises a plurality of image points, wherein each image point is defined by a first path from the image point to an entrance pupil of an observer, and each first path is at an angle to the display device. The method (caused by the computer program to cause the computer to perform) comprises:

[0067] Determine a first sub-hologram of a first subset of image points of the target picture;

[0068] Wherein determining the first sub-hologram comprises, for each image point of the first subset:

[0069] propagating the wave along a second path, the second path forming an angle with the display device that is reduced relative to an angle formed by a corresponding first path; and

[0070] A contribution of the wave at the display device is determined for each image point of the first subset.

[0071] In a tenth aspect according to the present disclosure, there is provided a computer-readable storage medium comprising instructions, which, when executed by a computer, causes the computer to perform the steps of a method for calculating a hologram of a target picture for display on a display device of an optical system, wherein the target picture comprises a plurality of image points, wherein each image point is defined by a first path from the image point to an entrance pupil of an observer, and each first path is at an angle to the display device. The method (caused by the computer to be executed by the instructions) comprises:

[0072] Determine a first sub-hologram of a first subset of image points of the target picture;

[0073] Wherein determining the first sub-hologram comprises, for each image point of the first subset:

[0074] propagating the wave along a second path, the second path forming an angle with the display device that is reduced relative to an angle formed by a corresponding first path; and

[0075] A contribution of the wave at the display device is determined for each image point of the first subset.

[0076] In the present disclosure, the term "replica" is used only when the reflected spatially modulated light is split so that the composite light field is directed along multiple different optical paths. The word "replica" is used to refer to each occurrence or instance of the composite light field after a replication event, such as a partial reflection-transmission of a pupil expander. Each replica travels along a different optical path. Some embodiments of the present disclosure relate to the propagation of light encoded with a hologram rather than an image, that is, light spatially modulated with a hologram of an image rather than the image itself. It can therefore be said that multiple replicas of the hologram are formed. Those skilled in the art of holography will understand that the composite light field associated with the propagation of light encoded with a hologram will change with the propagation distance. The use of the term "replica" herein is independent of the propagation distance, and therefore two branches or paths of light associated with a replication event are still referred to as "replicas" of each other, even if the branches have different lengths, so that the composite light field has evolved differently along each path. That is, according to the present disclosure, two composite light fields are still considered "replicas" even if they are associated with different propagation distances - assuming they come from the same replication event or a series of replication events.

[0077] According to the present disclosure, a "diffraction light field" is a light field formed by diffraction. The diffraction light field can be formed by illuminating a corresponding diffraction pattern. According to the present disclosure, an example of a diffraction pattern is a hologram and an example of a diffraction light field is a holographic light field or a light field of a holographic reconstruction that forms an image. The holographic light field forms a (holographic) reconstruction of an image on a playback plane. The holographic light field propagated from the hologram to the playback plane can be said to include light encoded with the hologram or light in the hologram domain. The diffraction light field is characterized by a diffraction angle determined by the minimum characteristic size of the diffraction structure and the wavelength of the light (of the diffraction light field). According to the present disclosure, it can also be said that the "diffraction light field" is a reconstructed light field formed on a plane separated from the corresponding diffraction structure space. This article discloses an optical system for propagating a diffraction light field from a diffraction structure to an observer. The diffraction light field can form an image.

[0078] The term "hologram" is used to refer to a record containing amplitude information or phase information or some combination thereof about an object. The term "holographic reconstruction" is used to refer to the optical reconstruction of an object formed by illuminating a hologram. The system disclosed herein is described as a "holographic projector" because the holographic reconstruction is a real image and is spatially separated from the hologram. The term "replay field" is used to refer to the 2D area in which the holographic reconstruction is formed and is fully focused. If the hologram is displayed on a spatial light modulator including pixels, the replay field will be repeated in the form of multiple diffraction orders, each of which is a replica of the zero-order replay field. The zero-order replay field usually corresponds to the preferred or main replay field because it is the brightest replay field. Unless otherwise explicitly stated, the term "replay field" should be regarded as referring to the zero-order replay field. The term "replay plane" is used to refer to a plane in space that contains all replay fields. The terms "image", "replay image" and "image area" refer to the area of ​​the replay field illuminated by the light of the holographic reconstruction. In some embodiments, an "image" may include discrete points, which may be referred to as "image points" or, for convenience only, may be referred to as "image pixels."

[0079] The terms "encoding", "writing" or "addressing" are used to describe the process of providing a corresponding plurality of control values ​​to a plurality of pixels of the SLM, which control values ​​respectively determine the modulation level of each pixel. It can be said that the pixels of the SLM are configured to "display" a light modulation distribution in response to receiving the plurality of control values. Therefore, the SLM can be said to "display" a hologram, and the hologram can be considered to be an array of light modulation values ​​or levels.

[0080] It has been found that a holographic reconstruction of acceptable quality can be formed from a "hologram" that contains only phase information related to the Fourier transform of the original object. Such a holographic recording may be referred to as a phase-only hologram. The embodiments relate to phase-only holograms, but the disclosure is equally applicable to amplitude-only holography.

[0081] The present disclosure is also equally applicable to forming a holographic reconstruction using amplitude and phase information associated with the Fourier transform of the original object. In some embodiments, this is achieved by using complex modulation of a so-called fully complex hologram, which contains amplitude and phase information associated with the original object. Such a hologram can be called a fully complex hologram because the value (gray level) assigned to each pixel of the hologram has an amplitude and a phase component. The value (gray level) assigned to each pixel can be represented as a complex number having both an amplitude component and a phase component. In some embodiments, a fully complex computer-generated hologram is calculated.

[0082] Reference may be made to the phase value, phase component, phase information, or simply phase of a pixel of a computer-generated hologram or spatial light modulator as shorthand for "phase delay". That is, any phase value described is actually a number (e.g., in the range of 0 to 2π) representing the amount of phase delay provided by that pixel. For example, a pixel of a spatial light modulator described as having a phase value of π∕2 will delay the phase of the received light by π∕2 radians. In some embodiments, each pixel of the spatial light modulator may be operated at one of a plurality of possible modulation values ​​(e.g., phase delay values). The term "grayscale" may be used to refer to a plurality of available modulation levels. For example, for convenience, the term "grayscale" may be used to refer to a plurality of available phase levels in a phase modulator alone, even though different phase levels do not provide different shades of gray. For convenience, the term "grayscale" may also be used to refer to a plurality of available complex modulation levels in a complex modulator.

[0083] Thus, a hologram comprises an array of gray levels, i.e. an array of light modulation values, such as phase delay values ​​or an array of complex modulation values. A hologram is also considered to be a diffraction pattern, since it is a pattern that causes diffraction when displayed on a spatial light modulator and illuminated with light having a wavelength comparable to (usually smaller than) the pixel pitch of the spatial light modulator. Reference is made here to combining a hologram with other diffraction patterns, such as diffraction patterns used as lenses or gratings. For example, a diffraction pattern used as a grating can be combined with a hologram to shift the replay field on the replay plane, or a diffraction pattern used as a lens can be combined with a hologram to focus the holographic reconstruction on the replay plane in the near field.

[0084] Although different embodiments and embodiment groups may be disclosed separately in the following specific embodiments, any feature of any embodiment or embodiment group may be combined with any other feature or feature combination of any embodiment or embodiment group. That is, all possible combinations and arrangements of the features disclosed in this disclosure are contemplated. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Specific embodiments are described by way of example only with reference to the following drawings:

[0086] Figure 1 is a schematic diagram showing a reflective SLM producing a holographic reconstruction on a screen;

[0087] Figure 2 An image for projection is shown, comprising eight image regions / components V1 to V8, and corresponding cross-sections of hologram channels H1 - H8;

[0088] Figure 3 shows a hologram displayed on an LCOS that directs light into multiple discrete regions;

[0089] Figure 4 A system including a display device is shown, the display device displays Figure 2 and Figure 3 The calculated hologram shown;

[0090] Figure 5A shows a perspective view of a first example two-dimensional pupil expander comprising two replicators, each replicator comprising a pair of stacked surfaces;

[0091] Figure 5B shows a perspective view of a first example two-dimensional pupil expander comprising two replicators, each in the form of a solid waveguide;

[0092] Figure 6 A schematic diagram representing a conventional point cloud hologram method is shown;

[0093] Fig. 7A Shown in isolation Figure 6 A hologram of a first image point of a point cloud;

[0094] Figure 7B Shown in isolation Figure 6 a hologram of another image point of the point cloud different from the first image point;

[0095] Fig. 8A showing a first image point of a point cloud in a point cloud hologram according to the present disclosure, including a ray path extending from the first image point;

[0096] Figure 8B shows the propagation path along which the wave propagates for calculating the contribution of the first image point to the hologram;

[0097] Fig. 9 An undersampled sub-hologram is shown;

[0098] Fig.10 A target picture is shown;

[0099] Fig.11 Shows Fig.10 The target image is divided into a plurality of units; and

[0100] Fig.12 The calculated method according to the present invention is shown. Fig.11 A representation of a holographic reconstruction of a sub-hologram of a first unit and before one or more grating functions have been applied to compensate for the reduction in the angle of the propagation path used in the calculation.

[0101] The same reference numbers will be used throughout the drawings to refer to the same or like parts. DETAILED DESCRIPTION

[0102] The present invention is not limited to the embodiments described below, but extends to the full scope of the appended claims. That is, the present invention can be implemented in different forms and should not be construed as limited to the described embodiments, which are set forth for illustrative purposes.

[0103] Unless otherwise specified, terms in the singular may include plural forms.

[0104] A structure described as being formed at the upper / lower portion of another structure or above / below another structure should be construed to include the case where these structures are in contact with each other and, further, include the case where a third structure is provided between these structures.

[0105] When describing a temporal relationship, for example, when the temporal order of events is described as "after," "subsequently," "next," "before," etc., unless otherwise specified, the disclosure should be considered to include both sequential and non-sequential events. For example, unless words such as "just," "immediately," or "directly" are used, the description should be considered to include non-sequential situations.

[0106] Although the terms "first", "second", etc. may be used herein to describe various elements, these elements are not limited by these terms. These terms are only used to distinguish one element from another element. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element, without departing from the scope of the appended claims.

[0107] The features of different embodiments may be coupled or combined with each other in part or in whole, and may interoperate with each other in various ways. Some embodiments may be performed independently of each other, or may be performed together in a mutually dependent relationship.

[0108] In the present disclosure, the term “substantially” when applied to a structural unit of a device may be interpreted as technical features of the structural unit being produced within the technical tolerance of a method for manufacturing the structural unit.

[0109] Conventional optical configuration for holographic projection

[0110] Figure 1 An embodiment is shown in which a computer generated hologram is encoded on a single spatial light modulator. The computer generated hologram is a Fourier transform of the object used for reconstruction. Thus, it can be said that the hologram is a Fourier domain or frequency domain or spectral domain representation of the object. In this embodiment, the spatial light modulator is a reflective liquid crystal on silicon "LCOS" device. The hologram is encoded on the spatial light modulator and the holographic reconstruction is formed at the replay field (e.g., a light receiving surface such as a screen or diffuser).

[0111] A light source 110, such as a laser or laser diode, is arranged to illuminate the SLM 140 via a collimating lens 111. The collimating lens causes a substantially planar wavefront of light to be incident on the SLM. Figure 1 In some embodiments, the direction of the wavefront is off-normal (e.g., two or three degrees from a plane that is truly normal to the transparent layer). However, in other embodiments, a substantially planar wavefront is provided at normal incidence, and a beam splitter arrangement is used to separate the input and output optical paths. Figure 1 In the embodiment shown, the arrangement is such that light from the light source reflects from the mirrored back surface of the SLM and interacts with the light modulating layer to form an exit wavefront 112. The exit wavefront 112 is applied to an optical device including a Fourier transform lens 120, which has its focus at a screen 125. More specifically, the Fourier transform lens 120 receives the modulated light beam from the SLM 140 and performs a frequency-space transform to produce a holographic reconstruction at the screen 125.

[0112] It is important to note that in this type of holography, each pixel of the hologram contributes to the entire reconstruction. There is no one-to-one correlation between a specific point on the replay field (or image pixel) and a specific light modulation element (or hologram pixel). In other words, the modulated light leaving the light modulation layer is distributed over the replay field.

[0113] In these embodiments, the position of the holographic reconstruction in space is determined by the refractive (focusing) power of the Fourier transform lens. Figure 1 In the illustrated embodiments, the Fourier transform lens is a physical lens. That is, the Fourier transform lens is an optical Fourier transform lens, and the Fourier transform is performed optically. Any lens can be used as a Fourier transform lens, but the performance of the lens will limit the accuracy of the Fourier transform it performs. Those skilled in the art understand how to use a lens to perform an optical Fourier transform. In some embodiments of the present disclosure, the lens of the observer's eye performs the hologram-to-image transformation.

[0114] Hologram computing

[0115] In some embodiments, the computer-generated hologram is a Fourier transform hologram, or simply a Fourier hologram or a Fourier-based hologram, in which the image is reconstructed in the far field by utilizing the Fourier transform properties of a positive lens. The Fourier hologram is calculated by Fourier transforming the desired light field in the replay plane back to the lens plane. The computer-generated Fourier hologram can be calculated using the Fourier transform. The embodiments refer to Fourier holography and Gerchberg-Saxton type algorithms only by way of example. The present disclosure is also applicable to Fresnel holography and Fresnel holograms that can be calculated by similar methods. In some embodiments, the hologram is a phase hologram or a phase-only hologram. However, the present disclosure is also applicable to holograms calculated by other techniques, such as techniques based on point cloud methods.

[0116] In some embodiments, the hologram engine is arranged to exclude from the hologram calculation the contribution of light blocked by the limiting aperture of the display system. UK patent application 2101666.2, filed on February 5, 2021 and incorporated herein by reference, discloses a first hologram calculation method in which eye tracking and ray tracing are used to identify sub-areas of a display device for calculating a point cloud hologram that eliminates ghost images. The sub-areas of the display device correspond to the aperture of the present disclosure and are used to exclude light paths from the hologram calculation. UK patent application 2112213.0, filed on August 26, 2021 and incorporated herein by reference, discloses a second method based on a modified Gerchberg-Saxton type algorithm, which includes a step of performing light field clipping according to the pupil of the optical system during the hologram calculation. The clipping of the light field corresponds to the determination of the limiting aperture of the present disclosure. UK patent application 2118911.3, filed on December 23, 2021 and incorporated herein by reference, discloses a third method of computing a hologram, the method comprising the step of determining the area of ​​a so-called extended modulator formed by a hologram replicator. According to the present disclosure, the area of ​​the extended modulator is also the aperture.

[0117] In some embodiments, a real-time engine is provided that is arranged to receive image data and use an algorithm to calculate a hologram in real time. In some embodiments, the image data is a video comprising a sequence of image frames. In other embodiments, the hologram is pre-calculated, stored in a computer memory and called up as needed for display on the SLM. That is, in some embodiments, a repository of predetermined holograms is provided.

[0118] Large field of view using a small display device

[0119] In general, the present disclosure relates to image projection. The present invention relates to an image projection method and an image projector including a display device. The present disclosure also relates to a projection system including an image projector and an observation system, wherein the image projector projects or relays light from the display device to the observation system. The present disclosure is equally applicable to monocular and binocular observation systems. The observation system may include one or more eyes of an observer. The observation system includes an optical element having an optical focal length (e.g., a lens of a human eye) and an observation plane (e.g., a retina of a human eye). The projector may be referred to as a "light engine". The display device and the image formed (or perceived) using the display device are spatially separated from each other. The image is formed or perceived by the observer on the display plane. In some embodiments, the image is a virtual image, and the display plane may be referred to as a virtual image plane. In other examples, the image is a real image formed by holographic reconstruction, and the image is projected or relayed to the observation plane. In these other examples, spatially modulated light of an intermediate holographic reconstruction formed in free space or on a screen or other light receiving surface between the display device and the observer is propagated to the observer. In both cases, the image is formed by illuminating a diffraction pattern (eg, a hologram or kinoform) displayed on a display device.

[0120] The display device includes pixels. The pixels of the display can display a diffraction pattern or structure of diffracted light. The diffracted light can form an image at a plane that is spatially separated from the display device. According to well-known optical devices, the size of the maximum diffraction angle is determined by the size of the pixel and other factors (such as the wavelength of the light).

[0121] In an embodiment, the display device is a spatial light modulator, such as a liquid crystal on silicon ("LCOS") spatial light modulator (SLM). Light propagates from the LCOS toward an observing entity / system (such as a camera or eye) over a range of diffraction angles (e.g., from zero to a maximum diffraction angle). In some embodiments, amplification techniques may be used to increase the range of available diffraction angles beyond the conventional maximum diffraction angle of the LCOS.

[0122] In some embodiments, (the light of) the hologram itself is transmitted to the eye. For example, the spatially modulated light of the hologram (which has not yet been fully transformed into a holographic reconstruction, i.e., an image) (which can be informally referred to as being "encoded with" / "encoded by" the hologram) is transmitted directly to the eye of the observer. The observer can perceive a real or virtual image. In these embodiments, no intermediate holographic reconstruction / image is formed between the display device and the observer. It is sometimes said that in these embodiments, the lens of the eye performs the conversion or transformation of the hologram to the image. The projection system or light engine can be configured so that the observer effectively directly observes the display device.

[0123] The "light field" referred to in this article is a "composite light field". The term "light field" simply refers to a pattern of light with finite size in at least two orthogonal spatial directions (e.g., 𝑥 and 𝑦). The word "complex" is used in this article simply to indicate that the light at each point in the light field can be defined by an amplitude value and a phase value, and can therefore be represented by a complex number or a pair of values. For the purpose of hologram calculation, the composite light field can be a two-dimensional array of complex numbers, where the complex numbers define the light intensity and phase at multiple discrete locations within the light field.

[0124] According to well-known optical principles, the angular range of light propagating from a display device that can be observed by an eye or other observing entity / system varies with the distance between the display device and the observing entity. For example, at a viewing distance of 1 meter, for a given eye position, only a small range of angles from the LCOS can propagate through the pupil of the eye to form an image at the retina. The angular range of light rays propagating from the display device that can successfully propagate through the pupil of the eye to form an image at the retina at a given eye position determines the portion of the image that is "visible" to the observer. In other words, not all portions of the image are visible from any one point on the viewing plane (e.g., any one eye position within a viewing window such as an eyebox)

[0125] In some embodiments, the image perceived by the observer is a virtual image that appears upstream of the display device, i.e., the observer perceives the image as being farther away from them than the display device. Thus, conceptually, the observer can be thought of as observing the virtual image through a "display device-sized window", which can be very small, such as 1 cm in diameter, at a relatively large distance, such as 1 meter. And the user will be observing the display device-sized window via the pupil of their eye, which can also be very small. Thus, at any given time, the field of view becomes smaller, and the specific range of angles that can be seen depends largely on the eye position.

[0126] Pupil expanders solve the problem of how to increase the angular range of light rays propagating from a display device that can successfully propagate through the pupil of the eye to form an image. Display devices are typically (relatively) small and have a (relatively) large projection distance. In some embodiments, the projection distance is at least one (such as at least two) orders of magnitude larger than the diameter or width of the entrance pupil and / or aperture of the display device (i.e., the size of the pixel array).

[0127] The use of a pupil expander increases the viewing area (i.e., the user's eye box) laterally, thereby enabling some movement of the eye to occur while still enabling the user to see the image. As will be appreciated by those skilled in the art, in an imaging system, the viewing area (the user's eye box) is the area where the viewer's eye can perceive the image. The present disclosure encompasses non-infinite virtual image distances, i.e., near-field virtual images.

[0128] Conventionally, a two-dimensional pupil expander comprises one or more one-dimensional optical waveguides, each formed using a pair of opposing reflective surfaces, wherein the output light from the surfaces forms an observation window or eye box. Light received from a display device (e.g., spatially modulated light from an LCOS) is replicated by the or each waveguide to increase the field of view (or observation area) in at least one dimension. In particular, the waveguide enlarges the observation window due to the generation of additional light rays or "replicas" by dividing the amplitude of the incident wavefront.

[0129] The display device may have an active area or display area having a first dimension that may be less than 10 cm, such as less than 5 cm or less than 2 cm. The propagation distance between the display device and the observation system may be greater than 1 m, such as greater than 1.5 m or greater than 2 m. The optical propagation distance within the waveguide may be up to 2 m, such as up to 1.5 m or up to 1 m. The method may be capable of receiving an image and determining a corresponding hologram of sufficient quality in less than 20 ms, such as less than 15 ms or less than 10 ms.

[0130] In some embodiments described only by examples of diffraction or holographic light fields according to the present disclosure, the hologram is configured to route light into multiple channels, each corresponding to a different portion (i.e., sub-region) of the image. The channels formed by the diffraction structure are referred to herein as "hologram channels" simply to reflect that they are channels of light encoded by a hologram with image information. It can be said that the light of each channel is in the hologram domain rather than the image or spatial domain. In some embodiments, the hologram is a Fourier or Fourier transform hologram, and the hologram domain is therefore the Fourier or frequency domain. The hologram can also be a Fresnel hologram or a Fresnel transform hologram. The hologram can also be a point cloud hologram. The hologram is described herein as routing light into multiple hologram channels to reflect that the image that can be reconstructed from the hologram has a finite size and can be arbitrarily divided into multiple image sub-regions, where each hologram channel will correspond to each image sub-region. Importantly, the hologram of this example is characterized by how it distributes the image content when it is illuminated. Specifically and uniquely, the hologram divides the image content by angle. That is, each point on the image is associated with a unique ray angle (at least a unique pair of angles) in the spatially modulated light formed by the hologram when illuminated, because the hologram is two-dimensional. For the avoidance of doubt, this hologram behavior is not conventional. When illuminated, the spatially modulated light formed by this special type of hologram can be divided into multiple hologram channels, where each hologram channel is defined by a range of ray angles (in two dimensions). From the foregoing, it will be understood that any hologram channel (i.e., a sub-range of ray angles) that can be considered in the spatially modulated light will be associated with a corresponding portion or sub-region of the image. That is, all of the information required to reconstruct that portion or sub-region of the image is contained within the sub-range of angles of the spatially modulated light formed by the hologram of the image. When the spatially modulated light is observed as a whole, there is not necessarily any evidence of multiple discrete light channels.

[0131] However, the hologram can still be identified. For example, if only continuous portions or sub-regions of the spatially modulated light formed by the hologram are reconstructed, only sub-regions of the image should be visible. If different continuous portions or sub-regions of the spatially modulated light are reconstructed, different sub-regions of the image should be visible. Another identifying feature of this type of hologram is that the shape of the cross-sectional area of ​​any hologram channel substantially corresponds to (i.e., is substantially the same as) the shape of the entrance pupil, although the size may be different - at least at the correct plane where the hologram is calculated. Each light / hologram channel propagates from the hologram at a different angle or range of angles. While these are example ways to characterize or identify this type of hologram, other ways may be used. In summary, the holograms disclosed herein are characterized and identified by how the image content is distributed within the light encoded by the hologram. Again, to avoid any doubt, references herein to holograms configured to guide light or to angularly divide an image into multiple hologram channels are only by way of example, and the present disclosure is equally applicable to any type of holographic light field or even any type of diffraction or pupil expansion of a diffracted light field.

[0132] The system can be provided in a compact and streamlined physical form. This enables the system to be suitable for a wide range of real-world applications, including those where space is limited and asset value is high. For example, it can be implemented in a head-up display (HUD) such as a vehicle or automotive HUD.

[0133] According to the present disclosure, a pupil expansion is provided for diffracted light, which may include a diverging bundle of light rays. The diffracted light field may be defined by a "light cone". Thus, the size of the diffracted light field (as defined in a two-dimensional plane) increases with propagation distance from the corresponding diffractive structure (i.e., the display device). It can be said that the pupil expander replicates the hologram or forms at least one replica of the hologram to convey that the light transmitted to the observer is spatially modulated according to the hologram.

[0134] In some embodiments, two one-dimensional waveguide pupil expanders are provided, each of which is arranged to effectively increase the size of the exit pupil of the system by forming multiple replicas or copies of the exit pupil (or the light of the exit pupil) of the spatial light modulator. The exit pupil can be understood as the physical area from which the system outputs light. It can also be said that each waveguide pupil expander is arranged to expand the size of the exit pupil of the system. It can also be said that each waveguide pupil expander is arranged to expand / increase the size of the eye box in which the observer's eyes can be located in order to see / receive the light output by the system.

[0135] Optical Channel

[0136] A hologram formed in accordance with some embodiments angularly divides the image content to provide a plurality of hologram channels that may have a cross-sectional shape defined by the aperture of the optical system. The hologram is calculated to provide such channel guidance of the diffracted light field. In some embodiments, this is achieved by taking into account the aperture (virtual or real) of the optical system during the calculation of the hologram, as described above.

[0137] Figure 2 and Figure 3 An example of this type of hologram that can be used in conjunction with the pupil expander disclosed herein is shown. However, this example should not be viewed as limiting the present disclosure.

[0138] Figure 2 An image 252 is shown for projection which comprises eight image regions / components V1 to V8. Figure 2 Eight image components are shown by way of example only, and image 252 may be divided into any number of components. Figure 2 Also shown is a coded light pattern 254 (ie, hologram) that can reconstruct the image 252, for example, when transformed by a lens of a suitable viewing system. The coded light pattern 254 includes first to eighth sub-holograms or components H1 to H8 corresponding to first to eighth image components / regions V1 to V8. Figure 2 It also shows how a hologram can decompose the image content by angle. Therefore, a hologram is characterized by its channel guidance of light. This is Figure 3 Specifically, the hologram in this example directs light into multiple discrete regions. In the example shown, the discrete regions are disks, but other shapes are contemplated. After propagation through the waveguide, the size and shape of the optimal disk can be related to the size and shape of the aperture of the optical system (e.g., the entrance pupil of the observation system).

[0139] Figure 4 A system 400 is shown, which includes a display such as Figure 2 and Figure 3 A display device for a computed hologram is shown.

[0140] The system 400 comprises a display device, which in this arrangement comprises an LCOS 402. The LCOS 402 is arranged to display a modulation pattern (or "diffraction pattern") comprising a hologram and projects light that has been holographically encoded to an eye 405, which comprises a pupil acting as an aperture 404, a lens 409 and a retina (not shown) acting as an observation plane. There is a light source (not shown) arranged to illuminate the LCOS 402. The lens 409 of the eye 405 performs the transformation of the hologram into an image. The light source may be of any suitable type. For example, it may comprise a laser light source.

[0141] The viewing system 400 also includes a waveguide 408 located between the LCOS 402 and the eye 405. The presence of the waveguide 408 enables all angular content from the LCOS 402 to be received by the eye, even at the relatively large projection distance shown. This is because the waveguide 408 acts as a pupil expander in a well-known manner and is therefore only briefly described herein.

[0142] in short, Figure 4 The waveguide 408 shown in comprises a substantially elongated structure. In this example, the waveguide 408 comprises an optical plate of a refractive material, but other types of waveguides are also known and may be used. The waveguide 408 is positioned to intersect the light cone (i.e., the diffracted light field) projected from the LCOS 402, for example, at an oblique angle. In this example, the size, position, and positioning of the waveguide 408 are configured to ensure that light from each of the eight light beams within the light cone enters the waveguide 408. The light from the light cone enters the waveguide 408 via its first planar surface (located closest to the LCOS 402) and is guided at least partially along the length of the waveguide 408 before being emitted via its second planar surface (substantially opposite to the first surface (located closest to the eye)). As will be well understood, the second planar surface is partially reflective and partially transmissive. In other words, when each ray of light travels from the first planar surface within the waveguide 408 and strikes the second planar surface, some of the light will be transmitted out of the waveguide 408, and some of the light will be reflected back to the first planar surface by the second planar surface. The first planar surface is reflective so that all light that strikes it from within the waveguide 408 will be reflected back toward the second planar surface. Thus, some light may simply refract between the two planar surfaces of the waveguide 408 before being transmitted, while other light may be reflected and may therefore experience one or more reflections (or "bounces") between the planar surfaces of the waveguide 408 before being transmitted.

[0143] Figure 4 A total of nine "bounce" points B0 to B8 are shown along the length of waveguide 408. Although similar to Figure 2 Light associated with all points of the image (V1-V8) shown is projected out of the waveguide at each "bounce" from the second planar surface of the waveguide 408, but light from only one angular portion of the image (e.g., light from one of V1 to V8) has a trajectory that enables it to reach the eye 405 from each corresponding "bounce" point B0 to B8. In addition, light from different angular portions of the image V1 to V8 reaches the eye 405 from each corresponding "bounce" point. Therefore, at Figure 4 In the example of FIG. 4 , each angular channel of coded light reaches the eye only once from waveguide 408 .

[0144] The waveguide 408 forms multiple replicas of the hologram at corresponding "bounce" points B1 to B8 along its length, corresponding to the direction of pupil expansion. As shown in FIG. 5 , the multiple replicas can be extrapolated back to the corresponding multiple replicas or virtual display device 402 'in a straight line. This process corresponds to the step of "expanding" the light path within the waveguide so that the light of the replica is extrapolated back to the "virtual surface" without internal reflection within the waveguide. Therefore, the light of the expanded exit pupil can be considered to originate from the virtual surface (also referred to herein as the "expansion modulator") including the display device 402 and the replica display device 402 '.

[0145] Although virtual images have been discussed generally herein requiring the eye to transform received modulated light in order to form a perceived image, the methods and arrangements described herein may be applied to real images.

[0146] 2D pupil expansion

[0147] Although Figure 4 The arrangement shown in comprises a single waveguide that provides pupil expansion in one dimension, but pupil expansion may be provided in more than one dimension, for example in two dimensions. Figure 4 The examples in use a hologram that has been calculated to create channels of light, each corresponding to a different portion of an image, but the present disclosure and the systems described below are not limited to this type of hologram.

[0148] Figure 5A A perspective view of a system 500 is shown comprising two replicators 504, 506 arranged to expand a light beam 502 in two dimensions.

[0149] exist Figure 5A In the system 500, the first replicator 504 includes a first pair of surfaces stacked parallel to each other and arranged to Figure 4 The waveguide 408 provides replication or pupil expansion in a similar manner. The size and shape of the first pair of surfaces are similar to each other (in some cases the same), and are substantially elongated in one direction. The collimated beam 502 is directed toward the input on the first replicator 504. Due to the internal reflection process between the two surfaces, and from one surface (such as Figure 5A As will be familiar to those skilled in the art, the light of the beam 502 is replicated in a first direction along the length of the first replicator 504. Thus, a first plurality of replicated light beams 508 are emitted from the first replicator 504 toward the second replicator 506.

[0150] The second replicator 506 includes a second pair of surfaces stacked parallel to one another, arranged to receive each of the collimated beams in the first plurality of beams 508, and further arranged to provide replication or pupil expansion by expanding each of the beams in a second direction that is substantially orthogonal to the first direction. The first pair of surfaces are similar in size and shape to one another (in some cases, identical), and are substantially rectangular. The rectangular shape is implemented for the second replicator so that it has a length along the first direction so as to receive the first plurality of beams 508, and a length along the second orthogonal direction so as to provide replication in the second direction. Due to internal reflection processes between the two surfaces, and from one of the surfaces (the upper surface, such as Figure 5A The light of each beam in the first plurality of light beams 508 is replicated in the second direction by partially transmitting the light of each of the plurality of output points on the first plurality of light beams 508 (as shown). Thus, a second plurality of light beams 510 is emitted from the second replicator 506, wherein the second plurality of light beams 510 includes a replica of the input light beam 502 along each of the first direction and the second direction. Thus, the second plurality of light beams 510 can be considered to include a two-dimensional grid or array of replicated light beams.

[0151] Therefore, it can be said Figure 5A The first replicator 504 and the second replicator 505 are combined to provide a two-dimensional replicator (or a "two-dimensional pupil expander"). Thus, the replicated light beam 510 can be emitted along an optical path to an expanded eye box of a display system (such as a head-up display).

[0152] exist Figure 5A In a system of the type described above, the first replicator 504 is a waveguide comprising a pair of elongated rectilinear reflective surfaces stacked parallel to each other, and similarly, the second replicator 504 is a waveguide comprising a pair of rectangular reflective surfaces stacked parallel to each other. In other systems, the first replicator may be a solid elongated rectilinear waveguide and the second replicator may be a solid planar rectangular waveguide, wherein each waveguide comprises a solid optically transparent material, such as glass. In this case, the pair of parallel reflective surfaces are formed by a pair of opposing major side walls, which optionally include corresponding reflective and reflective-transmissive surface coatings, as will be familiar to the reader in the art.

[0153] Figure 5B A perspective view of a system 500 is shown comprising two replicators 520 , 540 arranged for replicating a light beam 522 in two dimensions, wherein the first replicator is a solid elongated waveguide 520 and the second replicator is a solid planar waveguide 540 .

[0154] exist Figure 5BIn the system of FIG. 5 , the first replicator / waveguide 520 is arranged so that its pair of elongated parallel reflective surfaces 524a, 524b are perpendicular to the plane of the second replicator / waveguide 540. Thus, the system includes an optical coupler arranged to couple light from the output port of the first replicator 520 into the input port of the second replicator 540. In the arrangement shown, the optical coupler is a plane / folding mirror 530, which is arranged to fold or deflect the optical path of the light to achieve the desired optical coupling from the first replicator to the second replicator. Figure 5B As shown, the reflector 530 is arranged to receive light from the output port / reflective-transmissive surface 524a of the first replicator / waveguide 520 (comprising a one-dimensional array of replicas extending in a first dimension). The reflector 530 is tilted so as to redirect the received light at an angle into the optical path of the input port in the (totally) reflective surface of the second replicator 540 to provide a waveguide and replica formation along its length in the second dimension. It should be understood that the reflector 530 is one example of an optical element that can redirect light in the manner shown, and one or more other elements may alternatively be used to perform this task.

[0155] In the arrangement shown, the (partial) reflective-transmissive surface 524a of the first replicator 520 is adjacent to the input port of the first replicator / waveguide 520, which receives the input beam 522 at an angle to provide waveguide and replica formation along its length in the first dimension. Thus, the input port of the first replicator / waveguide 520 is positioned at the same surface as the reflective-transmissive surface 524a at its input end. Those skilled in the art will appreciate that the input port of the first replicator / waveguide 520 may be in any other suitable location.

[0156] therefore, Figure 5B The arrangement enables the first replicator 520 and the reflector 530 to be disposed in a plane of the first and third dimensions (shown as the xz plane) as part of a first relatively thin layer. In particular, the size or "height" of the first planar layer (in which the first replicator 520 is located) in the second dimension (shown as the y dimension) is reduced. The reflector 530 is configured to direct light away from the first layer / plane (i.e., the "first planar layer") where the first replicator 520 is located, and to direct it toward the second layer / plane (i.e., the "second planar layer") where the second replicator 540 is located, which is located above the first layer / plane and substantially parallel to the first layer / plane. Therefore, the overall size or "height" of the system is compact in the second dimension (shown as the y dimension), and the system includes the first replicator 520 and the second replicator 540 and the reflector 530 located in the stacked first and second planar layers in the first and third dimensions (shown as the xz plane). Those skilled in the art will appreciate that the invention for implementing the present disclosure will be described in detail in detail. Figure 5B Many variations of the arrangement of are possible and contemplated.

[0157] The image projector can be arranged to project a divergent or diffracted light field. In some embodiments, the light field is encoded with a hologram. In some embodiments, the diffracted light field includes a divergent beam of light rays. In some embodiments, the image formed by the diffracted light field is a virtual image.

[0158] In some embodiments, the first pair of parallel / complementary surfaces are elongated or slender surfaces that are relatively long along a first dimension and relatively short along a second dimension, such as relatively short along each of two other dimensions, each dimension being substantially orthogonal to each of the respective other dimensions. The process by which light is reflected / transmitted between / from the first pair of parallel surfaces is arranged to cause the light to propagate within the first waveguide pupil expander, wherein the general direction of light propagation is in the direction in which the first waveguide pupil expander is relatively long (i.e., in its "elongated" direction).

[0159] A system is disclosed herein that forms an image using diffracted light and provides an eyebox size and field of view suitable for real world applications (e.g., in the automotive industry via head-up displays). Diffracted light is light that forms a holographic reconstruction of an image from a diffractive structure, for example, a hologram such as a Fourier or Fresnel hologram. The use of diffraction and diffractive structures requires a high density of display devices with very small pixels (e.g., 1 micron), which in practice means small display devices (e.g., 1 cm). The inventors have solved the problem of how to provide a diffracted light field (e.g., diffracted light comprising a divergent (non-collimated) light beam) for 2D pupil expansion.

[0160] In some embodiments, a display system includes a display device, (such as a pixelated display device, e.g., a spatial light modulator (SLM) or a liquid crystal on silicon (LCoS) SLM) that is arranged to provide or form diffracted or divergent light. In these aspects, the aperture of the spatial light modulator (SLM) is the limiting aperture of the system. That is, the aperture of the spatial light modulator (more specifically, the size of the area that defines the array of light modulating pixels contained within the SLM) determines the size (e.g., spatial extent) of the bundle of light rays that can exit the system. According to the present disclosure, it is stated that by using at least one pupil expander, the exit pupil of the system is expanded to reflect that the exit pupil of the system (which is limited by a small display device having a pixel size for light diffraction) becomes larger in spatial extension.

[0161] A diffracted or divergent light field may be referred to as having a "light field size" that is defined in a direction substantially orthogonal to the direction of propagation of the light field. Because the light is diffracted / diverged, the light field size increases with propagation distance.

[0162] In some embodiments, the diffracted light field is spatially modulated according to a hologram. In other words, in such aspects, the diffracted light field comprises a "holographic light field". The hologram can be displayed on a pixelated display device. The hologram can be a computer generated hologram (CGH). It can be a Fourier hologram or a Fresnel hologram or a point cloud hologram or any other suitable type of hologram. Optionally, the hologram can be calculated so as to form channels of hologram light, where each channel corresponds to a different respective portion of an image intended to be observed (or perceived, if it is a virtual image) by an observer. The pixelated display device can be configured to display multiple different holograms continuously or sequentially. Each aspect and embodiment disclosed herein can be applied to the display of multiple holograms.

[0163] The output port of the first waveguide pupil expander may be coupled to the input port of the second waveguide pupil expander. The second waveguide pupil expander may be arranged to guide the diffracted light field (comprising some, preferably most, preferably all of the replica of the light field output by the first waveguide pupil expander) from its input port to the corresponding output port by internal reflection between a third pair of parallel surfaces of the second waveguide pupil expander.

[0164] The first waveguide pupil expander may be arranged to provide pupil expansion or replication in a first direction, and the second waveguide pupil expander may be arranged to provide pupil expansion or replication in a second, different direction. The second direction may be substantially orthogonal to the first direction. The second waveguide pupil expander may be arranged to maintain the pupil expansion provided by the first waveguide pupil expander in the first direction, and to expand (or replicate) some, preferably most, preferably all of the replicas it receives from the first waveguide pupil expander in the second, different direction. The second waveguide pupil expander may be arranged to receive the light field from the first waveguide pupil expander directly or indirectly. One or more other elements may be provided along the propagation path of the light field between the first waveguide pupil expander and the second waveguide pupil expander.

[0165] The first waveguide pupil expander can be substantially elongated and the second waveguide pupil expander can be substantially planar. The elongated shape of the first waveguide pupil expander can be defined by a length along a first dimension. The planar or rectangular shape of the second waveguide pupil expander can be defined by a length along the first dimension and a width or breadth along a second dimension substantially orthogonal to the first dimension. The size or length of the first waveguide pupil expander along its first dimension corresponds to the length or width of the second waveguide pupil expander along its first dimension or second dimension, respectively. The first of a pair of parallel surfaces of the second waveguide pupil expander including its input port can be shaped, sized and / or positioned so as to correspond to an area defined by an output port on the first of a pair of parallel surfaces on the first waveguide pupil expander, such that the second waveguide pupil expander is arranged to receive each replica output by the first waveguide pupil expander.

[0166] The first waveguide pupil expander and the second waveguide pupil expander may jointly provide pupil expansion in a first direction and in a second direction perpendicular to the first direction, optionally wherein a plane containing the first direction and the second direction is substantially parallel to a plane of the second waveguide pupil expander. In other words, a first dimension and a second dimension defining a length and a width, respectively, of the second waveguide pupil expander may be parallel to the first direction and the second direction, respectively, in which the waveguide pupil expander provides pupil expansion (or parallel to the second direction and the first direction, respectively). The combination of the first waveguide pupil expander and the second waveguide pupil expander may generally be referred to as a "pupil expander".

[0167] It can be said that the expansion / replication provided by the first waveguide expander and the second waveguide expander has the effect of expanding the exit pupil of the display system in each of two directions. The area defined by the expanded exit pupil can in turn define an expanded eye box area from which an observer can receive light of the input diffracted or divergent light field. The eye box area can be said to be located on or define the observation plane.

[0168] The two directions of the exit pupil expansion may be coplanar or parallel to the first and second directions in which the first and second waveguide pupil expanders provide replication / expansion. Alternatively, in an arrangement including other elements such as an optical combiner (e.g., a windscreen (or windshield) of a vehicle), the exit pupil may be considered to be the exit pupil from the other element (e.g., from the windscreen). In such an arrangement, the exit pupil may be non-coplanar and non-parallel to the first and second directions in which the first and second waveguide pupil expanders provide replication / expansion. For example, the exit pupil may be substantially perpendicular to the first and second directions in which the first and second waveguide pupil expanders provide replication / expansion.

[0169] The observation plane and / or the eye box region may be non-coplanar or non-parallel to the first and second directions in which the first and second waveguide pupil expanders provide replication / expansion. For example, the observation plane may be substantially perpendicular to the first and second directions in which the first and second waveguide pupil expanders provide replication / expansion.

[0170] In order to provide suitable activation conditions to achieve internal reflections within the first and second waveguide pupil expanders, the elongated dimension of the first waveguide pupil expander may be tilted relative to the first and second dimensions of the second waveguide pupil expander.

[0171] Combiner shape compensation

[0172] An advantage of projecting a hologram to the eyebox is that optical compensation can be encoded in the hologram (see, e.g., European Patent 2936252, which is incorporated herein by reference). The present disclosure is compatible with holograms that compensate for the complex curvature of an optical combiner used as part of a projection system. In some embodiments, the optical combiner is a windshield of a vehicle. Full details of the method are provided in European Patent 2936252 and are not repeated here because the detailed features of these systems and methods are not essential to the novel teachings of the present disclosure and are merely examples of configurations that benefit from the teachings of the present disclosure.

[0173] Controls

[0174] The present disclosure is also compatible with an optical configuration including a control device (e.g., an optical shutter device) to control the delivery of light from the optical channel hologram to the observer. The holographic projector may also include a control device arranged to control the delivery of the angle channel to the eye box position. UK patent application 2108456.1 filed on June 14, 2021, which is incorporated herein by reference, discloses at least one waveguide pupil expander and control device. The reader will understand from at least this prior disclosure that the optical configuration of the control device is substantially based on the user's eye box position and is compatible with any hologram calculation method that implements the optical channel described herein. It can be said that the control device is an optical shutter or aperture device. The optical shutter device may include a 1D array of apertures or windows, each of which is independently switchable between a light-transmitting and a light-non-transmitting state to control the delivery of the hologram optical channel and its replica to the eye box. Each aperture or window may include a plurality of liquid crystal cells or pixels.

[0175] Point cloud hologram calculation

[0176] Figure 6 A schematic diagram representing a conventional point cloud hologram method is shown. Figure 6The hologram includes a point cloud 602 (i.e., a cloud of image points), a display device 604 (or a hologram plane), and an entrance pupil 606. The display device 604 is located between the entrance pupil 606 and the point cloud 602, and in this example is a pixelated liquid crystal on silicon spatial light modulator. In this example, the position of the entrance pupil 606 has been determined by eye tracking of an observer. The methods described herein are performed virtually (i.e., are simulations). Therefore, Figure 6 is merely a representation of what is computationally performed during the point cloud hologram calculation. Furthermore, each step of the method is performed virtually. For example, in the case of describing waves being propagated, the waves are not physically propagated through the physical system. Instead, the virtual waves are propagated virtually through the features of the virtual system. The display device 604 in this example represents the hologram. Figure 1 As described in more detail below, the purpose of point cloud hologram calculation is to determine a hologram that can be (physically) displayed on the display device 604, such that when the hologram is appropriately (physically) illuminated by coherent light, a (physical) holographic wavefront is formed that is relayed to the real entrance pupil of the observer, giving the appearance of a holographic reconstruction of the three-dimensional point cloud.

[0177] In this example, point cloud 602 is a three-dimensional point cloud, and these points represent an object or scene. Specifically, point cloud 602 represents a target picture based on which a hologram is calculated. Figure 6 Four image points are shown: a first image point 610, a second image point 612, a third image point 614, and a fourth image point 616. The first to fourth image points are distributed throughout the point cloud 604. This is merely representative. It should be understood that, typically, the point cloud of a point cloud hologram will include thousands or millions of image points that together represent a picture.

[0178] Conventionally, the point cloud hologram calculation method includes simulating the propagation of a light wave (or wavelet) along a propagation path from each image point of the point cloud 604 toward the center of the entrance pupil 606. The propagation path from the first image point 610 to the fourth image point 614 is represented by Figure 6 606; a first propagation path 620 extends from the first image point 610 to the entrance pupil 606; a second propagation path 622 extends from the second image point 612 to the entrance pupil 606; a third propagation path 624 extends from the third image point 614 to the entrance pupil 606; and a fourth propagation path 626 extends from the fourth image point 616 to the entrance pupil 606. Those skilled in the art will be familiar with suitable methods and algorithms for performing such propagation of light waves.

[0179] It should be noted that Figure 6The optical system is simplified. In most examples, one or more optical components will be located between the point cloud 602 / display device 604 and the observer. The propagation path can pass through these one or more optical components. For example, as described above, the one or more optical components may include one or more lenses, one or more mirrors, and / or waveguides. When virtually propagating waves from the image points, the point cloud hologram calculation will need to take into account each of these optical components. In addition, Figure 6 Not drawn to scale. For example, in a head-up display for a vehicle, the distance between display device 604 and the observer may be 1 meter or more. The distance from the observer to point cloud 602 may be greater.

[0180] After simulating the propagation of the light waves, the next step of the method is to obtain complex values ​​at each point on the (virtual) display device 604 or hologram plane. The display device 604 is pixelated. Conventionally, this step of the method includes determining the complex values ​​of each pixel at the display device 604. In this example, a phase-only hologram is determined so that the phase value is extracted from the complex value of each pixel. Those skilled in the art will be familiar with determining these values. This example is iterative, iterating at each image point. In particular, in each iteration, the light wave propagates from the corresponding image point, and the complex value of each pixel at the display device 604 is determined, and then the phase value is determined. The phase value of each image point is summed, superimposed, accumulated or otherwise combined to output a hologram of the target picture.

[0181] Improved point cloud hologram calculation method

[0182] It is desirable to be able to compute point cloud holograms in real time, so that a sequence of holograms of different target pictures can be sequentially computed and displayed at a relatively high frame rate (such as, for example, at least 30 frames per second) and at a sufficiently high resolution that the reconstruction of the hologram appears to be of good quality. It is also desirable to achieve this with relatively limited computing resources. For example, a head-up display of a vehicle may have relatively limited on-board computing resources.

[0183] However, regarding Figure 6 The point cloud hologram calculation method described is computationally very expensive, which means that such real-time calculations are not practical, certainly not at high resolution and using devices with relatively limited on-board computing resources. Therefore, the inventors have designed an improved point cloud hologram calculation method described herein.

[0184] The inventors have realized that the spatial frequency of a hologram of an image point depends on the position of that image point in the point cloud hologram. In general, spatial frequency refers to the variation of the phase and / or intensity of the wavefront / hologram at different positions within the hologram. In this example, the hologram is a phase-only hologram, so the spatial frequency refers to the variation of the phase.

[0185] Fig. 7A Shown in isolation Figure 6 hologram of the first image point 610 . Figure 7B Shown in isolation Figure 6 A hologram of a third image point 614 of FIG. Fig. 7A The top pane 702 of shows the complete hologram of the first image point 610 . Fig. 7A The bottom pane 704 shows a close-up of a portion of the complete hologram of the first image point 610 . Figure 7B The top pane 712 shows the complete hologram of the third image point 614 . Figure 7B The bottom pane 714 shows a close-up of a portion of the complete hologram of the third image point 614 . Fig. 7A and 7B Both are greyscale and represent how the phase delay applied to the hologram varies spatially. In this example, the phase varies from dark (black), representing a phase delay of 0, to light (white), representing a phase delay of 2pi.

[0186] As in Fig. 7A and 7B As can be seen in the figure, the spatial frequency of the hologram of the first image point 610 (e.g. Fig. 7A ) is much higher than the spatial frequency of the third image point 614 (as shown Figure 7B In other words, Fig. 7A The phase ratio of the hologram Figure 7B The phase of the hologram changes much faster. Fig. 7A and 7B This difference in spatial frequency becomes particularly apparent when viewing the bottom panes 704, 714. Fig. 7A In the bottom pane 704 of FIG. , the phase changes by approximately pi between adjacent pixel columns. For example, see Fig. 7A The difference in darkness between the first column of pixels 706 and the second column of pixels 707 in FIG. Figure 7B The bottom pane 714 (which is Fig. 7A In the bottom pane 704 of FIG. 706 (which is at the same scale), the phase changes more gradually. The phase changes very smoothly from approximately 0 to 2pi along the entire length of the bottom pane 714.

[0187] A pixelated display device, such as display device 604, has a field of view that is limited by the wavelength of light and, more importantly for the context of the present disclosure, by the pixel pitch (or pixel size) of the display device. As the pixel pitch decreases, the field of view increases. Display device 604 is only able to holographically reconstruct points within the field of view (which corresponds to the maximum diffraction angle of the display device). The hologram of the first image point 610 has a very high spatial frequency because the first image point 610 is toward the edge of the point cloud 602, which in this example corresponds to being close to the edge of the field of view of display device 604, and such that the propagation axis 620 passes through display device 604 at an angle close to the diffraction limit of display device 604. The high spatial frequency is required to diffract the light incident thereon to form the reconstructed first image point 610 near the edge of the field of view (i.e., close to the diffraction limit of display device 604). In contrast, image point 610 is toward the center of point cloud 602, and the associated wave propagates substantially perpendicularly to display device 604. This means that the spatial frequency is relatively very low (when compared to the hologram of the first image point 610 ).

[0188] The inventors have recognized that when the spatial frequency of a hologram is low, the hologram may be computed at a lower resolution than the native resolution of the display device 604. In other words, a hologram of low spatial frequency may be undersampled, where only a portion of the phase values ​​are computed. For example, there may be one or more empty pixel values ​​between each phase value. The inventors have recognized that due to the slowly and smoothly varying nature of the phase of such a hologram, it is possible to accurately interpolate between the computed phase values ​​(without aliasing issues) to upscale the hologram to the full resolution of the display device 604. The inventors have exploited this concept in their proposed improved method for computing point cloud holograms to provide a counterintuitive but computationally very efficient method.

[0189] The core concept of the method according to the present disclosure is to modify the propagation path (at least for image points with associated high spatial frequencies). In particular, instead of propagating the wave from the image point along the propagation path towards the entrance aperture (so that the propagation path forms a relatively large angle with the display device), a different propagation path is used. The actual propagation path is arranged to form a reduced angle with the display device 804. This means that the starting point of the propagation path is not the image point in the target image. Another way to describe this is to effectively move the image point in the target image. Fig. 8A and 8B This is demonstrated relative to a single image point.

[0190] Fig. 8A and 8B Both represent point cloud hologram calculations of a single image point. Fig. 8A and 8B Both include point cloud 802 and display device 804. The entrance pupil is not Fig. 8A and 8B Shown in. Fig. 8A The point cloud includes Figure 6 , which has a ray path 820 extending therefrom. The ray path 820 corresponds to Figure 6 The propagation path 620, and in this example, is from the first image point 610 through the display device 804 to the entrance aperture ( Fig. 8A The waves of the hologram do not propagate along the light path 820. Fig. 8A An angle 822 formed by the ray path 820 and the normal of the display device 804 is shown. The angle 822 is Fig. 8A The important point is that the angle is non-zero and close to the diffraction limit of the display device 804.

[0191] Figure 8B The actual propagation path 830 (rather than the ray path 820) used in the point cloud hologram calculation is shown. The propagation path 830 is arranged substantially perpendicular to the display device 804. In other words, the angle 822 has been reduced (to substantially zero). This means that the propagation path 830 does not extend from the first image point 610. Instead, the propagation path 830 has a Figure 8B The starting point 832 is indicated by the dashed circle in FIG. Once the wave has propagated along the propagation path 830, the complex value and phase value are determined on the display device 604 or the hologram plane in a manner familiar to those skilled in the art. The complex / phase value to be calculated is actually the value of the hologram of the image point at the starting point 832 of the propagation path 830, rather than the value of the image point in the position of the first image point 610. Therefore, as part of the hologram calculation method, it can be said that the first image point 610 is effectively moved to the starting point 832. This movement is determined by Figure 8B The hologram at the image point at the starting point 832 will have a relatively very low spatial frequency and will in fact look similar to Figure 7B hologram.

[0192] Because in Figure 8B The hologram computed in (by propagating waves along propagation path 830) has a low spatial frequency, so the hologram computation may include undersampling the complex / phase values, as described above. A portion of the undersampled hologram displayed on display device 804 is shown in FIG. Fig. 9 Shown in.

[0193] Fig. 9The grid of represents pixels of the pixelated display device 804. Each square of the array represents a pixel. The black / filled squares 904 of the array represent pixels for which complex / phase values ​​have been determined in the initial hologram calculation (after the wave propagates along the propagation path 830). The white / empty squares 906 represent pixels for which complex / phase values ​​have not yet been determined. Each black square 904 is separated from its nearest neighboring black square by four white (empty) squares 906. Thus, Fig. 9 We show how to use undersampling to significantly reduce the number of pixels for which complex / phase values ​​need to be determined during the initial hologram calculation. This saves a significant amount of computation time and results in significantly more efficient hologram calculations.

[0194] As mentioned above, in Figure 8B The hologram determined in has very low spatial frequencies. Therefore, interpolation can be used based on the known values ​​in the black squares 904 to accurately determine the values ​​of the empty squares / pixels 906. For example, a bicubic interpolation scheme can be used for interpolation. This would not be possible if the waves calculated for the hologram propagated along the ray path 820 instead of the propagation path 830, because the high spatial frequencies of such a hologram would have significant aliasing, making interpolation impractical.

[0195] In the absence of further processing, such as Figure 8B The hologram calculated and displayed on the display device and properly illuminated will form a reconstruction of the first image point 810 substantially centered in the field of view of the system. This does not coincide with the position of the first image point 810 in the target image, which (e.g. Fig. 8A and 8B 802 ). The inventors have recognized that the original or intended position in the point cloud 802 can be restored by further processing the hologram to translate the reconstructed points. For example, one or more grating functions or phase ramp functions can be added to the hologram. In some examples, a first grating function can be added to the hologram to translate the reconstructed points in a first dimension (e.g., the x-direction), and a second grating function can be added to the hologram to translate the reconstructed points in a second dimension (e.g., the y-direction). The use of phase ramp functions to translate image points is familiar to those skilled in the art. Selecting a gradient or phase ramp function that compensates for the reduction in the angle of the propagation path 830 is straightforward because the reduction in angle is already known from the hologram calculation process. Therefore, an inverse / compensating phase ramp function can be easily determined / selected for the reduction.

[0196] To further describe the improved method for calculating the hologram of the target image, Figures 10 to 12 Indicates that for a specific target image (rather than Figure 8B The steps of calculating the hologram of a general single point 610). Fig.10A target picture 1000 of this example is shown. Figure 6 , Fig. 8A and Figure 8B The point cloud is shown in the yz plane. Fig.10 The target image 1000 is instead in the xy plane. Therefore, Fig.10 The view of the target image 1000 in is actually a front view (not a side view) of the target image 1000 from the viewpoint of the observation system. Fig.10 The target image 1000 in FIG. 1 appears to be two-dimensional. However, in some examples, the target image 1000 may also have depth in the z direction and thus may be three-dimensional.

[0197] Fig.10 The target image 1000 is used for a head-up display of a vehicle and therefore includes information useful to the vehicle driver. This includes speed information, the vehicle's state of charge, and direction information. The target image 1000 is a cloud of points (or point cloud). However, because the target image 1000 is high resolution, it cannot be viewed in a Fig.10 The scale of the target image 1000 shown distinguishes the individual image points.

[0198] In this example, an initial step in the method for computing a hologram includes dividing a target picture into a plurality of parts or cells. Fig.11 A target picture 1000 is shown divided into a cell array 1100. In this example, the cell array 1100 includes 12×9 cells extending in the x and y directions. In this example, dividing the target picture into a plurality of cells includes defining an angular range from an observer to the target picture (using ray tracing). It should be clear that this step is performed virtually. Adjacent angular ranges are adjacent to each other (adjoin each other) and do not overlap. In this example, each angular range has equal range in the x and y directions.

[0199] The method also includes identifying which image points of the target image 1000 fall into which cell. In this example, some cells (such as cell 1102) do not contain any image points. Other cells (such as cell 1104) do contain image points. Cells that do not contain any image points can be effectively ignored in the method. The image points contained in a single cell can be referred to as a subset of image points.

[0200] In this example, the method for calculating a hologram then includes calculating a sub-hologram for each cell (or at least each cell containing an image point). In other words, the method includes calculating a sub-hologram for a subset of image points. The calculation of the first sub-hologram of the first cell 1106 will be described herein. Fig.11 The first cell 1106 in FIG. 1 is shown only in gray shading to highlight the cell. Fig.11As can be seen in FIG. 1 , the first unit 1106 comprises an image point that forms part of a shape outlining the directional information in the target picture.

[0201] Calculating or determining a first sub-hologram 1106 of a first subset of image points contained in the first unit includes: for each image point of the first subset, propagating a wave along a propagation path whose angle with the display device is reduced relative to the angle formed by the corresponding light path, and then determining a contribution of the wave at the display device for each image point of the first subset.

[0202] This is about Figure 8B is the same as described. Thus, instead of each propagation path starting at the location of the image point in the first unit 1106, each propagation path starts at a location closer to the center of the target image 1000. In this particular example, the angle of each propagation path is reduced by the same amount. The amount of reduction is selected so that the propagation path of the central image point of the first unit 1106 has a starting point at the center of the target image 1000. The result of reducing the angle of the propagation path of the image point of the first unit 1106 in this way is that the first unit 1106 is effectively moved to the center of the target image 1000. Another way of saying this is that the determined first sub-hologram is a hologram of the first unit 1106 as if the first unit 1106 was located at the center of the target image 1000. Therefore, without further processing, if the first sub-hologram is displayed on a display device and is appropriately illuminated, the holographic reconstruction of the first unit 1106 will be located at the center of the field of view, rather than at the location indicated in the target image 1000. This is in Fig.12 It is shown in Fig.12 The contents of the first cell 1106 are shown substantially in the center.

[0203] As previously described, when the propagation path used in the calculation of a hologram (or sub-hologram) has a reduced angle (e.g., substantially equal to zero), the hologram will have a low spatial frequency. In this example, the sub-hologram is a hologram of multiple image points (i.e., the image points that make up the first unit 1106). Therefore, the sub-hologram will effectively include a superposition of multiple individual holograms for each individual point. The hologram for each individual point is similar to Figure 7B (having a low spatial frequency), but will vary slightly because the propagation path for each point will be slightly different (because the angle will be slightly different for different points within the unit). In other words, the hologram for each individual point will vary slowly and have a low spatial frequency. As will be appreciated by those skilled in the art, the sum or superposition of multiple slowly varying functions is also a slowly varying function. Therefore, the sub-holograms for multiple image points of the first unit 1106 will also have a low spatial frequency. This means that the sub-holograms may be undersampled and interpolated, as described with respect to Fig. 9 as described.

[0204] Once the sub-holograms are interpolated, two grating functions are added to the sub-holograms to compensate for the reduction in angle of the propagation path. In this example, a first grating function is added to the first sub-hologram to compensate for the reduction in angle in the x-direction, and a second grating function is added to the first sub-hologram to compensate for the reduction in angle in the y-direction. The effect of the first grating function and the second grating function on the reconstruction of the first sub-hologram (combined with the first grating function and the second grating function) is given by Fig.12 In particular, Fig.12 The first arrow 1202 in represents the translation or shift caused by the first grating function, and Fig.12 The second arrow 1204 in represents the translation or shift caused by the second grating function. Assuming that the original reduction amount (of the angle of the propagation path during the hologram calculation) is known and is the same for all image points in the first unit 1106, it is simple to select the first grating function and the second grating function that compensate for this known amount. In addition, the first grating function and the second grating function can be applied to the sub-hologram of the first unit 1106 as a whole (i.e., at once), without the need to apply the grating functions to the image points individually.

[0205] The output described above is a sub-hologram of the first unit 1106 combined with one or more grating functions. It should be clear to those skilled in the art that this process can be repeated for each unit (or at least for each unit including one or more image points). A sub-hologram (combined with the corresponding one or more grating functions) will be determined at each iteration. The sub-holograms can be combined (or accumulated) to form a hologram of the complete target picture. The hologram can be said to include information of the sub-hologram of each unit (combined with the appropriate grating function).

[0206] Additional Features

[0207] The methods and processes described herein may be embodied on a computer-readable medium. The term "computer-readable medium" includes media arranged to store data temporarily or permanently, such as random access memory (RAM), read-only memory (ROM), buffer memory, flash memory, and cache memory. The term "computer-readable medium" should also be deemed to include any medium or combination of media capable of storing instructions executed by a machine, such that when the instructions are executed by one or more processors, the machine causes the machine to perform, in whole or in part, any one or more of the methods described herein.

[0208] The term "computer-readable medium" also encompasses cloud-based storage systems. The term "computer-readable medium" includes, but is not limited to, one or more tangible and non-transitory data storage repositories (e.g., data volumes) in the example form of solid-state memory chips, optical disks, magnetic disks, or any suitable combination thereof. In some example embodiments, instructions for execution may be conveyed by a carrier medium. Examples of such carrier media include transient media (e.g., a propagated signal conveying the instructions).

[0209] It will be apparent to those skilled in the art that various modifications and variations may be made without departing from the scope of the appended claims. The present disclosure covers all modifications and variations within the scope of the appended claims and their equivalents.

Claims

1. A hologram engine for calculating a hologram of a target picture for display on a display device of an optical system, the target picture comprising a plurality of image points, wherein each image point is defined by a first path from the image point to an entrance pupil of an observer, each first path being at an angle to the display device; The hologram engine is arranged to determine a first sub-hologram of a first subset of the image points of the target picture by: For each image point in the first subset: propagating the wave along a second path, the second path forming an angle with the display device that is reduced relative to an angle formed by a corresponding first path; as well as A contribution of the wave at the display device is determined for each image point of the first subset.

2. The hologram engine according to claim 1, wherein: The hologram engine is arranged to apply one or more grating functions to the first sub-hologram, the one or more grating functions being arranged to compensate for a reduction in an angle of each second path of the image points of the first subset.

3. A hologram engine according to claim 1 or 2, wherein the angle formed by the second path associated with each image point of the first subset and the display device is reduced by substantially the same amount.

4. A hologram engine according to any one of the preceding claims, wherein: The second path of at least some of the image points of the first subset does not pass through the entrance pupil.

5. A hologram engine according to any one of the preceding claims, wherein each second path of the image points of the first subset is arranged such that a corresponding wave does not pass through a corresponding image point.

6. A hologram engine according to any of the preceding claims, wherein each second path of the image points of the first subset passes through a first portion of the target image, wherein a distance between the first portion and a center of the target image is less than a distance between at least one image point of the first subset and the center.

7. A hologram engine according to any one of the preceding claims, wherein: Each second path of the image points of the first subset is arranged as if the corresponding image point had been moved within the target picture closer to the center of the target picture.

8. A hologram engine according to any one of the preceding claims, wherein: The reduced angle of at least one of the second paths is substantially equal to zero.

9. A hologram engine according to any one of the preceding claims, wherein the hologram engine is arranged to initially undersample the first sub-hologram so that the value of the first sub-hologram is initially determined for a subset of pixels of the display device, wherein there are one or more undetermined pixels between each determined sub-hologram pixel.

10. A hologram engine according to claim 9, wherein the hologram engine is arranged to interpolate the undersampled first sub-hologram in at least a first dimension of the display device to determine values ​​of at least some non-determined values ​​of the first sub-hologram.

11. A hologram engine according to any one of the preceding claims, wherein: The hologram engine is arranged to divide the target picture into at least n subsets of image points, wherein n is greater than or equal to ten.

12. The hologram engine according to claim 11, wherein: For each of n subsets of image points, the hologram engine is arranged to: determine an nth sub-hologram of the nth subset by determining, for each image point of the nth subset, a contribution of the wave at the display device; Each wave propagates along a second path that forms a reduced angle with a normal to the display device relative to an angle formed by the corresponding first path.

13. The hologram engine of claim 12, wherein the angle formed by the second path associated with each image point within the nth subset and the display device is reduced by substantially the same amount.

14. The hologram engine according to claim 12 or 13, wherein: The hologram engine is arranged to apply one or more grating functions to the sub-hologram of each of the n subsets, the one or more grating functions being arranged to compensate for the reduced angle of the second path for the nth subset.

15. The hologram engine according to any one of claims 12 to 14, wherein: The amount by which the angle formed by each second path is reduced is different for different subsets.

16. The hologram engine according to any one of claims 12 to 15, wherein: The hologram engine dividing the target image into at least n subsets of image points includes: the hologram engine dividing the target image into multiple angle ranges.

17. A method of computing a hologram of a target picture for display on a display device of an optical system, the target picture comprising a plurality of image points, wherein each image point is defined by a first path from the image point to an entrance pupil of an observer, each first path being at an angle to the display device, the method comprising: determining a first sub-hologram of a first subset of the image points of the target picture; Wherein determining the first sub-hologram comprises, for each image point of the first subset: propagating the wave along a second path, the second path forming an angle with the display device that is reduced relative to an angle formed by a corresponding first path; as well as A contribution of the wave at the display device is determined for each image point of the first subset.

18. The method according to claim 17, wherein: The method is a computer-implemented method.

19. The method according to claim 17 or 18, further comprising applying one or more grating functions to the first sub-hologram, the one or more grating functions being arranged to compensate for a reduction in the angle of the second path of the image points of the first subset.

20. The method according to any one of claims 17 to 19, wherein: The method comprises: dividing the target picture into at least n subsets of image points, wherein n is greater than or equal to 10; and Wherein, for each of the n subsets of image points, the method comprises: determining an nth sub-hologram of the nth subset; Wherein determining each of the n sub-holograms comprises, for each image point of the nth subset: propagating a wave along a second path, the second path forming a reduced angle with a normal to the display device relative to an angle formed by a corresponding first path; A contribution of the wave at the display device is determined.

21. An optical system comprising a display device for displaying a hologram and a processor adapted to perform the steps of the method according to any one of claims 19 to 20.

22. A computer program comprising instructions which, when executed by a computer, cause the computer to perform the steps of the method according to any one of claims 19 to 20.

23. A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to perform the steps of the method according to any one of claims 19 to 20.

Images