Picture distortion

A real-time distortion map calculation method for holographic projectors addresses temperature and wavelength-induced distortions by modifying a single map, enhancing image quality and efficiency in holographic projections.

JP2024167160APending Publication Date: 2024-12-03ENVISICS LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2024065215
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-04
Filing Date
2024-04-15
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Holographic projectors experience distortion due to temperature changes and wavelength variations, leading to misalignment and reduced image quality, especially in polychromatic reconstructions, which existing methods address inefficiently through resource-intensive calibration and memory storage.

Method used

A real-time method calculates a distortion map by modifying a single predetermined map using a scalable vector array to compensate for current temperature and wavelength changes, allowing efficient distortion correction without extensive characterization.

Benefits of technology

This method ensures accurate and efficient distortion compensation in holographic projections across varying temperatures and wavelengths, improving image quality and reducing computational and memory requirements.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024167160000001_ABST
    Figure 2024167160000001_ABST
Patent Text Reader

Abstract

To calculate a distortion map for pre-distorting a target image to compensate for distortions caused by a holographic projector.SOLUTION: A target image projected by a holographic projector is distorted to compensate for a temperature change. Each mapping is for transforming an array of two-dimensional coordinates. An array of vectors including a vector for compensating for distortion at a predetermined temperature is received. Each vector represents a calibrated change of a corresponding two-dimensional coordinate over a predetermined temperature range. A scaling factor is determined based on a difference between a current temperature and the predetermined temperature. A scaled vector is output and applied to the mapping so as to calculate a modified map based on the current temperature and to output the modified map.SELECTED DRAWING: Figure 9
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to holographic projection of a target image. More specifically, the present disclosure relates to a method for calculating a distortion map for pre-distorting a target image to compensate for distortion caused by a holographic projector. Even more specifically, the present disclosure relates to said method, which is a method that executes in real time. Some embodiments relate to a holographic projector, an image generating unit, or a head-up display. [Background technology]

[0002] Light scattered from an object contains both amplitude and phase information. This amplitude and phase information can be captured, for example, by well-known interference techniques on a photosensitive plate to form a holographic recording or "hologram" containing interference fringes. The hologram can be reconstructed by illumination with appropriate light to form a two- or three-dimensional holographic reconstruction or reconstructed image that represents the original object.

[0003] Computer-generated holography may numerically simulate interference processes. Computer-generated holograms may be calculated by techniques based on mathematical transformations such as the Fresnel transform or the Fourier transform. These types of holograms are sometimes called Fresnel / Fourier transform holograms, or simply Fresnel / Fourier holograms. Fourier holograms can be thought of as a Fourier domain / planar representation of an object, or a frequency domain / planar representation of an object. Computer-generated holograms may also be calculated by, for example, coherent ray tracing or point cloud techniques.

[0004] The computer-generated hologram may be encoded on a spatial light modulator arranged to modulate the amplitude and / or phase of incident light. Light modulation can be achieved using, for example, electrically addressable liquid crystals, optically addressable liquid crystals, or micromirrors.

[0005] A spatial light modulator typically comprises a number of individually addressable pixels, also called cells or elements. The light modulation scheme may be binary, multi-level, or continuous. Alternatively, the device may be continuous (i.e., not composed of pixels), and thus the light modulation may be continuous across the device. Spatial light modulators may be of the reflective type, meaning that the modulated light is output in reflection. Spatial light modulators may also be of the transmissive type, meaning that the modulated light is output in transmission.

[0006] A holographic projector may be provided using the systems described herein, such a projector finding application in heads-up displays (HUDs). Summary of the Invention

[0007] Aspects of the present disclosure are defined in the accompanying independent claims.

[0008] In general terms, a real-time method is provided for computing a map for processing, e.g. (pre-)distorting, a target image to be projected by a holographic projector, the map being suitable for pre-distorting the target image to compensate for a temperature, such as the current temperature of the holographic projector.

[0009] The holographic projector may include a display device, such as a spatial light modulator. The display device may be configured to display a hologram of the target image (e.g., a hologram of a pre-distorted image). The hologram may be a computer-generated hologram. The display device may be configured to spatially modulate light incident on the display device according to a hologram displayed on the display device. The holographic projector may be configured to direct or relay the spatially modulated light (spatially modulated according to the hologram) towards a viewing surface or eyebox. The holographic projector may be positioned such that the spatially modulated light forms a holographic reconstruction of the image on a playback surface. The holographic projector may be positioned such that an image (e.g., a virtual image) of the holographic reconstruction (of the target image) is visible (i.e., viewable) from the viewing surface or eyebox.

[0010] The image of the holographic reconstruction of the target image may be distorted compared to the holographic reconstruction itself. Conceptually, it is possible to consider a number of points (e.g., pixels or groups of pixels) of the holographic reconstruction and a corresponding number of points of the image of the holographic reconstruction. The distribution of points, spots, pixels or features in (the image of) the holographic reconstruction may be positionally different from the respective points, spots, pixels or features of the target image (encoded in the hologram). For example, at least some of the image points of the image of the holographic reconstruction may appear positionally translated or shifted relative to the respective image points or image features of the holographic reconstruction relative to the adjacent image points. Different image points or image features in the image of the holographic reconstruction may be shifted or translated by different amounts. In other words, the amount of translation or shift of the image points may be non-uniform across the image of the holographic reconstruction. The holographic projector may comprise one or more optical components for relaying the spatially modulated light from the display device. Examples of optical components include lenses and mirrors. The optical components may increase magnification. One source of distortion may be the result of optical misalignment of optical components, for example due to manufacturing tolerances. Distortion may change as the temperature of the holographic projector changes, as temperature changes may cause the (optical) components of the holographic projector to expand or contract, thereby changing the distortion. Temperature changes may also cause changes in the wavelength of the light emitted by the light source of the holographic projector (the light used to illuminate the display device). For example, the light source may be a coherent light source, such as a laser. The wavelength of the coherent light source / laser may change with temperature changes. Distortion may cause the image of the holographic reconstruction to appear warped and / or distorted relative to the holographic reconstruction (and / or the target image), and thus the distortion may negatively affect the viewing experience and quality (e.g., accuracy) of the individual holographic reconstructions.Image quality or accuracy can be quantified in a variety of ways known in the field of optics / imaging. Those skilled in the art of optics are familiar with these techniques, and the present disclosure is not limited to a particular method of measuring / quantifying image quality or image distortion. Thus, in a broad sense, the present disclosure relates to a method of improving image quality or reducing image distortion.

[0011] It is particularly problematic that distortions vary with wavelength in holographic projectors configured to display multicolor reconstructions, for example using multiple monochromatic channels (i.e., multiple monochromatic channels operating at different wavelengths). In such an example, each monochromatic channel may be arranged to holographically project a monochromatic image (e.g., a red, green, or blue image). The holographic projector may be configured to independently (spatially and / or temporally) project multiple monochromatic images (e.g., independently project a red image, a green image, and a blue image). A (monochromatic) holographic reconstruction associated with each channel may be formed on the playback plane. A multicolor holographic reconstruction is formed as a combination of multiple monochromatic holographic reconstructions in the playback plane. Multiple monochromatic channels may at least partially overlap and may even use the same spatial light modulator, such as in a frame sequential color scheme. Ideally, multiple monochromatic holographic reconstructions would be aligned with each other. This may ideally mean that corresponding image points of each of the monochromatic holographic reconstructions are aligned with each other in the playback plane. However, each monochromatic image / single holographic reconstruction may have different distortions, as each has a different wavelength (since the amount of distortion is wavelength dependent), which may result in complex misregistrations between the different color channels. The misregistrations may be non-uniform. The human eye is sensitive to misregistrations, and even relatively small misregistrations between image pixels of a monochromatic image may significantly degrade the perceived image quality. This problem is even more pronounced in head-up displays, which have optical playback systems designed to perform complex optical processing and meet stringent optical and physical constraints.

[0012] It is possible to compensate for distortions in the (image of) the holographic reconstruction by pre-distorting the target image before display and / or hologram computation. This may involve transforming the image points of the target image in a manner opposite to the distortions caused by the holographic projector (at a given temperature and wavelength) and then displaying a hologram of the pre-distorted target image on the display device of the holographic projector. Thus, the distortions caused by the holographic projector are already compensated for, resulting in the holographic reconstruction being displayed as intended. A distortion map may be used to pre-distort the target image. The distortion map may include multiple mappings to an array of two-dimensional coordinates. The array of two-dimensional coordinates may relate to the coordinates of the target image. Each distortion mapping may transform one of the two-dimensional coordinates. The distortion map may have been determined and verified by experiment and / or simulation to ensure that the mapping pre-distorts the target image in a manner that accurately compensates for the distortions caused by the holographic projector such that the holographic reconstruction is displayed as intended. The transformed two-dimensional coordinates may be used to pre-distort the target image.

[0013] As mentioned above, the distortion of the holographic reconstruction may depend on the current temperature of the holographic projector and / or the wavelength of the spatially modulated light (according to the respective hologram). Thus, the first distortion map may only be able to accurately pre-distort the target image if a spatially modulated light of a first wavelength is used at a first temperature. For example, the first distortion map may not be able to accurately pre-distort the target image if a spatially modulated light of a second wavelength (different from the first wavelength) is used and / or if the holographic projector is at a second temperature (different from the first temperature). A conventional solution to this problem is to program the memory of the holographic projector to store multiple distortion maps that have been determined and verified at multiple different temperatures and wavelengths. The holographic projector may be configured to use a look-up table to select an appropriate distortion map from the memory for a particular scenario. However, the inventors recognize that this solution is resource inefficient and suboptimal, since a large number of distortion maps must be determined and verified. For example, a holographic projector intended for use in a vehicle, such as part of a vehicle's head-up display, must be able to operate over a wide temperature range (which may span at least 100 degrees Celsius). To cover such a wide range, a large number of distortion maps must be determined and validated at different temperatures (e.g., for each degree Celsius of the temperature range). For a multi-color holographic projector, this must be repeated for each color channel. The inventors recognized that determining and validating such a large number of distortion maps would be very time and resource intensive. Furthermore, this process must be repeated for each model of holographic projector or when the holographic projector is used in different situations (e.g., different vehicles with different shapes). Furthermore, the inventors recognized that the memory requirements for storing such a large number of distortion maps may be prohibitive.Furthermore, the distortion map described above may be part of only one of many calibration processes performed by the holographic projector. For example, a holographic projector may need to calibrate a hologram to compensate for the shape of an optical combiner that has a complex shape, such as a vehicle windshield, and / or may adjust the hologram based on eye tracking of the user of the holographic projector. Both of these examples may require a separate calibration process (in addition to the distortion compensation that is the subject of this disclosure). Thus, the predistortion described above to compensate for wavelength and temperature changes may be only a small part of the projection / calibration process. It is therefore important to make each calibration procedure as resource-efficient as possible. This is another reason why conventional methods of fully characterizing a holographic projector at multiple temperatures and wavelengths are not optimal.

[0014] To address the problem of determining and storing a large number of distortion maps, the inventors have developed a fast, resource-efficient, and time-efficient method for computing / determining distortion maps. The method can be performed in real-time or instantaneously, for example while the holographic projector is in operation (e.g., while a computer is generating a series of holograms). The method for computing (distortion) maps according to the present disclosure requires as input only one pre-determined and pre-validated distortion map (first distortion map) per color channel (at a specific first temperature). Thus, the method advantageously avoids the need to fully characterize the holographic projector to determine distortion maps at multiple temperatures.

[0015] The real-time projection of a series of computer-generated holograms is computationally very intensive and costly. To the inventors, it was counterintuitive to add an additional real-time computational step (computing a distortion map, rather than retrieving the map from memory) to a holographic projection method. However, the inventors surprisingly discovered that the method according to the present disclosure can correct the first distortion map with simple and inexpensive computations, and the corrected map can be used to predistort the target image so that the holographic reconstruction appears as intended. The multi-color holographic reconstruction is aligned using a process / computation that is performed in real-time during display. In particular, after extensive simulations and experiments, the inventors discovered that the change in position of the coordinate array described above follows a substantially predictable relationship over a range of temperatures. In some embodiments, this is a linear relationship. The inventors realized that this relationship can be used to simply calculate the distortion map for any current temperature by modifying the first distortion map with a scaled vector that represents how the array of coordinates changes between a second temperature and a third temperature. Importantly, according to the present disclosure, this method / computational step can be performed repeatedly by a holographic projector. For example, the method steps can be performed for every frame or sub-frame of a real-time image projection system, such as one based on data streaming. The method steps can be performed / repeated when the current temperature changes by or beyond a predetermined amount. Again, this runs counter to a significant preconception in the art that computation should be minimized to achieve high quality real-time performance. The inventors have identified a simple method that is worth the additional processing burden.

[0016] In a first aspect of the disclosure, a method is provided for computing a (distortion) map in real time. The (computed) distortion map is for distorting (also referred to herein as "pre-distortion") a target image projected by a holographic projector. The (pre-)distortion of the target image is for compensating for a current temperature of the holographic projection and changes in the current temperature of the holographic projector. The method includes receiving a calibrated map. The calibrated map includes a plurality of mappings. Each mapping is for transforming two-dimensional coordinates. Each two-dimensional coordinate corresponds to one or more image points of the target image. A point or location on a two-dimensional plane defined by the two-dimensional coordinates may be referred to as a calibration point. Each calibration point may correspond to one or more image points of the target image. In an embodiment, each two-dimensional coordinate corresponds to one or more image points of the undistorted target image. Each mapping of the calibrated map may be for transforming the respective two-dimensional coordinates to compensate for distortion at a predetermined first temperature. In other words, each mapping of the calibrated map may be for pre-distorting image points of the target image to compensate for distortion caused by the projection process at the first temperature. As used herein, a "calibrated map" refers to a distortion map that has been (fully) determined and verified through experiment and / or simulation. A process of completely determining a calibrated map that can be performed is not essential to the method of the present disclosure. Thus, a method step that includes receiving a calibrated map can include receiving a predefined map. The predefined calibration map is determined when the holographic projector is at a first temperature and when the spatially modulated light has a first wavelength. The method according to the present disclosure advantageously requires the reception of only one calibrated map (at the first temperature) to calculate a (corrected) map for the current temperature (when the spatially modulated light has the first wavelength). The calibrated map may be referred to herein as a first distortion map or an initial distortion map.

[0017] The method further includes receiving an array of vectors. The array of vectors includes / consists of a vector for each two-dimensional coordinate of the array of two-dimensional coordinates. There may be a one-to-one ratio between the number of vectors in the array of vectors and the number of two-dimensional coordinates in the array of two-dimensional coordinates. Each vector can be uniquely associated with a two-dimensional coordinate and vice versa. Similarly, there may be a one-to-one ratio between the number of vectors in the array of vectors and the number of mappings in the calibrated map. Each vector can be uniquely associated with a mapping in the calibrated map and vice versa. Each vector represents a calibrated change of a respective two-dimensional coordinate (of the array of two-dimensional coordinates) over a predetermined temperature range. As used herein, a calibrated change is a respective change of a respective two-dimensional coordinate from a second temperature to a third temperature, assuming that the two-dimensional coordinates are transformed to compensate for distortions at the second and third temperatures, respectively. The temperature range includes the second temperature and the third temperature. In some embodiments, the second temperature corresponds to the lowest temperature of the (predetermined) temperature range. In some embodiments, the third temperature corresponds to the highest temperature of the (predetermined) temperature range. As with the calibrated map, the method step of determining the sequence of vectors may not be essential to the method of the present disclosure. Thus, the method step includes receiving a predetermine sequence of vectors. The sequence of vectors may be referred to herein as a vector map or a simulated map.

[0018] The method further includes receiving or determining a current temperature of the holographic projector, the current temperature sometimes referred to herein as a fourth temperature.

[0019] The method further includes determining a scaling factor based on the difference between the current (fourth) temperature and the predetermined (first) temperature. Determining the scaling factor based on the difference between the current (fourth) temperature and the predetermined (first) temperature may include additional steps or mathematical operations (in addition to determining the difference between the current temperature and the predetermined temperature). For example, the output of the difference between the current (fourth) temperature and the predetermined (first) temperature may be multiplied or divided by a constant.

[0020] The method further includes calculating a modified map based on or for the current temperature. The step of calculating the modified map includes, for each coordinate in the array of two-dimensional coordinates, multiplying a vector associated with the respective coordinate in the array of two-dimensional coordinates by a scaling factor to output a scaled vector. Then, applying the scaled vector to each mapping of the calibrated map. The step of applying the scaled vector to each mapping of the calibrated map may include adding or subtracting the scaled vector to the respective mapping. The step of applying the scaled vector to each mapping of the calibrated map may include outputting a modified mapping for transforming the respective two-dimensional coordinate to compensate for distortion of the target image when projected by the holographic projector at the current (fourth) temperature.

[0021] The method further comprises the step of outputting a modified map, the modified map constituting an array of modified mappings (computed in the previous steps of the method).

[0022] In some embodiments, the method further comprises receiving an array of two-dimensional coordinates. In some embodiments, the number of image points in the target image and the number of two-dimensional coordinates in the array of two-dimensional coordinates are in a many-to-one ratio. In other words, each two-dimensional coordinate may correspond to one or more, optionally five or more, optionally ten or more image points. Alternatively, the number of image points may be less than the two-dimensional coordinates. Advantageously, the ratio of the number of image points in the target image and the number of two-dimensional coordinates in the array of two-dimensional coordinates is many-to-one, since the (calibrated and corrected) map of the method contains as many mappings as there are two-dimensional coordinates (and the calibrated map contains as many vectors as there are two-dimensional coordinates). 22 This is because fewer mappings are calculated in the corrected map for a given number of image points, making the method less computationally expensive than if there were a larger number of mappings. Interpolation can be advantageously used to determine appropriate predistortion of image points of the target image when the mappings in the corrected map have a many-to-one correspondence to the image points.

[0023] The vectors in the vector array may be non-uniformly distributed in direction and magnitude. In general, two-dimensional coordinates corresponding to image points closer to the center of the target image are less distorted than two-dimensional coordinates closer to the edge / periphery of the target image. Typically, two-dimensional coordinates corresponding to image points on opposite sides of the target image are distorted and transformed in opposite directions. The vector array may represent these two effects. For example, the magnitude of the vectors in the vector array may increase as the distance of each two-dimensional coordinate from the center of the array increases. The direction of the vectors in the vector array corresponding to two-dimensional coordinates on a first side of the center of the array may be opposite to the direction of the vectors in the vector array corresponding to two-dimensional coordinates on a second side of the center of the array. In some embodiments, the direction of each vector in the vector array may be substantially parallel to a direction defined from the center of the array to the respective two-dimensional coordinate.

[0024] In some embodiments, the scaling factor is linearly dependent on the current / fourth temperature. After extensive simulations and experiments, the inventors surprisingly discovered that the distortion of each image point of the target image / each calibration point represented by its respective two-dimensional coordinate varies substantially linearly with temperature. In some embodiments, the step of determining the scaling factor includes determining a difference between the current temperature and a predetermined temperature and dividing the difference by a predetermined temperature range. In such embodiments, the scaling factor is also referred to as a percentage. Such a scaling factor represents the change from the first temperature to the current temperature as a percentage relative to the predetermined temperature range. If the current temperature is equal to the first temperature, the rate of change will be zero. Thus, the scaling factor will be equal to zero. Otherwise, the scaling factor may be non-zero.

[0025] In some embodiments, the step of determining the scaling factor comprises determining the scaling factor by max -T min ) where T is the current temperature, T0 is a given temperature, and T max is the highest (third) temperature in a given temperature range, T min is the lowest (second) temperature in the given temperature range. In other words, the scaling factor is (T-T0) / (T max -T min )

[0026] In some embodiments, the method further includes applying the modified map to an array of two-dimensional coordinates to output an array of modified two-dimensional coordinates. In some embodiments, the method further includes receiving a target image including a plurality of image points, each two-dimensional coordinate of the array of two-dimensional coordinates corresponding to one or more image points of the target image. Each two-dimensional coordinate of the modified array may correspond to the same respective one or more image points of the target image. In some embodiments, the method further includes warping the target image based on the modified array of two-dimensional coordinates. In some embodiments, the method further includes calculating a hologram of the warped target image.

[0027] In some embodiments, determining the current / fourth temperature may include measuring the current / fourth temperature of the holographic projector, for example using a temperature sensor. Optionally, the temperature sensor may be positioned to measure the temperature of a light source, such as a laser, of the holographic projector. In other embodiments, determining the current / fourth temperature may include inferring the current / fourth temperature of the holographic projector by measuring a wavelength of the spatially modulated light. For example, the method may include comparing a current wavelength of the spatially modulated light to an expected wavelength of the spatially modulated light. As will be appreciated by those skilled in the art, the current temperature of the holographic projector / light source of the holographic projector may be determined based on a deviation of the wavelength of the light from the expected wavelength.

[0028] As mentioned above, the method according to the present disclosure can be performed in real time. The method can be performed repeatedly during the operation of the holographic projector. In some embodiments, the holographic projector is configured to project a series of (image) frames, for example at video rates, in a manner familiar to the reader. Each frame can have a different hologram (with a different target image associated with it). In some embodiments, the method according to the present disclosure can be performed once for each frame of the series. Thus, a modified (distortion) map is recalculated for each frame of the series. The recalculated modification (distortion) map is then applied for each target image of each frame of the sequence (to pre-distort the target image), after which a hologram of the pre-distorted target image is calculated. In other embodiments, the method according to the present disclosure can be repeated only if it is detected that the current temperature has changed by an amount that exceeds a threshold. In such an example, the method may include calculating a first modification (distortion) map at an initial temperature. The method may include monitoring the current temperature for each frame of the series. If the difference between the current temperature and the initial temperature is less than the threshold, the method may include reusing the first correction (distortion) map to pre-distort the target image for the respective frame. If the difference between the current temperature and the initial temperature is greater than the threshold, the method may include recalculating the corrected map (according to the method described above). The method may then include pre-distorting the target image for the respective frame using the recalculated correction map. In some embodiments, the method is performed for at least two consecutive frames of the input stream of image frames. In some embodiments, the method is performed for every frame of the received stream of image frames.

[0029] In some embodiments, the target image is a first target image. In some embodiments, the array of two-dimensional coordinates is a first array of two-dimensional coordinates. In some embodiments, the calibrated map is a first calibrated map. In some embodiments, the array of vectors is a first array of vectors. In some embodiments, the corrected map is a first corrected map. In some embodiments, the method further includes calculating a second map for distorting a second target image projected by the holographic projector. The calculation of the second map may include receiving a second array of two-dimensional coordinates. Each two-dimensional coordinate may correspond to one or more image points of the second target image. The second target image may be different from the first target image. The calculation of the second map may include receiving a second calibrated map for the second array of two-dimensional coordinates. The second calibrated map may include a mapping for each two-dimensional coordinate of the second array to transform the respective two-dimensional coordinate to compensate for distortion at a predetermined temperature. The predetermined temperature may be the same as or different from the predetermined first temperature described above. The calculation of the second map may include receiving a second array of vectors, the second array of vectors including a vector for each two-dimensional coordinate (in the second array of two-dimensional coordinates). Each vector may represent a change in position of the respective two-dimensional coordinate over a predetermined temperature range. The calculation of the second map may include, for each coordinate in the second array of two-dimensional coordinates, calculating a second correction map based on the current temperature as follows: multiplying a vector associated with each coordinate in the second array of two-dimensional coordinates by a scaling factor to output a scaled vector; and applying the scaled vector to each mapping in the second calibrated map to output the second correction map.

[0030] A mapping of the first calibrated map and the vectors of the first vector array can be determined when a first wavelength is used for the holographic projection of a first target image, and a mapping of the second calibrated map and the vectors of the second calibrated map can be determined when a second wavelength (different from the first wavelength) is used for the holographic projection of a second target image.

[0031] The method may further include applying the second correction map to the second two-dimensional coordinate array to output a second corrected two-dimensional coordinate array. The method may further include distorting the second target image based on the correction map. The method may further include calculating a second hologram of the distorted second target image.

[0032] In a second aspect, a method of computing a mapping of a map is provided, the mapping being for distorting at least a portion of a target image projected by a holographic projector to compensate for changes in a current temperature of the holographic projector. The method includes receiving a first mapping for transforming a first two-dimensional coordinate to compensate for the distortion at a predetermined (first) temperature. The method further includes receiving a first vector representing a calibrated change in the first two-dimensional coordinate over a predetermined temperature range (e.g., a change in the first two-dimensional coordinate between a second / minimum temperature and a third / maximum temperature). The method further includes receiving or determining a current (fourth) temperature of the holographic projector. The method further includes determining a scaling factor based on a difference between the current temperature and the predetermined temperature. The method further includes computing a first modified mapping based on / to compensate for the current temperature (by modifying the first mapping). The first modified mapping can form part of the first modified map. The step of calculating the first modified mapping includes multiplying the first vector by a scaling factor to output a scaled vector. The method further includes applying the scaled vector to the first mapping. The method further includes outputting the modified first mapping to transform the first two-dimensional coordinates for purposes of compensating for distortion of a portion of the target image when projected by the holographic projector at the current (fourth) temperature.

[0033] The method according to the second aspect may be repeated to calculate any number of mappings of the first map. For example, the first map may include multiple first mappings. Each first mapping may be for transforming each two-dimensional coordinate. Each two-dimensional coordinate may form part of an array of two-dimensional coordinates. The method according to the second aspect may be repeated for multiple first mappings of the first map. In some embodiments, the method according to the second aspect may be repeated for a subset or part of the first mappings of the first map. In some embodiments, the method according to the second aspect may be repeated for all first mappings of the first map. Each time the method is repeated, the method may include receiving a further vector. For example, if the method includes receiving a second mapping (for transforming the second two-dimensional coordinate), the method includes receiving a second vector (representing a change in the second two-dimensional coordinate).

[0034] In a third aspect, a holographic projector is provided that includes a display device configured to display a hologram of an image and spatially modulate incident light according to the hologram, the holographic projector configured to form a holographic reconstruction of the image at a reconstruction plane. The holographic projector further includes a controller that includes a memory in which a calibrated map and an array of vectors are stored. The calibrated map includes a plurality of mappings. Each mapping is for transforming a respective two-dimensional coordinate of the array of two-dimensional coordinates to compensate for distortion at a predetermined temperature, each two-dimensional coordinate corresponding to one or more image points of the target image. The array of vectors includes a vector for each two-dimensional coordinate, each vector representing a calibrated change of the respective two-dimensional coordinate over a predetermined temperature range. The controller is further configured to determine a current temperature of the holographic projector (in particular, a temperature of the light source). For example, the holographic projector may include a temperature sensor configured to measure a temperature of the light source of the holographic projector. The controller may be configured to receive a signal from the temperature sensor. The controller may be further configured to determine a scaling factor based on a difference between the current temperature and the predetermined temperature. The controller may be further configured to calculate a modified map based on the current temperature by, for each coordinate in the array of two-dimensional coordinates, multiplying a vector associated with the respective coordinate in the array of two-dimensional coordinates by a scaling factor to output a scaled vector and applying the scaled vector to each mapping in the calibrated map. The controller may be further configured to output the modified map.

[0035] Features and advantages described in relation to one aspect of the present disclosure may also be applicable to other aspects of the present disclosure, in particular the features and advantages described in relation to the method of the first aspect may be applied to the method of the second aspect and the holographic projector of the third aspect.

[0036] In this disclosure, the term "replica" is used simply to reflect that the spatially modulated light is split so that the complex light field is directed along multiple different optical paths. The word "replica" is used to refer to each occurrence or instance of the complex light field after a replication event, such as partial reflection / transmission by a pupil dilator. Each replica travels along a different optical path. Some embodiments of the present disclosure relate to the propagation of light encoded with a hologram rather than an image, i.e., light spatially modulated with a hologram of an image rather than the image itself. Thus, it can be said that multiple replicas of a hologram are formed. Those skilled in the art of holography will appreciate that the complex light field associated with the propagation of holographically encoded light changes with propagation distance. The use of the term "replica" in this disclosure is independent of propagation distance, so that two branches or paths of light associated with a replication event are still referred to as "replicas" of each other even if the branches have different lengths and even if the complex light field has evolved differently along each path. That is, two complex light fields, even if they are associated with different propagation distances, are still considered "replicas" according to this disclosure if they result from the same replication event or series of replication events.

[0037] A "diffracted light field" or "diffractive light field" according to the present disclosure is a light field formed by diffraction. A diffracted light field can be formed by illuminating a corresponding diffraction pattern. According to the present disclosure, an example of a diffraction pattern is a hologram, and an example of a diffracted light field is a holographic light field, or a light field forming a holographic reconstruction of an image. The holographic light field forms a (holographic) reconstruction of an image on a playback surface. The holographic light field propagating from a hologram to a playback surface can be said to be composed of light encoded in the hologram or light within the hologram region. The diffracted light field is characterized by a diffraction angle determined by the minimum feature size of the diffracting structure and the wavelength of light (of the diffracted light field). According to the present disclosure, a "diffracted light field" can also be said to be a light field forming a reconstruction on a plane spatially separated from a corresponding diffracting structure. Disclosed herein is an optical system for propagating a diffracted light field from a diffracting structure to an observer. The diffracted light field can form an image.

[0038] The term "hologram" is used to refer to a recording that contains amplitude information, phase information, or a combination thereof about an object. The term "holographic reconstruction" is used to refer to an optical reconstruction of an object formed by illuminating a hologram. Because the holographic reconstruction is a real image and spatially separated from the hologram, the system disclosed herein is described as a "holographic projector". The term "replay field" is used to refer to the 2D region in which the holographic reconstruction is formed and perfectly focused. When a hologram is displayed on a spatial light modulator made of pixels, the replay field is repeated in the form of multiple diffraction orders, each diffraction order being a replica of the 0th order replay field. The 0th order replay field is the brightest replay field, and therefore typically corresponds to the preferred or 1st order replay field. Unless otherwise noted, the term "replay field" should be interpreted as referring to the 0th order replay field. The term "replay plane" is used to refer to a plane in space that contains all the replay fields. The terms "image", "replay image", and "image area" refer to the area of ​​the replay field that is illuminated by the light of the holographic reconstruction. In some embodiments, an "image" may be made up of discrete spots called "image spots" or, for convenience only, "image pixels."

[0039] The terms "encoding", "writing", or "addressing" are used to describe the process of providing a number of pixels of an SLM with a number of control values ​​that respectively determine the modulation level of each pixel. The pixels of the SLM can be said to be configured to "display" a light modulation distribution in response to receiving the number of control values. The SLM can therefore be said to "display" a hologram, and a hologram can be thought of as an array of light modulation values ​​or levels.

[0040] It has been found that holographic reconstructions of acceptable quality can be formed from "holograms" that contain only phase information related to the Fourier transform of the original object. Such holographic recordings are sometimes called phase-only holograms. Although the embodiments relate to phase-only holograms, the disclosure is equally applicable to amplitude-only holography.

[0041] The present disclosure is equally applicable to forming a holographic reconstruction using amplitude and phase information associated with the Fourier transform of the original object. In some embodiments, this is achieved by complex modulation using a so-called fully complex hologram, which contains both amplitude and phase information associated with the original object. Such a hologram is sometimes called a fully complex hologram, because the value (gray level) assigned to each pixel of the hologram has an amplitude and a phase component. The value (gray level) assigned to each pixel can be expressed as a complex number having both an amplitude and a phase component. In some embodiments, a fully complex computer-generated hologram is calculated.

[0042] The phase of a pixel of a computer-generated hologram or spatial light modulator may also be referred to as a phase value, a phase component, phase information, or simply an abbreviation of "phase delay." That is, the phase value described is actually a number (e.g., ranging from 0 to 2π) that represents the amount of phase delay provided by that pixel. For example, a pixel of a spatial light modulator described as having a phase value of π / 2 will delay the phase of the received light by π / 2 radians. In some embodiments, each pixel of a spatial light modulator is operable at one of a number of possible modulation values ​​(e.g., phase delay values). The term "gray level" may be used to refer to a number of available modulation levels. For example, the term "gray level" may be used for convenience to refer to a number of phase levels available in a phase-only modulator, even if the different phase levels do not provide different shades of gray. The term "gray level" may also be used for convenience to refer to a number of complex modulation levels available in a complex modulator.

[0043] Thus, a hologram is composed of an array of gray levels, i.e., an array of light modulation values, such as an array of phase delay values ​​or complex modulation values. A hologram is also considered a diffraction pattern, since it is a pattern that is displayed on a spatial light modulator and causes diffraction when illuminated with light having a wavelength comparable to, and generally shorter than, the pixel pitch of the spatial light modulator. Reference is made herein to combining a hologram with other diffraction patterns, such as a diffraction pattern that acts as a lens or a grating. For example, a diffraction pattern that acts as a grating can be combined with a hologram to translate the replay field on the replay plane, or a diffraction pattern that acts as a lens can be combined with a hologram to focus the holographic reconstruction on the replay plane in the near field.

[0044] In the following detailed description, different embodiments and groups of embodiments may be disclosed separately, but any feature of any embodiment or group of embodiments may be combined with any other feature or combination of features of any embodiment or group of embodiments, i.e., all possible combinations and permutations of features disclosed in this disclosure are contemplated. [Brief description of the drawings]

[0045] Specific embodiments will now be described, by way of example only, with reference to the following figures: [Figure 1] FIG. 1 is a schematic diagram showing a reflective SLM generating a holographic reconstruction on a screen. [Figure 2A] FIG. 1 illustrates the first iteration of an example Gurchberg-Saxton algorithm. [Figure 2B] The second and subsequent iterations of an example Gertchberg-Saxton algorithm are shown. [Figure 2C] This figure shows alternatives for the second and subsequent iterations of the Gertchberg-Saxton type algorithm example. [Diagram 3] FIG. 1 is a schematic diagram of a reflective LCO SLM. [Figure 4] An example of a HUD in a vehicle is shown. [Diagram 5] 1 illustrates the process of pre-distorting a target image for holographic projection. [Figure 6A] FIG. 6 illustrates a pre-distorted target image after a distortion map has been applied to the target image of FIG. 5. [Figure 6B] 6B depicts a virtual image of a holographic reconstruction of the pre-distorted target image of FIG. 6A. [Figure 7] 1 shows a flow diagram illustrating a method according to the present disclosure. [Figure 8] 1 shows a schematic of one vector being scaled. [Figure 9] FIG. 13 is a schematic diagram illustrating a process of pre-distorting a target image prior to computing a hologram of the target image using a modified distortion map computed using a method according to the present disclosure. [Figure 10] FIG. 1 shows a schematic diagram of a multi-color holographic projector with multiple color channels. [Figure 11] 1 illustrates the misalignment of image points in the holographic reconstruction from the first (green) and second (blue) channels. The same reference numbers are used throughout the drawings to refer to identical or similar parts. Detailed Description of the Embodiments

[0046] The present invention is not limited to the embodiments described below, but rather encompasses the full scope of the appended claims, i.e., the present invention may be embodied in different forms and should not be construed as being limited to the described embodiments set forth for illustrative purposes.

[0047] A structure described as being formed on or above / below another structure should be interpreted to include the structures in contact with each other and even the structures having a third structure disposed therebetween.

[0048] When describing temporal relationships, for example when the temporal order of events is described as "after," "succeeding," "next," "before," etc., the disclosure should be considered to include sequential and non-sequential events unless otherwise specified. For example, non-sequential events are included unless words such as "just," "immediately," "directly," etc. are used.

[0049] In this specification, terms such as "first", "second", etc. may be used to describe various elements, but these elements are not limited by these terms. These terms are used only to distinguish one element from another element. For example, a first element can be called a second element, and similarly, a second element can be called a first element, without departing from the scope of the appended claims.

[0050] Features of different embodiments may be combined or combined with each other in part or in whole and may interoperate in various ways. Some embodiments may be performed independently of each other or together in a co-dependent manner.

[0051] optical configuration FIG. 1 shows an embodiment in which a computer-generated hologram is encoded on a single spatial light modulator. A computer-generated hologram is a Fourier transformed reconstruction of an object. Thus, a hologram can be said to be a Fourier domain, frequency domain, or spectral domain representation of an object. In this embodiment, the spatial light modulator is a reflective liquid crystal on silicon "LCOS" device. The hologram is encoded on the spatial light modulator, and a holographic reconstruction is formed on a replay field, e.g., a receiving surface such as a screen or diffuser.

[0052] A light source 110, e.g. a laser or laser diode, is arranged to illuminate the SLM 140 through a collimating lens 111. The collimating lens makes the wavefront of the light approximately planar and makes it incident on the SLM. In FIG. 1, the direction of the wavefront is off-normal (e.g. 2 or 3 degrees away from true orthogonal to the plane of the transparent layer). However, in other embodiments, a substantially planar wavefront is provided at normal incidence and a beam splitter device is used to separate the input and output optical paths. In the embodiment shown in FIG. 1, the light from the light source is arranged to be reflected off the mirrored back surface of the SLM and interact with the light modulating layer to form an output wavefront 112. The output wavefront 112 is applied to an optical system including a Fourier transform lens 120 having a focus on a screen 125. More specifically, the Fourier transform lens 120 receives the modulated beam of light from the SLM 140 and performs a frequency-space transformation to generate a holographic reconstruction at the screen 125.

[0053] In particular, in this type of holography, each pixel of the hologram contributes to the overall reconstruction: there is no one-to-one correlation between a specific point on the replay field (or image pixel) and a specific light-modulating element (or hologram pixel). In other words, the modulated light leaving the light-modulating layer is distributed throughout the entire replay field.

[0054] In these embodiments, the position of the holographic reconstruction in space is determined by the refractive power (light-gathering power) of the Fourier transform lens. In the embodiment shown in FIG. 1, the Fourier transform lens is a physical lens. That is, the Fourier transform lens is an optical Fourier transform lens, and the Fourier transform is performed optically. Any lens can function as a Fourier transform lens, but the performance of the lens limits the accuracy of the performed Fourier transform. Those skilled in the art understand how to use lenses to perform an optical Fourier transform.

[0055] Hologram Calculation In some embodiments, the computer-generated hologram is a Fourier transform hologram, or simply a Fourier hologram or Fourier-based hologram, where an image is reconstructed in the far field by exploiting the Fourier transform properties of a positive lens. A Fourier hologram is calculated by Fourier transforming the desired light field in the reconstruction plane back to the lens plane. A computer-generated Fourier hologram can be calculated using the Fourier transform.

[0056] The phase information associated with the object is effectively "retrieved" from the amplitude-only information in the spatial domain. In some embodiments, the computer-generated hologram is calculated from the amplitude-only information using the Gertchberg-Saxton algorithm or a variation thereof.

[0057] The Gertchberg-Saxton type algorithm considers the situation where the intensity cross-sections of a light beam in planes A and B, IA(x,y) and IB(x,y), respectively, are known and IA(x,y) and IB(x,y) are related by a single Fourier transform. Using the specified intensity cross-sections, approximations of the phase distributions in planes A and B, ΨA(x,y) and ΨB(x,y), respectively, are found. The Gertchberg-Saxton type algorithm follows an iterative process to find a solution to this problem. More specifically, the Gertchberg-Saxton type algorithm iteratively applies spatial and spectral constraints while iteratively transferring data sets (amplitude and phase) representing IA(x,y) and IB(x,y) between the spatial and Fourier (spectral or frequency) domains. The corresponding computer-generated hologram in the spectral domain is obtained through at least one iteration of the algorithm. The algorithm is convergent and is configured to generate a hologram representing the input image. The hologram may be an amplitude-only hologram, a phase-only hologram, or a full complex hologram.

[0058] In some embodiments, the phase-only hologram is calculated using an algorithm based on the Gertchberg-Saxton type algorithm, such as described in British Patent Nos. 2,498,170 or 2,501,112, the entireties of which are incorporated herein by reference. However, the embodiments disclosed herein describe the calculation of a phase-only hologram by way of example only. In these embodiments, the Gertchberg-Saxton type algorithm obtains phase information Ψ[u,v] of the Fourier transform of a data set that gives rise to known amplitude information T[x,y], where the amplitude information T[x,y] represents a target image (e.g., a photograph). Since amplitude and phase are inherently combined in the Fourier transform, the transformed amplitude and phase contain useful information about the accuracy of the calculated data set. Thus, the algorithm can be used iteratively with feedback on both the amplitude and phase information. However, in these embodiments, only the phase information Ψ[u,v] is used as the hologram to form a holographic representation of the target image in the image plane. The hologram is a data set (e.g., a 2D array) of phase values.

[0059] We use an algorithm based on the Gertchberg-Saxton type algorithm to calculate a full complex hologram. A full complex hologram is a hologram that has an amplitude and a phase component. A hologram is a data set (e.g., a 2D array) that contains an array of complex data values, where each complex data value contains an amplitude and a phase component.

[0060] In some embodiments, the algorithm processes complex data and the Fourier transform is a complex Fourier transform. The complex data can be considered to include (i) a real component and an imaginary component, or (ii) an amplitude component and a phase component. In some embodiments, the two components of the complex data are processed differently at various stages of the algorithm.

[0061] FIG. 2A illustrates a first iteration of an algorithm according to some embodiments for computing a phase-only hologram. The input to the algorithm is an input image 210 that includes a 2D array of pixels or data values, where each pixel or data value is a magnitude or amplitude value. That is, each pixel or data value of the input image 210 does not have a phase component. The input image 210 can therefore be considered as a distribution of magnitude only, amplitude only, or intensity only. An example of such an input image 210 is a frame of a photograph or video that includes a temporal frame sequence. The first iteration of the algorithm begins with a data formation step 202A that includes assigning a random phase value to each pixel of the input image using a random phase distribution (or random phase seed) 230 to form a starting complex data set, where each data element of the set is composed of a magnitude and a phase. The starting complex data set can be said to represent the input image in the spatial domain.

[0062] A first processing block 250 receives a starting complex data set and performs a complex Fourier transform to form a Fourier transformed complex data set. A second processing block 253 receives the Fourier transformed complex data set and outputs a hologram 280A. In some embodiments, the hologram 280A is a phase-only hologram. In these embodiments, the second processing block 253 quantizes each phase value and sets each amplitude value to one to form the hologram 280A. Each phase value is quantized according to a phase level that can be represented on a pixel of a spatial light modulator used to "display" the phase-only hologram. For example, if each pixel of the spatial light modulator provides 256 different phase levels, each phase value of the hologram is quantized to one of the 256 possible phase levels. The hologram 280A is a phase-only Fourier hologram representing an input image. In other embodiments, the hologram 280A is a full complex hologram that includes an array of complex data values ​​(each including an amplitude component and a phase component) derived from the received Fourier transformed complex data set. In some embodiments, the second processing block 253 constrains each complex data value to one of a number of allowable complex modulation levels to form the hologram 280A. The constraining step may include setting each complex data value to the closest allowable complex modulation level in the complex plane. The hologram 280A may be said to represent the input image in the spectral, Fourier, or frequency domains. In some embodiments, the algorithm stops at this point.

[0063] However, in other embodiments, the algorithm continues as represented by the dotted arrow in Figure 2 A. In other words, the steps following the dotted arrow in Figure 2 A are optional (i.e., not required for all embodiments).

[0064] A third processing block 256 receives the modified complex data set from the second processing block 253 and performs an inverse Fourier transform to form an inverse Fourier transformed complex data set. The inverse Fourier transformed complex data set may be said to represent the input image in the spatial domain.

[0065] The fourth processing block 259 receives the inverse Fourier transformed complex data set and extracts the distribution of magnitude values ​​211A and the distribution of phase values ​​213A. Optionally, the fourth processing block 259 evaluates the distribution of magnitude values ​​211A. In particular, the fourth processing block 259 can compare the distribution of magnitude values ​​211A of the inverse Fourier transformed complex data set with the input image 510, which is of course a distribution of magnitude values ​​itself. If the difference between the distribution of intensity values ​​211A and the input image 210 is sufficiently small, the fourth processing block 259 can determine that the hologram 280A is acceptable. That is, if the difference between the distribution of intensity values ​​211A and the input image 210 is sufficiently small, the fourth processing block 259 can determine that the hologram 280A represents the input image 210 sufficiently accurately. The distribution of phase values ​​213A of the inverse Fourier transformed complex data set is ignored for the purposes of comparison. It will be appreciated that any number of different methods can be used to compare the distribution of magnitude values ​​211A to the input image 210, and the disclosure is not limited to any particular method. In some embodiments, the mean squared difference is calculated, and if the mean squared difference is less than a threshold, the hologram 280A is deemed acceptable. If the fourth processing block 259 determines that the hologram 280A is unacceptable, further iterations of the algorithm may be performed. However, this comparison step is not required, and in other embodiments, the number of iterations of the algorithm that are performed is predetermined, preset, or user-defined.

[0066] 2B represents a second iteration of the algorithm and a further iteration of the algorithm. The distribution of phase values ​​213A of the previous iteration is fed back through the processing blocks of the algorithm. The distribution of intensity values ​​211A is rejected in favor of the distribution of intensity values ​​of the input image 210. In the first iteration, the data formation step 202A formed a first complex data set by combining the distribution of intensity values ​​of the input image 210. However, in the second and subsequent iterations, the data formation step 202B includes forming a complex data set by combining (i) the distribution of phase values ​​213A from the previous iteration of the algorithm and (ii) the distribution of magnitude values ​​of the input image 210.

[0067] The composite data set formed by the data formation step 202B of FIG. 2B is then processed in the same manner as described with reference to FIG. 2A to form a second iteration hologram 280B. Therefore, the description of the process will not be repeated here. The algorithm may stop when the second iteration hologram 280B has been calculated. However, any number of further iterations of the algorithm may be performed. It will be appreciated that the third processing block 256 is only required if the fourth processing block 259 is required or if further iterations are required. The output hologram 280B will generally improve with each iteration. However, in practice, it is usual to reach a point where no measurable improvement is observed or the positive effects of performing further iterations outweigh the negative effects of the additional processing time. Therefore, the algorithm is described as iterative and convergent.

[0068] 2C represents an alternative embodiment for the second and subsequent iterations. The distribution of phase values ​​213A of the previous iteration is fed back through the processing blocks of the algorithm. The distribution of magnitude values ​​211A is rejected in favor of an alternative distribution of magnitude values. In this alternative embodiment, the alternative distribution of intensity values ​​is derived from the distribution of magnitude values ​​211 of the previous iteration. Specifically, processing block 258 subtracts the distribution of magnitude values ​​of the input image 210 from the distribution of magnitude values ​​211 of the previous iteration, scales the difference by a gain factor α, and subtracts the scaled difference from the input image 210. This is expressed as follows: Mathematically, it is calculated by the following equation: The subscripts and numbers indicate the iteration number.

number

[0069] The gain factor α may be fixed or variable. In some embodiments, the gain factor α is determined based on the size and rate of the input target image data. In some embodiments, the gain factor α depends on the number of iterations. In some embodiments, the gain factor α is simply a function of the number of iterations.

[0070] The embodiment of Figure 2C is in all other respects identical to the embodiment of Figures 2A and 2B. A phase-only hologram ψ(u,v) can be said to contain a phase distribution in the frequency or Fourier domain.

[0071] In some embodiments, the Fourier transform is performed using a spatial light modulator. Specifically, the hologram data is combined with second data that provides the refractive power. That is, the data written to the spatial light modulator includes hologram data representing the object and lens data representing the lens. When displayed on the spatial light modulator and illuminated with light, the lens data emulates a physical lens; that is, it focuses light in the same way as a corresponding physical optical element. Thus, the lens data provides a refractive, or focusing, power. In these embodiments, the physical Fourier transform lens 120 of FIG. 1 can be omitted. In the field of computer-generated holography, methods are known for calculating data representing lenses. The data representing lenses are sometimes referred to as software lenses. For example, a phase-only lens can be formed by calculating the phase delay caused at each point of the lens due to its refractive index and the spatially varying optical path length. For example, the optical path length at the center of a convex lens will be longer than the optical path length at the edge of the lens. An amplitude-only lens may be formed by a Fresnel zone plate. In the field of computer-generated holography, it is also known how to combine data representing lenses with a hologram so that the Fourier transform of the hologram can be performed without the need for a physical Fourier lens. In some embodiments, the lens data is combined with the hologram by simple addition, such as a simple vector addition. In some embodiments, a physical lens is used in combination with a software lens to perform the Fourier transform. Alternatively, in other embodiments, the Fourier transform lens is omitted entirely, so that the holographic reconstruction is performed in the far field. In further embodiments, the hologram can be combined in the same way with grating data, i.e. data arranged to perform the function of a grating, such as beam steering. Again, in the field of computer-generated holography, methods are known for calculating such data. For example, a phase-only grating can be formed by modeling the phase delay caused by each point on the surface of a blazed grating.An amplitude-only grating can be simply superimposed with an amplitude-only hologram to provide angular steering of the holographic reconstruction.

[0072] In some embodiments, the Fourier transform is performed jointly by a physical Fourier transform lens and a software lens, i.e., a portion of the refractive power contributing to the Fourier transform is provided by the software lens and the remaining portion of the refractive power contributing to the Fourier transform is provided by the physical optics.

[0073] In some embodiments, a real-time engine is provided that is configured to receive image data and use an algorithm to compute a hologram in real-time. In some embodiments, the image data is a video that includes a series of image frames. In other embodiments, the hologram is pre-computed and stored in computer memory and recalled as needed for display on the SLM. That is, in some embodiments, a repository of pre-defined holograms is provided.

[0074] These embodiments relate to Fourier holography and Gertschberg-Saxton type algorithms by way of example only, but the present disclosure is equally applicable to holograms computed by other techniques, such as Fresnel holography and techniques based on point cloud methods.

[0075] Light Modulation Spatial light modulators can be used to display diffraction patterns, including computer-generated holograms. If the hologram is a phase-only hologram, a spatial light modulator that modulates the phase is required. If the hologram is a fully complex hologram, a spatial light modulator that modulates the phase and amplitude can be used, or a first spatial light modulator that modulates the phase and a second spatial light modulator that modulates the amplitude can be used.

[0076] In some embodiments, the light modulation elements (i.e., pixels) of the spatial light modulator are cells that include liquid crystals. That is, in some embodiments, the spatial light modulator is a liquid crystal device in which the optically active components are liquid crystals. Each liquid crystal cell is configured to selectively provide a plurality of light modulation levels. That is, each liquid crystal cell is always configured to operate at one light modulation level selected from a plurality of possible light modulation levels. Each liquid crystal cell is dynamically reconfigurable to a light modulation level different from the plurality of light modulation levels. In some embodiments, the spatial light modulator is a reflective liquid crystal on silicon (LCOS) spatial light modulator, although the disclosure is not limited to this type of spatial light modulator.

[0077] LCOS devices provide a dense array of light-modulating elements or pixels within a small aperture (e.g. a few centimeters wide). Typically the pixels are around 10 microns or less, with a diffraction angle of a few degrees. This means that the optical system can be compact. It is easier to properly illuminate the small aperture of an LCOS SLM than the large aperture of other liquid crystal devices. LCOS devices are usually reflective, which means that the circuitry that drives the pixels of an LCOS SLM can be embedded under the reflective surface. The result is a higher aperture ratio. In other words, the pixels are densely packed, meaning there is very little dead space between them. This is advantageous as it reduces optical noise in the replay field. LCOS SLMs use a silicon backplane, which has the advantage that the pixels are optically flat. This is particularly important for phase modulation devices.

[0078] A suitable LCOS SLM will now be described, by way of example only, with reference to FIG. 3. The LCOS device is formed using a single crystal silicon substrate 302. It has a 2D array of square planar aluminum electrodes 301 spaced apart by gaps 301a and disposed on the top surface of the substrate. Each of the electrodes 301 can be addressed via circuitry 302a embedded in the substrate 302. Each electrode forms a respective planar mirror. An alignment layer 303 is disposed on the electrode array, and a liquid crystal layer 304 is disposed on the alignment layer 303. A second alignment layer 305 is disposed on a planar transparent layer 306, for example of glass. A single transparent electrode 307, for example of ITO, is disposed between the transparent layer 306 and the second alignment layer 305.

[0079] Each of the square electrodes 301, together with the area overlying the transparent electrode 307 and the intervening liquid crystal material, defines a controllable phase modulating element 308, often referred to as a pixel. The effective pixel area, or fill factor, is the percentage of the entire pixel that is optically active, taking into account the space between the pixels 301a. By controlling the voltage applied to each electrode 301 relative to the transparent electrode 307, the properties of the liquid crystal material in each phase modulating element can be changed, thereby imparting a variable retardation to light incident thereon. The effect is to impart a phase-only modulation to the wavefront; that is, no amplitude effects occur.

[0080] The described LCOS SLMs reflect and output spatially modulated light. Reflective LCOS SLMs have the advantage that the signal lines, gate lines and transistors are located under the mirror surface, resulting in a high fill factor (typically 90% or more) and high resolution. Another advantage of using reflective LCOS spatial light modulators is that the liquid crystal layer can be half the thickness that would be required using a transmissive device. This greatly increases the switching speed of the liquid crystal (an important advantage in projecting moving images). However, the teachings of this disclosure can be implemented using transmissive LCOS SLMs as well.

[0081] Head-up display In some embodiments, a holographic projection system is provided as part of a head-up display (or "HUD"). Figure 4 shows a HUD mounted on a vehicle, such as an automobile. The vehicle's windshield 430 and bonnet (or hood) 435 are shown in Figure 4. The HUD comprises an image generation unit "PGU" 410 and an optical system 420. The PGU 410 and optical system 420 may be collectively referred to as a holographic projector.

[0082] The PGU 410 comprises a processor (or computer) configured for computer control of a light source, a receiving surface, and image content of a photograph. The PGU 410 is configured to generate an image or a series of images on the receiving surface. The receiving surface may be a screen or a diffuser. In some embodiments, the receiving surface is plastic (i.e., made of plastic). The receiving surface is placed on the primary playback surface, i.e., the holographic playback surface on which the image is initially formed.

[0083] The optical system 420 comprises an input port, an output port, a first mirror 421 and a second mirror 422. The first mirror 421 and the second mirror 422 are arranged to direct light from the input port of the optical system to the output port of the optical system. More specifically, the second mirror 442 is arranged to receive the image light from the PGU 410, and the first mirror 421 is arranged to receive the image light from the second mirror 422. The first mirror 421 is further arranged to reflect the received light to the output port. Thus, the optical path from the input port to the output port includes a first optical path 423 (or a first optical path component) from the input to the second mirror 422 and a second optical path 424 (or a second optical path component) from the second mirror. Of course, there is a third optical path (or optical path component) from the first mirror to the output port, but it is not assigned a reference number in FIG. 4. The optical configuration shown in FIG. 4 is sometimes called a "z-fold" configuration due to the shape of the optical path.

[0084] The HUD is configured and positioned within the vehicle such that the light of the image from the output port of the optical system 420 enters the windshield 430 and is at least partially reflected by the windshield 430 towards a user 440 of the HUD. Thus, in some embodiments, the optical system is configured to form a virtual image of each image within the windshield by reflecting the spatially modulated light from the windshield. The user 440 of the HUD (e.g., the driver of the car) sees a virtual image 450 of the photograph within the windshield 430. Thus, in embodiments, the optical system is configured to form a virtual image of each photograph on the windshield of the vehicle. The virtual image 450 is formed a certain distance below the hood 435 of the car. For example, the virtual image may be 1 to 2.5 meters away from the user 440. The output port of the optical system 420 is aligned with an opening in the dashboard of the car such that the light of the image is directed by the optical system 420, and in this configuration the windshield 430 acts as an optical combiner. In some embodiments, the optical system is configured to form a virtual image of each image on an additional optical combiner included in the system. The windshield 430, or additional optical combiners if included, combines light from the real world scene with the light of the photograph. It will thus be appreciated that the HUD can provide augmented reality including a virtual image of the photograph. For example, the augmented reality information may include navigation information or information about the speed of the vehicle. In some embodiments, the light forming the image is output by being incident on the windshield at the Brewster angle (also known as the polarization angle) or within 5 degrees of the Brewster angle, for example within 2 degrees of the Brewster angle.

[0085] In some embodiments, the first and second mirrors are positioned to fold the optical path from the input to the output to increase the optical path length without unduly increasing the physical size of the HUD.

[0086] The image formed on the light receiving surface of the PGU 410 is only a few centimeters in width and height. The light receiving surface of the PGU 410 may be an alignment type display surface. The first mirror 421 and the second mirror 422 perform the magnification collectively or individually. That is, the first mirror and / or the second mirror may have a refractive power (i.e., a refractive power or a focusing power). Thus, the user 440 sees a magnified virtual image 450 of the image formed by the PGU. The first mirror 421 and the second mirror 422 can also compensate for optical distortions such as those caused by the windshield 430, which typically has a complex curved shape. The folded optical path and the refractive power of the mirrors allow the virtual image of the image to be appropriately magnified.

[0087] The PGU 410 of this disclosure comprises a holographic projector and a light receiving surface, such as a screen or diffuser.

[0088] Distortion Compensation According to the above disclosure, the holographic projector comprises a light source, a spatial light modulator, and a hologram processor. The spatial light modulator is configured to spatially modulate light according to one or more (usually a series of) holograms represented on the spatial light modulator. The hologram processor is configured to provide a computer-generated hologram. In some embodiments, the hologram processor calculates and outputs the computer-generated hologram in real time. In some embodiments, each image formed by the PGU 410 is a holographic reconstruction on a receiving surface. That is, in some embodiments, each image is formed by the interference of the spatially modulated light on the receiving surface.

[0089] Each hologram represented (or displayed) on the spatial modulator may be a hologram of a target image. The holographic reconstruction is a holographic reconstruction of the image. The virtual image 450 is a virtual image of the holographic reconstruction. The virtual image of the holographic reconstruction may be distorted with respect to the target image encoded by the hologram. In other words, the image points or pixels of the virtual image of the holographic reconstruction may have a different spatial distribution with respect to the target image. In particular, the spacing of adjacent pixels in the virtual image of the holographic reconstruction may be different from the spacing of the respective adjacent pixels in the target image. In other words, the pixels of the virtual image of the holographic reconstruction may be mutually shifted with respect to the respective pixels of the target image. The distortion will typically be non-uniform across the virtual image of the holographic reconstruction. In other words, the distortion will not just uniformly expand the holographic reconstruction with respect to the target, but will actually tilt or distort the image. There are several sources of this distortion. The present disclosure is particularly concerned with compensating for distortions associated with current or temperature changes in a holographic projector. For example, optical components of a holographic projector typically expand or contract in response to temperature changes. This can lead to optical misalignment. Furthermore, temperature changes in the holographic projector (and in particular the light source of the holographic projector) typically result in a change in the wavelength of the light emitted by the light source (in this example a coherent light source such as a laser). As will be appreciated by those skilled in the art, the positions of the pixels of the holographic reconstruction will shift in response to changes in the wavelength of the light emitted by the light source.

[0090] If the amount of distortion for each pixel (at a particular temperature and wavelength of light) is known, it is possible to compensate for distortions in virtual image 450 by pre-distorting the target image in a manner opposite to the distortion caused by, for example, optical misalignment of a holographic projector at a particular temperature. In this way, when a hologram of the pre-distorted target image is calculated / generated (computer-generated) and displayed, the holographic reconstruction of the target image will appear as intended. This pre-distortion is shown diagrammatically in FIG. 5.

[0091] FIG. 5 illustrates a process of pre-distorting a target image 500 prior to the computation of a hologram of the target image 500. In the representative example of FIG. 5, a virtual image is intended to be received by a user of a holographic projector. The array of points 502 includes black points 502 in FIG. 5. The spacing between the points 502 is uniform throughout the array. The array in the example of FIG. 5 consists of 6 points by 6 points. If a hologram is computed from the uniform array of points 502 and displayed on a spatial light modulator, the subsequent virtual image 450 projected by the projector will be distorted as a result of the above-mentioned effects. Thus, the virtual image 450 (received by the user of the projector) will appear as a non-uniform array. The target image 500 can be pre-distorted using a pre-defined distortion map. The pre-defined distortion includes a mapping of each pixel of the target image. Each mapping functions as a transformation. In particular, each mapping transforms the two-dimensional coordinates associated with one or more pixels of the target image to new (pre-distorted) positions. Each mapping compensates for the distortion experienced by each pixel of the target image in the virtual image 450. The transformation / (pre)distortion of the target image 500 is represented in FIG. 5 by the white points 504. In particular, there is a white point 504 for each black point 502. The transformation of the position of each point 504 is represented by the arrows 506 from the black points 502 to the respective white points 504. The amount of predistortion / the amount by which each black point 502 is shifted is represented by the length of the arrows 506 in FIG. 5. The length of the arrows 506 in FIG. 5 is non-uniform throughout the target image 500, and therefore the amount of predistortion is non-uniform. For example, the amount of predistortion required is least at the center 510 of the target image and increases for black points 502 closer to the edge 512 of the target image. Furthermore, the direction of predistortion is non-uniform throughout the target image 500. In particular, the direction of predistortion is approximately parallel to a direction defined from the center 510 of the target image through each black point. Thus, in the example of FIG. 5, black dots 502 that are opposite each other with respect to the center 510 are pre-distorted in opposite directions.Once the target image 500 has been pre-distorted, a hologram of the pre-distorted image is created and the target image is calculated and displayed on the spatial light modulator and projector to form a holographic reconstruction, as shown in FIG.

[0092] Fig. 6A represents a pre-distorted target image 600 after a predefined distortion map has been applied to the target image 500. Fig. 6B represents a virtual image 602 of a holographic reconstruction of the pre-distorted target image 600. As mentioned above, this is the image that is intended to be projected by a holographic projector. Thus, the pre-distortion of the target image 500 can successfully compensate for certain distortions caused by the holographic projector.

[0093] The target image in Figure 5 shows an array of black dots 502 merely as a convenient example to illustrate the process of pre-distorting a target image to compensate for distortions caused by the holographic project / current environmental conditions. It should be apparent to one skilled in the art that pre-distortion can be applied to any target image (e.g., a target image that includes a non-uniform distribution of pixels).

[0094] Real-time distortion map calculation The distortion compensation described in connection with FIGS. 5 and 6 above requires a pre-determined distortion map. Typically, significant experimental and simulation work is required to properly characterize a holographic projector to arrive at a pre-determined distortion map that sufficiently pre-distorts the target image. Even so, a pre-determined distortion map is only accurate for compensating for distortions at a particular temperature and / or wavelength of light. Thus, the pre-distortion of the target image 500 shown in FIGS. 5 and 6 will only accurately compensate for distortions if the holographic projector is at that temperature and / or uses a wavelength for which the pre-determined distortion is specifically determined. However, a holographic projector must typically be able to operate over a range of temperatures and provide high-quality (distortion-free) holographic reconstructions. For a holographic projector in a vehicle, the required operating temperature range is typically relatively wide, e.g., at least 100 degrees Celsius. Clearly, a single pre-determined distortion map is not suitable for use over such a wide temperature range. However, the inventors have recognized that characterizing a large number of pre-determined distortion maps (for different temperatures and / or wavelengths) is not practical. This is because a) doing so would require a very large amount of validation work, and b) the memory requirements would be prohibitively high. Instead, the inventors have developed a fast, computationally efficient method to calculate the distortion map for the current temperature. This method can be performed instantly in real-time. This means there is no need to characterize the holographic projector at a large number of temperatures and store each map in large memories or cumbersome look-up tables. Instead, the method can take as input a single (validated) pre-determined map and scale / modify that single map for the current temperature. The inventors developed this method following the discovery, after extensive simulations and experiments, that there is a predictable (linear) relationship between the current temperature and the amount of pre-distortion required to compensate for a particular distortion.

[0095] FIG. 7 shows a flow diagram illustrating a method according to the present disclosure.

[0096] Step 702 of the method includes receiving an array of two-dimensional coordinates. In some examples, each two-dimensional coordinate represents or corresponds to one or more pixels of a target image, such as target image 500 without predistortion applied. The method according to the present disclosure calculates a distortion map suitable for transforming each two-dimensional coordinate of the array to compensate for distortion at the current temperature of the holographic projector. The transformed position of each two-dimensional coordinate is used to predistort the target image. In some examples, there is a one-to-one relationship between the number of two-dimensional coordinates in the array and the number of pixels in the target image. In other words, each two-dimensional coordinate can be considered to represent (or be) one of the pixels of the target image. In such examples, the array of transformed two-dimensional coordinates can be used to directly predistort / shift each pixel of the target image. In other examples, there is a one-to-many relationship between the number of two-dimensional coordinates in the array and the number of pixels in the target image. In other words, each two-dimensional coordinate can be considered to represent or correspond to multiple pixels of the target image. In such examples, the distortion map may be more efficient to compute because it involves fewer transformations / mappings (each of which needs to be calculated). However, in such examples, the array of transformed two-dimensional coordinates cannot be used to directly pre-distort or shift each pixel of the target image. Instead, an interpolation step is required to determine how to pre-distort or shift each pixel of the target image using the location of each transformed two-dimensional coordinate. Suitable interpolation methods will be familiar to those skilled in the art.

[0097] Step 704 of the method includes receiving a single calibrated distortion map M0. The single calibrated distortion map includes multiple mappings, each mapping for transforming one of the two-dimensional coordinates in the received array of two-dimensional coordinates. The mappings of the single calibrated distortion map are mappings that have been determined and verified by prior experimentation and / or simulation. In some examples, the steps of determining / verifying these mappings do not form part of the method according to the present disclosure. The mappings of the single calibrated distortion map are mappings that have been determined by experimentation and / or simulation to compensate for distortions of the target image at a single first predetermined temperature T0.

[0098] Step 706 of the method includes receiving an array of vectors C. The array of vectors C includes a vector for each two-dimensional coordinate in the array of two-dimensional coordinates. Each vector in the array of vectors C represents a calibrated change in a respective two-dimensional coordinate across a predetermined temperature range. In this example, each vector has been determined by simulating how each two-dimensional coordinate needs to be transformed to calibrate / pre-distort the coordinate at a minimum temperature in the predetermined temperature range and a maximum temperature in the predetermined temperature range. The vector represents the change in the two-dimensional coordinate between these two extreme temperature ranges. In one example, determining the array of vectors includes taking a single calibrated distortion map as an input and simulating how that distortion map changes with a change in temperature. In particular, how the distortion map changes at the minimum and maximum temperatures may be simulated. The inventors have found in examples that the change in the mapping of the distortion map with respect to temperature is linear. In embodiments, the simulation of the distortion map at the minimum and maximum temperatures depends on this linear relationship. For clarity, the method does not include a step of determining the vectors in the array of vectors (i.e., the magnitude and / or direction of the vectors) in most examples. Instead, the method simply involves receiving an array of separate or pre-determined vectors, although for completeness a description of how the vectors are determined is included above.

[0099] Step 708 of the method includes determining a current temperature T of the holographic projector. In this example, step 708 of the method includes measuring the temperature of the holographic projector (and in some examples, the temperature of the light source of the holographic projector) using a temperature sensor. In other examples, step 708 of the method includes measuring or determining a wavelength of light emitted by a light source of the holographic projector and inferring a temperature of the light source based on the measured or determined wavelength, in a manner familiar to those skilled in the art.

[0100] Step 710 of the method includes determining a scaling factor using the measured or determined current temperature. In one example, determining the scaling factor includes determining the difference between the current temperature and a first predetermined temperature (associated with the calibrated distortion map). This difference is then divided by the predetermined temperature range (associated with the array of vectors). Thus, the scale factor represents the change in the current (measured or determined) temperature from the first predetermined temperature (associated with the calibrated distortion map) as a percentage of the predetermined temperature range (associated with the array of vectors).

[0101] Step 712 of the method includes the steps of: for each coordinate in the two-dimensional coordinate array, calculating a modified map based on the current temperature; multiplying a vector associated with each two-dimensional coordinate in the array of two-dimensional coordinates by a scaling factor to output a scaled vector; and outputting the modified mapping for transforming each two-dimensional coordinate by applying (e.g., adding or subtracting) the scaled vector to each mapping in the calibrated map to compensate for distortion of the target image when projected by the holographic projector at the current temperature.

[0102] A step 714 of the method includes outputting a correction map that includes an array of the correction mappings determined in step 712 .

[0103] The method illustrated in FIG. 7 is suitable for real-time or instantaneous execution. In particular, the method illustrated in FIG. 7 may be executed while a holographic projector is in operation. The method of FIG. 7 may be followed by a step of pre-distorting a target image using the mapping determined by the method of FIG. 7. This may then be followed by computing a hologram of the pre-distorted target image. This may then be followed by displaying a hologram of the pre-distorted target image to form a holographic reconstruction of the target image. This entire process may be repeated for a second (different) target image.

[0104] In one example, the method according to the present disclosure can be expressed by the following equation:

number

number

[0105] FIG. 8 shows a schematic of one vector (of an array of vectors C(x,y)) that is scaled based on the current temperature T and a first predefined temperature T0. In FIG. 8, the vector is represented by an arrow 802. FIG. 8 also includes two black points 804, 806, which are at positions of their respective two-dimensional coordinates at the two extremes of the predefined temperature range. In particular, the position of black point 804 is simulated at Tmin, and the position of black point 806 is simulated at Tmax. Vector 802 represents the magnitude and direction of the change in position from Tmin to Tmax. In other words, vector 802 represents how the respective two-dimensional coordinates change over the operable temperature range of the holographic projector (when properly calibrated to compensate for temperature distortions). Arrow 808 indicates the magnitude and direction of the change in position of the vector.

number

[0106] 9 shows a schematic representation of the process of pre-distorting a target image 900 using a modified distortion map M (calculated in real time). In FIG. 9, similar to FIG. 5, it is intended that a user of a holographic projector receives a virtual image of a uniform array of points 902. The array of points 902 includes black points 902 in FIG. 9. The points 902 are uniform throughout the array. The example array in FIG. 9 consists of 6 points by 6 points. Returning to the above equations, the calculation of the modified distortion map effectively consists of two terms. The first term corresponds to the calibrated distortion map M0. The second term corresponds to the array of scaled vectors. FIG. 9 represents both terms as separate distortions. In particular, in FIG. 9, each point 902 is first distorted using the calibrated distortion map M0 and then adjusted using the scaled vectors. The components of the pre-distortion of each black point 902 as a result of the calibrated distortion map M0 are represented by arrows 904 and white points 906. The components of the predistortion of each white point 906 as a result of the scaled vector are: The two different components are shown as separate transformations / distortions in Fig. 9 for illustrative purposes only. In reality, when the modified map M is used to transform / predistort the target image, each black point 902 would be directly shifted to the respective position shown, represented by the dashed white point 910.

[0107] Single Color Channel The methods according to the present disclosure have particular application in multi-color holographic projectors, due to the need to ensure that pixels / image features in the holographic reconstructions of different colors are reliably aligned, an example of such a holographic projector is described herein.

[0108] An example of the present disclosure relates to a holographic projector with multiple monochromatic channels. Each monochromatic channel constitutes a monochromatic holographic projector that forms a monochromatic holographic reconstruction (i.e., an image or a photograph). The monochromatic images are formed on a common reconstruction surface. A full-color image can be formed using red, green, and blue images simultaneously.

[0109] FIG. 10 shows red, green and blue color channels. The red channel comprises a first spatial light modulator 1001r, a first lens 1020r and a first mirror 1027r. The green channel comprises a second spatial light modulator 1001g, a second lens 1020g and a second mirror 1017g. The blue channel comprises a third spatial light modulator 1001b, a third lens 1020b and a third mirror 1007b. Each single color channel forms a single color holographic reconstruction (or image) on the reproduction plane 1050. The first lens 1020r, the second lens 1020g and the third lens 1020b are optional. If each hologram displayed is a Fourier hologram, the first lens 1020r, the second lens 1020g, and the third lens 1020b may contribute to the Fourier transform of the respective hologram.

[0110] The first spatial light modulator 1001r displays a hologram corresponding to a red image. The first spatial light modulator 1001r is illuminated with red light. The first lens 1020r receives the spatially modulated light from the first spatial light modulator 1001r and forms a red image on the reconstruction plane 1050. The first mirror 1027r is disposed between the first lens 1020r and the reconstruction plane 1050.

[0111] The second spatial light modulator 1001g displays a hologram corresponding to a green image. The second spatial light modulator 1001g is illuminated with green light. The second lens 1020g receives the spatially modulated light from the second spatial light modulator 1001g and forms a green image on the reconstruction plane 1050. The second mirror 1017g is disposed between the second lens 1020g and the reconstruction plane 1050.

[0112] The third spatial light modulator 1001b displays a hologram corresponding to a blue image. The third spatial light modulator 1001b is illuminated with blue light. The third lens 1020b receives the spatially modulated light from the third spatial light modulator 1001b and forms a blue image on the reconstruction plane 1050. The third mirror 1007b is disposed between the third lens 1020b and the reconstruction plane 1050.

[0113] The first mirror 1027r is a first dichroic mirror that reflects red light and transmits green and blue light. The second mirror 1017g is a second dichroic mirror that reflects green light and transmits blue light. The third mirror 1007b reflects blue light.

[0114] Each monochromatic optical path comprises a first portion from the spatial light modulator to the mirror and a second portion from the mirror to the reproduction surface. In an embodiment, the first portions of the single channels are spatially offset but substantially parallel. In an embodiment, the second portions of the single channels are substantially collinear.

[0115] The red light path from the first spatial light modulator 1001r to the reproducing plane 1050 includes a reflection from the first mirror 1027r. The green light path from the second spatial light modulator 1001g to the reproducing plane 1050 includes a reflection from the second mirror 1017g followed by a transmission through the first mirror 1027r. The blue light path from the third spatial light modulator 1001b to the reproducing plane includes a reflection from the third mirror 1007b followed by a transmission through the second mirror 1017g followed by a transmission through the first mirror 1027r. The reproducing plane 1050, the first mirror 1027r, the second mirror 1017g and the third mirror 1007b are substantially collinear. The blue light path length is longer than the green light path length, which is longer than the red light path length. Specifically, in embodiments, the second portion of the blue light path is longer than the second portion of the green light path, and the second portion of the green light path is longer than the second portion of the red light path, and in these embodiments, the lengths of the first portions may be substantially equal.

[0116] Each single color channel can be used to form a holographic reconstruction within the replay field region. The red replay field may contain the red image content of the image. The green replay field may contain the green image content of the image. The blue replay field may contain the blue image content of the image. Those skilled in the art will be familiar with the idea of ​​forming an image by using the red, green, and blue color channels to superimpose the red, green, and blue image content. Alignment of the red, green, and blue replay fields is critical to image quality.

[0117] As mentioned above, the holographic reconstruction may be distorted with respect to the target image encoded in the respective hologram displayed on the spatial light modulator. This may result in the pixels / light spots of the holographic reconstruction having a different spatial distribution with respect to their respective spatial distribution in the target image. We have already described how a distortion map can be calculated in real time to pre-distort the target image to compensate for this distortion. However, there is another problem with multi-color holographic projectors, which is that the amount of distortion / shift of corresponding image spots / pixels in the holographic reconstruction associated with different channels is usually different. Those skilled in the art will recognize that this is because a) the wavelengths of light in the different color channels are different, and b) the misalignment (e.g. due to manufacturing tolerances) of the different channels may be different. This may result in misalignment of corresponding pixels of different light channels, which may negatively affect image quality. This misalignment is illustrated in FIG. 11.

[0118] FIG. 11 shows a first array of green light spots 1102G formed by a first (green) holographic channel (represented by white dots in FIG. 11) and a second array of blue light spots 1102B formed by a second (blue) holographic channel (represented by white dots in FIG. 11 (solid lines in FIG. 11). The light spots of the first holographic channel are misaligned with respect to the light spots of the second holographic channel.

[0119] As mentioned above, the distortion can be compensated for by pre-distorting the target image. However, as will be appreciated by those skilled in the art, the same distortion map cannot be used for red content, green content, and blue content because the amount of distortion for different color channels is different. In other words, the correction map M calculated in the above manner may be suitable for compensating for the distortion of only one channel. However, if the same modified map is applied to the other channels, the different color points / pixels will not align. Thus, for a multi-color holographic projector, the above method can be repeated for each color channel to determine the correction for each color channel. There may be different adjusted distortion maps for each color channel and different arrays of vectors for each color channel.

[0120] Additional Features The methods and processes described herein may be embodied on a computer-readable medium. The term "computer-readable medium" includes a medium configured to store data temporarily or permanently, such as a random access memory (RAM), a read-only memory (ROM), a buffer memory, a flash memory, a cache memory, etc. The term "computer-readable medium" is also interpreted to include any medium, or combination of media, capable of storing instructions for execution by a machine, which, when executed by one or more processors, cause the machine to perform, in whole or in part, any one or more of the methodologies described herein.

[0121] The term "computer-readable medium" also includes cloud-based storage systems. The term "computer-readable medium" includes one or more tangible, non-transitory data repositories (e.g., data volumes), examples of which include, but are not limited to, solid-state memory chips, optical disks, magnetic disks, disks, or any suitable combination thereof. In some exemplary embodiments, the instructions for execution may be conveyed by a carrier medium. Examples of such carrier media include transitory media (e.g., a propagated signal conveying the instructions).

[0122] It will be apparent to one skilled in the art that various modifications and variations can be made without departing from the scope of the appended claims. This disclosure covers all modifications and variations that come within the scope of the appended claims and their equivalents.

Claims

1. A method for computing a map in real time, the map being for distorting a target image projected by a holographic projector to compensate for changes in a current temperature of the holographic projector; receiving a calibrated map including a plurality of mappings, each mapping transforming a respective two-dimensional coordinate of an array of two-dimensional coordinates to compensate for distortion at a given temperature, each two-dimensional coordinate corresponding to one or more image points of a target image; receiving an array of vectors including a vector for each two-dimensional coordinate, each vector representing a calibrated change in each two-dimensional coordinate over a predetermined temperature range; receiving a current temperature of the holographic projector; determining a scaling factor based on a difference between the current temperature and the predetermined temperature; calculating, for each coordinate in the array of two-dimensional coordinates, a modified map based on a current temperature; multiplying a vector associated with each of the coordinates in the array of two-dimensional coordinates by the scaling factor to output a scaled vector; by applying the scaled vector to the respective mapping of the calibrated map; outputting the modified map; The method includes:

2. The method of claim 1 , wherein the scaling factor has a linear dependence on temperature.

3. 3. The method of claim 2, wherein the step of determining the scaling factor comprises determining a difference between the current temperature and the predetermined temperature and dividing the difference by the predetermined temperature range.

4. The scaling factor is (T−T 0 ) / (T max -T min ), T is the current temperature, T 0 is the predetermined temperature, T maxは The maximum temperature of the predetermined temperature range, T min The method of claim 1 , wherein:

5. The method of claim 1 , further comprising receiving the array of two-dimensional coordinates.

6. The method of claim 1 , further comprising applying the modified map to the array of two-dimensional coordinates to output a modified array of two-dimensional coordinates.

7. receiving a target image including a plurality of image points, each two-dimensional coordinate in the array of two-dimensional coordinates corresponding to one or more image points in the target image; pre-warping the target image based on the array of modified two-dimensional coordinates; The method of claim 6 further comprising:

8. The method of claim 7 , further comprising: computing a hologram of the pre-distorted target image.

9. the target image is a first target image, the array of two-dimensional coordinates is a first array of two-dimensional coordinates, the calibrated map is a first calibrated map, the array of vectors is a first array of vectors, and the corrected map is a first corrected map; a second map for distorting a second target image projected by the holographic projector; receiving a second calibrated map including a plurality of second mappings, each second mapping transforming a respective two-dimensional coordinate of the second array of two-dimensional coordinates to compensate for distortion at a predetermined temperature, each two-dimensional coordinate corresponding to one or more image points of the target image; receiving a second array of vectors including a vector for each two-dimensional coordinate, each vector in the second array representing a calibrated change in each two-dimensional coordinate over a predetermined temperature range; For each coordinate in the second array of two-dimensional coordinates, multiplying a vector associated with each of the coordinates in the second array of two-dimensional coordinates by the scaling factor to output a scaled vector; calculating a second modified map based on the current temperature by applying the scaled vector to the respective mappings of the second calibrated map; outputting the second modified map; and The method of claim 1 , further comprising:

10. 10. The method of claim 9, wherein the mapping of the first calibrated map and the vectors of the first array of vectors are determined for when a first wavelength is used in holographic projection of the first target image.

11. 11. The method of claim 10, wherein the mapping of the second calibrated map and the vectors of the second array of vectors are determined for when a second wavelength is used in the holographic projection of the second target image.

12. 12. The method of claim 9, further comprising applying the second modified map to the second array of two-dimensional coordinates to output a modified array of second two-dimensional coordinates.

13. The method of claim 12 , further comprising warping the second target image based on the modified map.

14. The method of claim 13 , further comprising: computing a second hologram of the distorted second target image.

15. 1. A holographic projector comprising a display device arranged to display a hologram of an image and to spatially modulate light incident thereon in accordance with said hologram, said display device being arranged to form a holographic reconstruction of said image in a reproduction plane, a calibrated map including a plurality of mappings, each mapping transforming a respective two-dimensional coordinate of an array of two-dimensional coordinates to compensate for distortion at a given temperature, each two-dimensional coordinate corresponding to one or more image points of a target image; an array of vectors including a vector for each two-dimensional coordinate, each vector representing a calibrated change in each two-dimensional coordinate over a predetermined temperature range; and a controller having a memory in which is stored: The controller: determining a current temperature of the holographic projector; determining a scaling factor based on a difference between the current temperature and the predetermined temperature; For each coordinate in the array of two-dimensional coordinates, multiplying a vector associated with each of the coordinates in the array of two-dimensional coordinates by the scaling factor to output a scaled vector; a holographic projector arranged to calculate a revised map based on the current temperature by applying the scaled vector to the respective mapping of the calibrated map.

Citation Information

Patent Citations

  • Spatial light modulation device and spatial light modulation method

    JP2012173565A

  • Holographic image alignment

    US20210191321A1

  • Image display device and head-up display system

    WO2022264642A1