Picture distortion
By employing a real-time image pre-distortion calculation method, using a single calibration distortion map and vector array, the image distortion problem caused by temperature and wavelength variations in holographic projectors is solved, thereby improving the image quality and color channel alignment of holographic reconstruction.
Patent Information
- Application Number
- CN202410546802.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-05-04
- Filing Date
- 2024-05-06
- Publication Date
- 2024-11-05
AI Technical Summary
Existing holographic projectors suffer from image distortion due to temperature and wavelength changes, especially in multicolor holographic projection where misalignment between different color channels affects image quality.
By employing a fast and resource-efficient method that computes in real time, using a single calibration distortion map and vector array, image pre-distortion correction is performed based on the current temperature and wavelength to compensate for distortion caused by temperature and wavelength variations in the holographic projector.
It achieves image quality stability and alignment under temperature and wavelength variations, improving the clarity and consistency of holographic reconstruction, especially the alignment of different color channels in multicolor holographic projection.
Smart Images

Figure CN118900318A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to holographic projection of a target image. More specifically, the present disclosure relates to a method of calculating a distortion map for pre-distorting a target image to compensate for distortion caused by a holographic projector. Even more specifically, the present disclosure relates to a method for performing the method in real time. Some embodiments relate to a holographic projector, an image generation unit, or a head-up display. Background Art
[0002] Light scattered from an object contains both amplitude and phase information. This amplitude and phase information can be captured, for example, on a photographic plate by well-known interference techniques to form a holographic recording or "hologram" comprising interference fringes. The hologram can be reconstructed by illumination with appropriate light to form a two-dimensional or three-dimensional holographic reconstruction or replay image representing the original object.
[0003] Computer generated holography can numerically simulate interference processes. Computer generated holograms can be calculated by techniques based on mathematical transformations such as Fresnel or Fourier transforms. These types of holograms may be referred to as Fresnel / Fourier transform holograms or simply Fresnel / Fourier holograms. Fourier holograms may be considered as Fourier domain / plane representations of objects or frequency domain / plane representations of objects. Computer generated holograms may also be calculated, for example, by coherent ray tracing or point cloud techniques.
[0004] The computer generated hologram may be encoded on a spatial light modulator arranged to modulate the amplitude and / or phase of the incident light. For example, the light modulation may be achieved using electrically addressable liquid crystals, optically addressable liquid crystals or micromirrors.
[0005] A spatial light modulator typically includes a plurality of individually addressable pixels, which may also be referred to as cells or elements. The light modulation scheme may be binary, multi-level, or continuous. Alternatively, the device may be continuous (i.e., not including pixels), and thus the light modulation may be continuous across the device. A spatial light modulator may be reflective, meaning that the light is modulated to be output in reflection. A spatial light modulator may also be transmissive, meaning that the light is modulated to be output in transmission.
[0006] Using the system described herein, a holographic projector can be provided. Such projectors have been used, for example, in head-up displays "HUDs". Summary of the invention
[0007] Aspects of the disclosure are defined in the accompanying independent claims.
[0008] In summary, a real-time method is provided for computing a map for processing (e.g. (pre)distorting) a target image to be projected by a holographic projector, the map being suitable for pre-distorting the target image to compensate for temperature, e.g. the current temperature of the holographic projector.
[0009] The holographic projector may include a display device, such as a spatial light modulator. The display device may be arranged to display a hologram of a target image (e.g., a hologram of a predistorted image). The hologram may be a computer-generated hologram. The display device may be arranged to spatially modulate the light incident thereon according to the hologram displayed on the display device. The holographic projector may be arranged to direct or relay the spatially modulated light (spatial modulated according to the hologram) to an observation plane or an eye box. The holographic projector may be arranged so that the spatially modulated light forms a holographic reconstruction of an image on a playback plane. The holographic projector may be arranged so that the holographically reconstructed image (e.g., a virtual image) (of the target image) is visible (i.e., observable) from the observation plane or the eye box.
[0010] The holographically reconstructed image of the target picture may be distorted relative to the holographic reconstruction itself. Conceptually, multiple points (e.g., pixels or groups of pixels) of the holographic reconstruction and corresponding multiple points of the holographically reconstructed image may be considered. The distribution of points, spots, pixels, or features in the holographic reconstruction (image) may be different in position from the corresponding points, spots, pixels, or features of the target picture (encoded in the hologram). For example, at least some image points of the holographically reconstructed image may be positionally translated or shifted relative to the corresponding image points or image features of the holographic reconstruction with respect to adjacent image points. Different image points or image features in the holographically reconstructed image may have been shifted or translated by different amounts. In other words, the amount of translation or shift of the image points may be uneven across the holographically reconstructed image. The holographic projector may include one or more optical components to relay spatially modulated light from a display device. Examples of optical components include lenses and mirrors. Optical components may provide magnification. One source of distortion may be the result of optical misalignment of optical components, for example due to manufacturing tolerances. Since temperature changes may cause the (optical) components of the holographic projector to expand or contract and thereby change the distortion, the distortion may change in response to temperature changes of the holographic projector. Temperature changes may also cause the wavelength of the light emitted by the light source of the holographic projector to change (the light is used to illuminate the display device). For example, the light source can be a coherent light source, such as a laser. The wavelength of the coherent light source / laser can change with temperature changes. Distortion may cause the holographic reconstructed image to be distorted and / or skewed relative to the holographic reconstruction (and / or the target image), so the distortion may have an adverse effect on the viewing experience and the quality (e.g., authenticity) of a single holographic reconstruction. The quality or fidelity of the image can be quantified using many different methods known in the field of optics / image formation. Those skilled in the art of optics will be familiar with these techniques, and the present disclosure is not limited to any particular method of measuring / quantifying image quality or image distortion. It can therefore be said that, in a broad sense, the present disclosure relates to methods for improving image quality or reducing image distortion.
[0011] A particular problem is that in a holographic projector arranged to display a multi-color reconstruction, for example using multiple monochromatic channels (i.e. multiple single channels operating at different wavelengths), the distortion varies with wavelength. In such an example, each monochromatic channel can be arranged to holographically project a monochromatic image (e.g. a red, green or blue image). The holographic projector can be arranged to independently (in space and / or time) project multiple monochromatic images (e.g. independently project a red image, a green image and a blue image). A (monochromatic) holographic reconstruction associated with each channel can be formed on the replay plane. The multi-color holographic reconstruction is formed as a combination of multiple monochromatic holographic reconstructions at the replay plane. Multiple monochromatic channels can at least partially overlap and can even use the same spatial light modulator, for example in a frame sequential color scheme. Ideally, multiple monochromatic holographic reconstructions will be aligned with each other. This may mean that, ideally, the corresponding image points of each monochromatic holographic reconstruction are aligned with each other at the replay plane. However, because each monochromatic image / single holographic reconstruction may be distorted differently from each other due to each monochromatic having a different wavelength (and the amount of distortion depends on the wavelength), complex misalignments may occur between different color channels. The misalignment may be non-uniform. The human eye is sensitive to misalignment, and even relatively small misalignments between image pixels of a monochrome image can significantly degrade the perceived image quality. This problem is exacerbated in heads-up displays, which include optical playback systems designed to perform complex optical processing and meet stringent optical and physical constraints.
[0012] The distortion of the holographic reconstruction (image) can be compensated by pre-distorting the target picture before display and / or before the hologram is calculated. This can include transforming the image points of the target picture (at a given temperature and wavelength) in a manner opposite to the distortion caused by the holographic projector, and then displaying the hologram of the pre-distorted target picture on the display device of the holographic projector. As a result, the distortion caused by the holographic projector may have been compensated, resulting in the holographic reconstruction looking as expected. The distortion map can be used to pre-distort the target picture. The distortion map may include multiple mappings of a two-dimensional coordinate array. The two-dimensional coordinate array may be associated with the coordinates of the target picture. Each mapping of the distortion can transform one of the two-dimensional coordinates. The distortion map may have been determined and verified by experiments and / or simulations to ensure that the mapping pre-distorts the target picture in a manner that accurately compensates for the distortion caused by the holographic projector so that the holographic reconstruction looks as expected. The transformed two-dimensional coordinates can be used to pre-distort the target picture.
[0013] As described above, the distortion of the holographic reconstruction may depend on the current temperature of the holographic projector and / or the wavelength of the spatially modulated light (according to the corresponding hologram). Therefore, when using spatially modulated light of a first wavelength and at a first temperature, the first distortion map may only accurately pre-distort the target image. For example, when using spatially modulated light of a second wavelength (different from the first wavelength) and / or when the holographic projector is at a second temperature (different from the first temperature), the first distortion map may not accurately pre-distort the target image. A conventional solution to this problem is to program the memory of the holographic projector to store multiple distortion maps determined and verified at multiple different temperatures and wavelengths. The holographic projector can be arranged to select an appropriate distortion map from the memory for a given scene using a lookup table. However, the inventors have recognized that this solution is resource inefficient / non-optimal because it requires determining and verifying a very large number of distortion maps. For example, a holographic projector intended for use in a vehicle, such as part of a head-up display in a vehicle, must be able to operate over a large temperature range (which may extend over a range of at least 100 degrees Celsius). In order to cover such a large range, a large number of distortion maps need to be determined and verified at different temperatures (e.g., for every degree Celsius of the temperature range). For a multi-color holographic projector, this needs to be repeated for each color channel. The inventors have recognized that determining and verifying such a large number of distortion maps is very time-consuming and resource-consuming. In addition, the process needs to be repeated for each model of holographic projector, or when the holographic projector is used in different environments (e.g., in different vehicles with different geometries). In addition, the inventors have recognized that the memory requirements for storing such a large number of distortion maps may be prohibitive. In addition, the above-mentioned distortion map may be only part of one of the multiple calibration processes performed by the holographic projector. For example, the holographic projector may also need to calibrate the hologram to compensate for the shape of an optical combiner with a complex shape, such as a windshield of a vehicle, and / or the hologram may be adjusted based on the eye tracking of the user of the holographic projector. Both examples may require a separate calibration process (in addition to the distortion correction that is the subject of the present disclosure). Therefore, the above-mentioned pre-distortion to compensate for wavelength and temperature changes may be only a small part of the projection / calibration process. It is therefore important that each calibration step is as resource-efficient as possible. This is another reason why conventional methods for fully characterizing holographic projectors at multiple temperatures and wavelengths are not optimal.
[0014] To address the problem of determining and storing a large number of distortion maps, the inventors have developed a fast, resource-efficient and time-efficient method of calculating / determining distortion maps. The method can be performed in real-time / on-the-fly, such as when the holographic projector is running (e.g., when a series of holograms are generated by a computer). The method of calculating (distortion) maps according to the present disclosure requires only one distortion map (a first distortion map) as input, which is predetermined and pre-validated (at a specific first temperature) for each color channel. Therefore, the method advantageously avoids the need to fully characterize the holographic projector to determine the distortion maps at multiple temperatures.
[0015] Real-time projection of a series of computer-generated holograms is computationally very intensive and expensive. It is counterintuitive for the inventors to add an additional real-time computational step (computing a distortion map, rather than retrieving a map from memory) to the holographic projection method. However, the inventors surprisingly discovered that the method according to the present disclosure can modify the first distortion map with simple and inexpensive calculations, and the modified map can be used to pre-distort the target picture so that the holographic reconstruction looks as expected, and the multi-color holographic reconstruction is aligned using processing / computation that occurs in real time during display. In particular, after thorough simulations and experiments, the inventors found that the position changes of the above-mentioned coordinate arrays follow a basically predictable relationship over the temperature range. In some embodiments, this is a linear relationship. The inventors have realized that by modifying the first distortion map using a scaling vector that represents how the coordinate array changes between the second temperature and the third temperature, the relationship can be used to simply calculate the distortion map for any current temperature. Importantly, according to the present disclosure, the steps of the method / computation can be repeatedly performed by a holographic projector. For example, the steps of the method can be performed in each frame or subframe of a real-time image projection system such as a data stream. The steps of the method may be performed / repeated when the current temperature changes by or exceeds a predetermined amount. Again, this goes against a significant bias in the prior art, which teaches that computations should be minimized in order to achieve high quality real-time performance. The inventors have identified a simple approach that they have found to be worth the additional processing burden.
[0016] In a first aspect of the present disclosure, a method for calculating a (distortion) map in real time is provided. The (calculated) distortion map is used to distort (also referred to herein as "pre-distort") a target image to be projected by a holographic projector. The (pre-) distortion of the target image is to compensate for the current temperature of the holographic projection and the current temperature change of the holographic projector. The method includes the step of receiving a calibration map. The calibration map includes a plurality of mappings. Each mapping is used to transform a two-dimensional coordinate. Each two-dimensional coordinate corresponds to one or more image points of the target image. A point or position on a two-dimensional plane defined by a two-dimensional coordinate may be referred to as a calibration point. Each calibration point may correspond to one or more image points of the target image. In an embodiment, each two-dimensional coordinate corresponds to one or more image points of an undistorted target image. Each mapping of the calibration map may be used to transform the corresponding two-dimensional coordinate to compensate for the distortion at a predetermined first temperature. In other words, each mapping of the calibration map may be used to pre-distort the image points of the target image to compensate for the distortion caused by the projection process at the first temperature. As used herein, a "calibration map" refers to a distortion map that has been (fully) determined and verified by experiments and / or simulations. The process by which the calibration map can be fully determined is not important for the method of the present invention. Thus, the method step comprising receiving a calibration map may comprise receiving a predetermined map. The predetermined calibration map is determined when the holographic projector is at a first temperature and the spatially modulated light has a first wavelength. The method according to the present disclosure advantageously requires receiving only one calibration map (at the first temperature) to calculate the (modified) map for the current temperature (when the spatially modulated light has the first wavelength). The calibration map may be referred to herein as a first distortion map or an initial distortion map.
[0017] The method also includes the step of receiving a vector array. The vector array includes / consists of a vector for each two-dimensional coordinate in the two-dimensional coordinate array. There may be a one-to-one ratio between the number of vectors in the vector array and the number of two-dimensional coordinates in the two-dimensional coordinate array. Each vector may be uniquely associated with a two-dimensional coordinate, and vice versa. Similarly, there may be a one-to-one ratio between the number of vectors in the vector array and the number of mappings in the calibration map. Each vector may be uniquely associated with a mapping in the calibration map, and vice versa. Each vector represents a calibration change of a corresponding two-dimensional coordinate (two-dimensional coordinate array) within a predetermined temperature range. As used herein, a calibration change is a change of each corresponding two-dimensional coordinate from a second temperature to a third temperature, assuming / when the two-dimensional coordinates have been transformed to compensate for distortion at the second temperature and the third temperature, respectively. The temperature range includes a second and a third temperature. In some embodiments, the second temperature corresponds to the lowest temperature of the (predetermined) temperature range. In some embodiments, the third temperature corresponds to the highest temperature of the (predetermined) temperature range. Image calibration Figure 1 Similarly, the steps of the method for determining a vector array may not be necessary for the method of the present disclosure. Thus, the steps of the method include receiving a predetermined vector array. The vector array may be referred to herein as a vector map or a simulation map.
[0018] The method also includes receiving or determining a current temperature of the holographic projector. The current temperature may be referred to herein as a fourth temperature.
[0019] The method also includes the step of determining a scaling factor based on the difference between the current (fourth) temperature and the predetermined (first) temperature. Determining the scaling factor based on the difference between the current (fourth) temperature and the predetermined (first) temperature may include additional steps or mathematical operations (besides determining the difference between the current temperature and the predetermined temperature). For example, the output of the difference between the current (fourth) temperature and the predetermined (first) temperature may be multiplied or divided by a constant.
[0020] The method also includes the step of calculating a modification map based on / for the current temperature. For each coordinate in the two-dimensional coordinate array, the step of calculating the modification map includes: multiplying a vector associated with the corresponding coordinate in the two-dimensional coordinate array by a scaling factor to output a scaling vector; and applying the scaling vector to the corresponding mapping of the calibration map. The step of applying the scaling vector to the corresponding mapping of the calibration map may include adding the scaling vector to the corresponding mapping or subtracting the scaling vector from it. The step of applying the scaling vector to the corresponding mapping of the calibration map may include outputting a modification map for transforming the corresponding two-dimensional coordinates to compensate for distortion of the target image when it is projected by the holographic projector at the current (fourth) temperature.
[0021] The method further comprises the step of outputting a modified map. Modifying the map comprises modifying the mapping array (computed in the previous step of the method).
[0022] In some embodiments, the method further comprises the step of receiving the two-dimensional coordinate array. In some embodiments, there is a many-to-one ratio between the number of image points in the target picture and the number of two-dimensional coordinates in the two-dimensional coordinate array. In other words, each two-dimensional coordinate may correspond to more than one, optionally more than five, optionally more than 10 image points. Alternatively, there may be fewer image points than two-dimensional coordinates. Because the (calibration and modification) map of the method includes as many mappings as two-dimensional coordinates (and the calibration map includes as many vectors as two-dimensional coordinates), it is advantageous to have a many-to-one ratio between the number of image points in the target picture and the number of two-dimensional coordinates in the two-dimensional coordinate array. This is because for a given number of image points, there are fewer mappings to be calculated in the modified map, so the method is computationally cheaper than when there is a larger number of mappings. When the mappings of the modified map have a many-to-one correspondence with the image points, interpolation can be advantageously used to determine appropriate pre-distortion of the image points of the target picture.
[0023] The vectors in the vector array may have an uneven distribution of directions and amplitudes. Typically, the two-dimensional coordinates corresponding to image points closer to the center of the target image will be less distorted than the two-dimensional coordinates closer to the edge / periphery of the target image. Typically, the two-dimensional coordinates corresponding to image points on opposite sides of the target image will be distorted / transformed in opposite directions to each other. The vector array may represent these two effects. For example, the vector amplitude of the vector array may increase as the distance of the corresponding two-dimensional coordinate from the center of the (two-dimensional coordinate) array increases. The vector direction of the vector array corresponding to the two-dimensional coordinates on the first side of the array center may be opposite to the vector direction of the vector array corresponding to the two-dimensional coordinates on the second side of the array center (opposite to the first side). In some embodiments, each vector direction of the vector array may be substantially parallel to the direction defined from the center of the (two-dimensional coordinate) array to the corresponding two-dimensional coordinate.
[0024] In some embodiments, the scaling factor has a linear dependence on the current / fourth temperature. After thorough simulations and experiments, the inventors were surprised to find that the distortion of each image point / each calibration point of the target picture represented by the corresponding two-dimensional coordinates varies substantially linearly with temperature. In some embodiments, the step of determining the scaling factor includes determining the difference between the current temperature and the predetermined temperature, and dividing the difference by the predetermined temperature range. In such embodiments, the scaling factor may also be referred to as a percentage. Such a scaling factor represents the change in the current temperature relative to the first temperature as a proportion of a predetermined temperature range. If the current temperature is equal to the first temperature, the percentage change will be zero. Therefore, the scaling factor is equal to zero. Otherwise, the scaling factor may be non-zero.
[0025] In some embodiments, the step of determining the scaling factor includes calculating as follows: (T-T0) / (Tmax-Tmin). T is the current temperature, T0 is the predetermined temperature, Tmax is the highest (third) temperature of the predetermined temperature range, and Tmin is the lowest (second) temperature of the predetermined temperature range. In other words, the scaling factor may be equal to (T-T0) / (Tmax-Tmin).
[0026] In some embodiments, the method further includes applying the modification map to the two-dimensional coordinate array to output a modified two-dimensional coordinate array. In some embodiments, the method further includes receiving a target image including a plurality of image points, wherein each two-dimensional coordinate in the two-dimensional coordinate array corresponds to one or more image points of the target image. Each corresponding two-dimensional coordinate in the modification array may correspond to the same corresponding one or more image points of the target image. In some embodiments, the method further includes distorting the target image based on the modified two-dimensional coordinate array. In some embodiments, the method further includes calculating a hologram of the distorted target image.
[0027] In some embodiments, the step of determining the current / fourth temperature may include measuring the current / fourth temperature of the holographic projector, for example using a temperature sensor. Optionally, the temperature sensor may be arranged to measure the temperature of a light source (e.g., a laser) of the holographic projector. In other embodiments, the step of determining the current / fourth temperature may include inferring the current / fourth temperature of the holographic projector by measuring the wavelength of the spatially modulated light. For example, the method may include comparing the current wavelength of the spatially modulated light with the expected wavelength of the spatially modulated light. As will be appreciated by the skilled person, the current temperature of the holographic projector / holographic projector light source may be determined based on the deviation of the wavelength of the light from the expected wavelength.
[0028] As described above, the method according to the present disclosure can be performed in real time. The method can be repeatedly performed during the operation of the holographic projector. In some embodiments, the holographic projector is arranged to project a series of (image) frames in a manner familiar to the reader—for example, at a video rate. Each frame can have a different hologram (with a different target picture associated). In some embodiments, the method according to the present disclosure can be performed once for each frame in a series. Therefore, the modification (distortion) map is recalculated for each frame in a series. Then, the recalculated modification (distortion) map can be applied to the corresponding target picture of the corresponding frame in a series (to pre-distort the target picture), and then the hologram of the pre-distorted target picture can be calculated. In other embodiments, the method according to the present disclosure can be repeated only when it is detected that the current temperature has changed by an amount exceeding a threshold. In such an example, the method may include calculating a first modification (distortion) map at an initial temperature. The method may include, for each frame in a series, monitoring the current temperature. If the difference between the current temperature and the initial temperature is less than a threshold, the method may include reusing the first modification (distortion) map to pre-distort the target picture of the corresponding frame. If the difference between the current temperature and the initial temperature is greater than a threshold, the method may include recalculating the modification map (according to the above method). The method may then include pre-distorting the target picture of the corresponding frame using the recomputed modification map. In some embodiments, the method is performed on at least two consecutive frames in the input image frame stream. In some embodiments, the method is performed on each frame in the input image frame stream.
[0029] In some embodiments, the target image is a first target image. In some embodiments, the two-dimensional coordinate array is a first two-dimensional coordinate array. In some embodiments, the calibration map is a first calibration map. In some embodiments, the vector array is a first vector array. In some embodiments, the modification map is a first modification map. In some embodiments, the method further comprises calculating a second map for distorting a second target image to be projected by the holographic projector. The calculation of the second map may include receiving a second two-dimensional coordinate array. Each two-dimensional coordinate may correspond to one or more image points of the second target image. The second target image may be different from the first target image. The calculation of the second map may include receiving a second calibration map of the second two-dimensional coordinate array. The second calibration map may include a mapping for each two-dimensional coordinate in the second array to transform the corresponding two-dimensional coordinates to compensate for the distortion at a predetermined temperature. The predetermined temperature may be the same or different from the predetermined first temperature mentioned above. The calculation of the second map may include receiving a second vector array, the second vector array including a vector of each two-dimensional coordinate (of the second two-dimensional coordinate array). Each vector may represent a position change of a corresponding two-dimensional coordinate within a predetermined temperature range. The calculation of the second map may include calculating a second modification map based on the current temperature for each coordinate in the second two-dimensional coordinate array by:
[0030] multiplying the vector associated with the corresponding coordinate of the second two-dimensional coordinate array by the scaling factor to output a scaled vector; and
[0031] applying the scaling vector to the corresponding map of the second calibration map;
[0032] Output the second modified map.
[0033] When a first wavelength is used in the holographic projection of the first target picture, a mapping of the first calibration map and the vectors of the first vector array may have been determined. When a second wavelength (different from the first wavelength) is used in the holographic projection of the second target picture, a mapping of the second calibration map and the vectors of the second calibration map may have been determined.
[0034] The method may also include applying the second modification map to the second two-dimensional coordinate array to output a second modified two-dimensional coordinate array. The method may also include distorting the second target image based on the modification map. The method may also include calculating a second hologram of the distorted second target image.
[0035] In a second aspect, a method for calculating a mapping of a graph is provided, the mapping being used to distort at least a portion of a target image to be projected by a holographic projector and to compensate for current temperature changes of the holographic projector. The method includes a step of receiving a first mapping for transforming a first two-dimensional coordinate to compensate for distortion at a predetermined (first) temperature. The method also includes a step of receiving a first vector, the first vector representing a calibration change of the first two-dimensional coordinate within a predetermined temperature range (e.g., a change of the first two-dimensional coordinate between a second / lowest temperature and a third / highest temperature). The method also includes receiving or determining the current (fourth) temperature of the holographic projector. The method also includes determining a scaling factor based on the difference between the current temperature and the predetermined temperature. The method also includes calculating a first modified mapping based on / compensating for the current temperature (by modifying the first mapping). The first modified mapping may form part of a first modified graph. The step of calculating the first modified mapping includes multiplying the first vector by a scaling factor to output a scaling vector. The method also includes applying the scaling vector to the first mapping. The method may also include outputting a modified first mapping for transforming the first two-dimensional coordinate to compensate for distortion of a portion of the target image when projected by the holographic projector at the current (fourth) temperature.
[0036] The method according to the second aspect can be repeated to calculate any number of mappings of the first figure. For example, the first figure may include multiple first mappings. Each first mapping can be used to transform a corresponding two-dimensional coordinate. Each two-dimensional coordinate can form a part of a two-dimensional coordinate array. The method according to the second aspect can be repeated for multiple first mappings of the first figure. In some embodiments, the method according to the second aspect can be repeated for a subset or part of the first mapping of the first figure. In some embodiments, the method according to the second aspect can be repeated for all first mappings of the first figure. Each time the method is repeated, the method may include receiving another vector. For example, if the method includes receiving a second mapping (for transforming a second two-dimensional coordinate), the method includes receiving a second vector (representing a change in the second two-dimensional coordinate).
[0037] In a third aspect, a holographic projector is provided, including a display device arranged to display a hologram of a picture to spatially modulate the light incident thereon according to the hologram, wherein the holographic projector is arranged to form a holographic reconstruction of the picture at a playback plane. The holographic projector also includes a controller, which includes a memory storing a calibration diagram and a vector array. The calibration diagram includes a plurality of mappings. Each mapping is used to transform the corresponding two-dimensional coordinates of the two-dimensional coordinate array to compensate for the distortion at a predetermined temperature, and each two-dimensional coordinate corresponds to one or more image points of the target picture. The vector array includes a vector of each two-dimensional coordinate, and each vector represents a calibration change of each corresponding two-dimensional coordinate within a predetermined temperature range. The controller is also arranged to determine the current temperature of the holographic projector (particularly the temperature of the light source). For example, the holographic projector may include a temperature sensor arranged to measure the temperature of the light source of the holographic projector. The controller may be arranged to receive a signal from the temperature sensor. The controller may be further arranged to determine the scaling factor based on the difference between the current temperature and the predetermined temperature. The controller may be further arranged to calculate the modification map based on the current temperature for each coordinate in the two-dimensional coordinate array by: multiplying a vector associated with the corresponding coordinate in the two-dimensional coordinate array by a scaling factor to output a scaling vector; and applying the scaling vector to the corresponding mapping of the calibration map. The controller may also be arranged to output the modification map.
[0038] The features and advantages described in relation to one aspect of the present disclosure may be applicable to other aspects of the present disclosure. In particular, the features and advantages described in relation to the method of the first aspect may be applicable to the method of the second aspect and the holographic projector of the third aspect.
[0039] In the present disclosure, the term "replica" is used only to reflect that the spatially modulated light is split so that the composite light field is directed along multiple different optical paths. The word "replica" is used to refer to each occurrence or instance of the composite light field after a replication event - such as a partial reflection-transmission of a pupil expander. Each replica propagates along a different optical path. Some embodiments of the present disclosure relate to the propagation of light encoded with a hologram rather than an image - that is, light that is spatially modulated with a hologram of an image rather than the image itself. It can therefore be said that multiple replicas of the hologram are formed. Those skilled in the art of holography will understand that the composite light field associated with the propagation of light encoded with a hologram will vary with the propagation distance. The term "replica" as used herein is independent of the propagation distance, so two branches or paths of light associated with a replication event are still referred to as "replicas" of each other, even if the branches have different lengths, so that the composite light field evolves differently along each path. That is, according to the present disclosure, even if two composite light fields are associated with different propagation distances, they are still considered "replicas" - as long as they are generated from the same replication event or series of replication events.
[0040] According to the present disclosure, a "diffraction light field" or "diffracted light field" is a light field formed by diffraction. A diffraction light field can be formed by illuminating a corresponding diffraction pattern. According to the present disclosure, an example of a diffraction pattern is a hologram, and an example of a diffraction light field is a holographic light field or a holographically reconstructed light field that forms an image. The holographic light field forms a (holographic) reconstruction of the image on a playback plane. The holographic light field propagated from the hologram to the playback plane can be said to include light encoded with the hologram or light in the holographic domain. The diffraction light field is characterized by a diffraction angle determined by the minimum characteristic size of the diffraction structure and the wavelength of light (of the diffraction light field). According to the present disclosure, it can also be said that the "diffraction light field" is a reconstructed light field formed on a plane spatially separated from the corresponding diffraction structure. An optical system for propagating a diffraction light field from a diffraction structure to an observer is disclosed herein. The diffraction light field can form an image.
[0041] The term "hologram" is used to refer to a record containing amplitude information or phase information or some combination thereof about an object. The term "holographic reconstruction" is used to refer to the optical reconstruction of an object formed by illuminating a hologram. The system is described as a "holographic projector" because the holographic reconstruction is a real image and is spatially separated from the hologram. The term "replay field" is used to refer to the 2D area within which the holographic reconstruction is formed and is fully focused. If the hologram is displayed on a spatial light modulator comprising pixels, the replay field will be repeated in the form of multiple diffraction orders, each of which is a replica of the zero-order replay field. The zero-order replay field usually corresponds to the preferred or main replay field because it is the brightest replay field. Unless otherwise explicitly stated, the term "replay field" should be considered to refer to the zero-order replay field. The term "replay plane" is used to refer to a plane in space that contains all replay fields. The terms "image", "replay image" and "image area" refer to the area of the replay field illuminated by the light of the holographic reconstruction. In some embodiments, an "image" may include discrete points, which may be referred to as "image points," or, for convenience only, "image pixels."
[0042] The terms "encoding", "writing" and "addressing" are used to describe the process of providing a plurality of pixels of the SLM with a corresponding plurality of control values that respectively determine the modulation level of each pixel. It can be said that the pixels of the SLM are configured to "display" a light modulation distribution in response to receiving the plurality of control values. Thus, the SLM can be said to "display" a hologram, and a hologram can be considered to be an array of light modulation values or levels.
[0043] It has been found that holographic reconstructions of acceptable quality can be formed from "holograms" that contain only phase information related to the Fourier transform of the original object. Such holographic recordings may be referred to as phase-only holograms. The embodiments relate to phase-only holograms, but the disclosure is equally applicable to amplitude-only holography.
[0044] The present disclosure is also equally applicable to forming a holographic reconstruction using amplitude and phase information related to the Fourier transform of the original object. In some embodiments, this is achieved by using complex modulation of a so-called fully complex hologram that contains amplitude and phase information related to the original object. Because the value (gray level) assigned to each pixel of the hologram has an amplitude and a phase component, such a hologram may be referred to as a fully complex hologram. The value (gray level) assigned to each pixel can be represented as a complex number with an amplitude and a phase component. In some embodiments, a fully complex computer-generated hologram is calculated.
[0045] Reference may be made to the phase value, phase component, phase information, or simply phase, of a pixel of a computer-generated hologram or spatial light modulator as shorthand for "phase delay". That is, any phase value described is actually a number (e.g., in the range of 0 to 2π) representing the amount of phase delay provided by that pixel. For example, a pixel of a spatial light modulator described as having a phase value of π / 2 will delay the phase of the received light by π / 2 radians. In some embodiments, each pixel of the spatial light modulator may be operated at one of a plurality of possible modulation values (e.g., phase delay values). The term "grayscale" may be used to refer to a plurality of available modulation levels. For example, the term "grayscale" may be used for convenience to refer to a plurality of available phase levels in a phase modulator alone, even if the different phase levels do not provide different shades of gray. For convenience, the term "grayscale" may also be used to refer to a plurality of available complex modulation levels in a complex modulator.
[0046] Thus, a hologram comprises an array of gray levels, i.e. an array of light modulation values, such as phase delay values or an array of complex modulation values. A hologram is also considered to be a diffraction pattern, since it is a pattern that causes diffraction when displayed on a spatial light modulator and illuminated with light of a wavelength relative to (usually less than) the pixel pitch of the spatial light modulator. Reference is made herein to combining a hologram with other diffraction patterns, such as a diffraction pattern used as a lens or a grating. For example, a diffraction pattern used as a grating may be combined with a hologram to shift the replay field on the replay plane, or a diffraction pattern used as a lens may be combined with a hologram to focus the holographic reconstruction on the replay plane in the near field.
[0047] Although different embodiments and embodiment groups may be disclosed separately in the following detailed description, any feature of any embodiment or embodiment group may be combined with any other feature or combination of features of any embodiment or embodiment group. That is, all possible combinations and permutations of the features disclosed in this disclosure are contemplated. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] Specific embodiments are described by way of example only with reference to the following drawings:
[0049] Figure 1is a schematic diagram showing a reflective SLM producing a holographic reconstruction on a screen;
[0050] Figure 2A A first iteration of an example Gerchberg-Saxton type algorithm is shown;
[0051] Figure 2B The second and subsequent iterations of an example Gerchberg-Saxton type algorithm are shown;
[0052] Figure 2C An alternative second and subsequent iteration of an example Gerchberg-Saxton type algorithm is shown;
[0053] Figure 3 is a schematic diagram of a reflective LCOS SLM;
[0054] Figure 4 An example HUD in a vehicle is shown;
[0055] Figure 5 shows the process of pre-distorting a target image for holographic projection;
[0056] Fig. 6A Indicates that when the distortion map is applied to Figure 5 a pre-distorted target image after the target image of ;
[0057] Figure 6B express Fig. 6A A virtual image of a holographic reconstruction of a pre-distorted target image;
[0058] Figure 7 A flow chart representing a method according to the present disclosure is shown;
[0059] Figure 8 Schematically shows a vector being scaled;
[0060] Fig. 9 Schematically represents a process of pre-distorting a target image using a modified distortion map calculated using the method according to the present disclosure before calculating a hologram of the target image;
[0061] Fig.10 A schematic diagram showing a multi-color holographic projector including multiple color channels; and
[0062] Fig.11 Represents the misalignment of image points in the holographic reconstruction from the first (green) channel and the second (blue) channel.
[0063] The same reference numbers will be used throughout the drawings to refer to the same or like parts. DETAILED DESCRIPTION
[0064] The present invention is not limited to the embodiments described below, but extends to the full scope of the appended claims. That is, the present invention can be implemented in different forms and should not be construed as limited to the described embodiments, which are set forth for illustrative purposes.
[0065] A structure described as being formed on / under another structure or above / below another structure should be construed to include the case where the structures are in contact with each other and, further, also include the case where a third structure is provided therebetween.
[0066] When describing a temporal relationship, for example, when the temporal order of events is described as "after," "subsequently," "next," "before," etc., the present disclosure should be considered to include both sequential and non-sequential events unless otherwise specified. For example, unless terms such as "just," "immediately," or "directly" are used, the description should be considered to include non-sequential situations.
[0067] Although the terms "first", "second", etc. may be used herein to describe various elements, these elements are not limited by these terms. These terms are only used to distinguish between various elements. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element without departing from the scope of the appended claims.
[0068] The features of different embodiments may be coupled or combined with each other in part or in whole, and may interoperate with each other in different ways. Some embodiments may be performed independently of each other, or may be performed together in a mutually dependent relationship.
[0069] Optical Configuration
[0070] Figure 1 An embodiment is shown in which a computer generated hologram is encoded on a single spatial light modulator. The computer generated hologram is a Fourier transform of the object used for reconstruction. Therefore, it can be said that the hologram is a Fourier domain or frequency domain or spectral domain representation of the object. In this embodiment, the spatial light modulator is a reflective liquid crystal on silicon "LCOS" device. The hologram is encoded on the spatial light modulator and a holographic reconstruction is formed at a playback field, such as a light receiving surface such as a screen or diffuser.
[0071] A light source 110, such as a laser or laser diode, is arranged to illuminate the SLM 140 via a collimating lens 111. The collimating lens causes a substantially planar wavefront of light to be incident on the SLM. Figure 1 In some embodiments, the direction of the wavefront is off-normal (e.g., two or three degrees from a plane that is truly normal to the transparent layer). However, in other embodiments, a substantially planar wavefront is provided at normal incidence and a beam splitter arrangement is used to separate the input and output optical paths. Figure 1In the embodiment shown, the arrangement is such that light from the light source reflects from the mirrored rear surface of the SLM and interacts with the light modulating layer to form an exit wavefront 112. The exit wavefront 112 is applied to an optical device including a Fourier transform lens 120, the focus of which is located at a screen 125. More specifically, the Fourier transform lens 120 receives the modulated light beam from the SLM 140 and performs a frequency-space transform to produce a holographic reconstruction at the screen 125.
[0072] It is important to note that in this type of hologram, every pixel of the hologram contributes to the entire reconstruction. There is no one-to-one correlation between a specific point on the replay field (or image pixel) and a specific light modulation element (or hologram pixel). In other words, the modulated light leaving the light modulation layer is distributed over the entire replay field.
[0073] In these embodiments, the position of the holographic reconstruction in space is determined by the power (focus) of the Fourier transform lens. Figure 1 In the illustrated embodiment, the Fourier transform lens is a physical lens. That is, the Fourier transform lens is an optical Fourier transform lens and performs a Fourier transform optically. Any lens can act as a Fourier transform lens, but the performance of the lens will limit the accuracy of the Fourier transform it performs. The skilled person understands how to use a lens to perform an optical Fourier transform.
[0074] Hologram computing
[0075] In some embodiments, the computer generated hologram is a Fourier transform hologram, or simply a Fourier hologram or a Fourier based hologram, where the image is reconstructed in the far field by utilizing the Fourier transform properties of a positive lens. The Fourier hologram is calculated by Fourier transforming the desired light field in the replay plane back to the lens plane. The computer generated Fourier hologram can be calculated using the Fourier transform.
[0076] The Fourier transform hologram may be calculated using an algorithm such as the Gerchberg-Saxton algorithm. In addition, the Gerchberg-Saxton algorithm may be used to calculate a hologram in the Fourier domain (i.e., a Fourier transform hologram) from amplitude-only information in the spatial domain (e.g., a photograph). Phase information about the object is effectively "retrieved" from amplitude-only information in the spatial domain. In some embodiments, a computer-generated hologram is calculated from amplitude-only information using the Gerchberg-Saxton algorithm or a variation thereof.
[0077] The Gerchberg-Saxton algorithm considers that when the intensity cross-section I of the beam in planes A and B is known, A (x,y) and I B(x,y) and I A (x,y) and I B When (x,y) is related by a single Fourier transform. For a given intensity cross section, the phase distribution approximation Ψ in planes A and B is obtained separately A (x,y) and Ψ B (x, y). The Gerchberg-Saxton algorithm finds a solution to this problem by following an iterative process. More specifically, the Gerchberg-Saxton algorithm iteratively applies spatial and spectral constraints while repeatedly transferring representations of I between the spatial domain and the Fourier (spectral or frequency) domain. A (x,y) and I B A data set (amplitude and phase) of (x,y). A corresponding computer generated hologram in the spectral domain is obtained by at least one iteration of the algorithm. The algorithm is convergent and arranged to produce a hologram representing the input image. The hologram can be an amplitude-only hologram, a phase-only hologram, or a full complex hologram.
[0078] In some embodiments, a phase-only hologram is calculated using an algorithm based on the Gerchberg-Saxton algorithm, such as the algorithm described in British Patent Nos. 2,498,170 or 2,501,112, the entire contents of which are incorporated herein by reference. However, the embodiments disclosed herein describe calculating a phase-only hologram by way of example only. In these embodiments, the Gerchberg-Saxton algorithm retrieves the phase information Ψ[u,v] of the Fourier transform of a data set, which produces known amplitude information T[x,y], where the amplitude information T[x,y] represents a target image (e.g., a photograph). Since amplitude and phase are essentially combined in the Fourier transform, the transformed amplitude and phase contain useful information about the accuracy of the calculated data set. Therefore, the algorithm can be used iteratively with feedback of the amplitude and phase information. However, in these embodiments, only the phase information Ψ[u,v] is used as a hologram to form a holographic representation of the target image at the image plane. The hologram is a data set (e.g., a 2D array) of phase values.
[0079] In other embodiments, an algorithm based on the Gerchberg-Saxton algorithm is used to calculate a full complex hologram. A full complex hologram is a hologram having an amplitude component and a phase component. A hologram is a data set (e.g., a 2D array) comprising an array of complex data values, where each complex data value comprises an amplitude component and a phase component.
[0080] In some embodiments, the algorithm processes complex data and the Fourier transform is a complex Fourier transform. The complex data can be considered to include (i) a real component and an imaginary component, or (ii) an amplitude component and a phase component. In some embodiments, the two components of the complex data are processed differently at various stages of the algorithm.
[0081] Figure 2A A first iteration of an algorithm for calculating a phase-only hologram according to some embodiments is shown. The input to the algorithm is an input image 210 comprising a 2D array of pixels or data values, each of which is an amplitude or magnitude value. That is, each pixel or data value of the input image 210 does not have a phase component. Therefore, the input image 210 can be viewed as an amplitude-only or amplitude-only or intensity-only distribution. An example of such an input image 210 is a frame of a photograph or a video comprising a time sequence of frames. The first iteration of the algorithm begins with a data formation step 202A, which includes assigning a random phase value to each pixel of the input image using a random phase distribution (or random phase seed) 230 to form a starting complex data set, wherein each data element of the data set comprises an amplitude and a phase. It can be said that the starting complex data set represents the input image in the spatial domain.
[0082] The first processing block 250 receives the starting complex data set and performs a complex Fourier transform to form a Fourier transformed complex data set. The second processing block 253 receives the Fourier transformed complex data set and outputs a hologram 280A. In some embodiments, the hologram 280A is a phase-only hologram. In these embodiments, the second processing block 253 quantizes each phase value and sets each amplitude value to 1 to form the hologram 280A. Each phase value is quantized according to the phase level that can be represented on the pixel of the spatial light modulator that will be used to "display" the phase-only hologram. For example, if each pixel of the spatial light modulator provides 256 different phase levels, each phase value of the hologram is quantized to one of the 256 possible phase levels. The hologram 280A is a phase-only Fourier hologram representing the input image. In other embodiments, the hologram 280A is a full complex hologram that includes an array of complex data values (each including an amplitude component and a phase component) derived from the received Fourier transformed complex data set. In some embodiments, the second processing block 253 constrains each complex data value to one of a plurality of allowable complex modulation levels to form the hologram 280A. The constraining step may include setting each complex data value to the closest allowable complex modulation level in the complex plane. It can be said that the hologram 280A represents the input image in the spectral or Fourier or frequency domain. In some embodiments, the algorithm stops at this point.
[0083] However, in other embodiments, the algorithm continues as Figure 2AIn other words, following Figure 2A The steps indicated by dashed arrows are optional (ie, not essential to all embodiments).
[0084] The third processing block 256 receives the modified complex data set from the second processing block 253 and performs an inverse Fourier transform to form an inverse Fourier transformed complex data set. The inverse Fourier transformed complex data set can be said to represent the input image in the spatial domain.
[0085] The fourth processing block 259 receives the inverse Fourier transformed complex data set and extracts the distribution of amplitude values 211A and the distribution of phase values 213A. Optionally, the fourth processing block 259 evaluates the distribution of amplitude values 211A. Specifically, the fourth processing block 259 can compare the distribution of amplitude values 211A of the inverse Fourier transformed complex data set with the input image 510, which itself is of course a distribution of amplitude values. If the difference between the distribution of amplitude values 211A and the input image 210 is small enough, the fourth processing block 259 can determine that the hologram 280A is acceptable. That is, if the difference between the distribution of amplitude values 211A and the input image 210 is small enough, the fourth processing block 259 can determine that the hologram 280A is a sufficiently accurate representation of the input image 210. In some embodiments, for comparison purposes, the distribution of phase values 213A of the inverse Fourier transformed complex data set is ignored. It will be appreciated that any number of different methods may be employed to compare the distribution of amplitude values 211A and the input image 210, and the present disclosure is not limited to any particular method. In some embodiments, a mean square error is calculated, and if the mean square error is less than a threshold, the hologram 280A is deemed acceptable. If the fourth processing block 259 determines that the hologram 280A is unacceptable, further iterations of the algorithm may be performed. However, this comparison step is not required, and in other embodiments, the number of iterations of the algorithm performed is predetermined or preset or user defined.
[0086] Figure 2B 202B. The second iteration of the algorithm and any further iterations of the algorithm are represented. The distribution of phase values 213A of the previous iteration is fed back through the processing blocks of the algorithm. The distribution of amplitude values 211A is rejected in favor of the distribution of amplitude values of the input image 210. In the first iteration, the data forming step 202A forms a first complex data set by combining the distribution of amplitude values of the input image 210 with the random phase distribution 230. However, in the second and subsequent iterations, the data forming step 202B includes forming a complex data set by combining (i) the distribution of phase values 213A from the previous iteration of the algorithm with (ii) the distribution of amplitude values of the input image 210.
[0087] Then, refer to Figure 2AThe same method as described is handled by Figure 2B The complex data set formed by the data forming step 202B is used to form a second iterative hologram 280B. Therefore, the description of the process is not repeated here. When the second iterative hologram 280B has been calculated, the algorithm can stop. However, any number of further iterations of the algorithm can be performed. It will be understood that the third processing block 256 is only required when the fourth processing block 259 is required or further iterations are required. The output hologram 280B generally gets better with each iteration. However, in practice, a point is often reached where no measurable improvement can be observed, or the positive benefits of performing further iterations are offset by the negative effects of the additional processing time. Therefore, the algorithm is described as iterative and convergent.
[0088] Figure 2C 213A for a previous iteration. The distribution of phase values 213A for a previous iteration is fed back through a processing block of the algorithm. The distribution of amplitude values 211A is rejected in favor of an alternative distribution of amplitude values. In this alternative embodiment, the alternative distribution of amplitude values is derived from the distribution of amplitude values 211 for a previous iteration. Specifically, processing block 258 subtracts the distribution of amplitude values for input image 210 from the distribution of amplitude values 211 for a previous iteration, scales the difference by a gain factor α, and subtracts the scaled difference from input image 210. This is mathematically expressed by the following equation, where the subscript text and number represent the number of iterations:
[0089] R n+1 [x,y]=F'{exp(iψ n [u,v])}
[0090] ψ n [u,v]=∠F{η·exp(i∠R n [x,y])}
[0091] η=T[x,y]-α(|R n [x,y]|-T[x,y])
[0092] in:
[0093] F' is the inverse Fourier transform;
[0094] F is the forward Fourier transform;
[0095] R[x,y] is the complex data set output by the third processing block 256;
[0096] T[x,y] is the input or target image;
[0097] ∠ is the phase component;
[0098] Ψ is the phase-only hologram 280B;
[0099] η is the new distribution of amplitude values 211B; and
[0100] α is the gain factor.
[0101] The gain factor α may be fixed or variable. In some embodiments, the gain factor α is determined based on the size and rate of the input target picture data. In some embodiments, the gain factor α depends on the number of iterations. In some embodiments, the gain factor α is only a function of the number of iterations.
[0102] In all other respects, Figure 2C Examples and Figure 2A and Figure 2B It can be said that only the phase hologram Ψ(u,v) comprises the phase distribution in the frequency or Fourier domain.
[0103] In some embodiments, a spatial light modulator is used to perform a Fourier transform. Specifically, the hologram data is combined with second data that provides optical power. That is, the data written to the spatial light modulation includes hologram data representing the object and lens data representing the lens. When displayed on the spatial light modulator and illuminated with light, the lens data simulates a physical lens - that is, it focuses light in the same way as a corresponding physical optical element. Therefore, the lens data provides optical power or focusing power. In these embodiments, the lens data can be omitted. Figure 1 A physical Fourier transform lens 120 of FIG. 120. It is known in the art of computer generated holography how to calculate data representing a lens. The data representing a lens may be referred to as a software lens. For example, a phase-only lens may be formed by calculating the phase delay caused by each point of the lens due to its refractive index and the spatially varying optical path length. For example, the optical path length at the center of a convex lens is greater than the optical path length at the edge of the lens. An amplitude-only lens may be formed by a Fresnel zone plate. It is also known in the art of computer generated holography how to combine data representing a lens with a hologram so that a Fourier transform of the hologram may be performed without the need for a physical Fourier lens. In some embodiments, the lensing data is combined with the hologram by a simple addition, such as a simple vector addition. In some embodiments, a physical lens is used in combination with a software lens to perform a Fourier transform. Alternatively, in other embodiments, the Fourier transform lens is omitted entirely, so that holographic reconstruction occurs in the far field. In further embodiments, the hologram may be combined in the same manner with grating data - i.e., data arranged to perform a grating function such as beam steering. Again, it is known in the art of computer generated holography how to calculate such data. For example, a phase-only grating can be formed by modeling the phase delay caused by each point on the surface of a blazed grating. An amplitude-only grating can simply be superimposed with an amplitude-only hologram to provide angular steering of the holographic reconstruction.
[0104] In some embodiments, the Fourier transform is performed jointly by a physical Fourier transform lens and a software lens. That is, some of the optical power that contributes to the Fourier transform is provided by the software lens, while the remaining optical power that contributes to the Fourier transform is provided by one or more physical optical devices.
[0105] In some embodiments, a real-time engine is provided that is arranged to receive image data and calculate a hologram in real time using an algorithm. In some embodiments, the image data is a video comprising a sequence of image frames. In other embodiments, the hologram is pre-calculated, stored in a computer memory and called up as needed for display on the SLM. That is, in some embodiments, a repository of predetermined holograms is provided.
[0106] The embodiments relate to Fourier holography and Gerchberg-Saxton type algorithms by way of example only. The present disclosure is equally applicable to Fresnel holography and holograms calculated by other techniques, such as those based on point cloud methods.
[0107] Light Modulation
[0108] A spatial light modulator may be used to display a diffraction pattern including a computer generated hologram. If the hologram is a phase-only hologram, a spatial light modulator that modulates the phase is required. If the hologram is a full complex hologram, a spatial light modulator that modulates both phase and amplitude may be used, or a first spatial light modulator that modulates the phase and a second spatial light modulator that modulates the amplitude may be used.
[0109] In some embodiments, the light modulation elements (i.e., pixels) of the spatial light modulator are cells comprising liquid crystals. That is, in some embodiments, the spatial light modulator is a liquid crystal device in which the optically active component is a liquid crystal. Each liquid crystal cell is configured to selectively provide multiple light modulation levels. That is, each liquid crystal cell is configured to operate at a light modulation level selected from a plurality of possible light modulation levels at any time. Each liquid crystal cell can be dynamically reconfigured to a light modulation level different from the plurality of light modulation levels. In some embodiments, the spatial light modulator is a reflective liquid crystal on silicon (LCOS) spatial light modulator, but the present disclosure is not limited to this type of spatial light modulator.
[0110] LCOS devices provide a dense array of light modulating elements or pixels within a small aperture (e.g., a few centimeters wide). Pixels are typically about 10 microns or smaller, which results in a diffraction angle of a few degrees, meaning that the optical system can be compact. It is much easier to fully illuminate the small aperture of an LCOS SLM than the large apertures of other liquid crystal devices. LCOS devices are typically reflective, which means that the circuitry that drives the LCOS SLM pixels can be buried under the reflective surface. This results in a higher aperture ratio. In other words, the pixels are densely packed, meaning there is almost no dead space between pixels. This is advantageous because it reduces optical noise in the playback field. LCOS SLMs use a silicon backplane, which has the advantage that the pixels are optically flat. This is particularly important for phase modulation devices.
[0111] The following is just an example, Figure 3 A suitable LCOS SLM is described below. An LCOS device is formed using a single crystal silicon substrate 302. It has a 2D array of square planar aluminum electrodes 301, separated by gaps 301a, arranged on the upper surface of the substrate. Each electrode 301 can be addressed by a circuit 302a buried in the substrate 302. Each electrode forms a respective plane mirror. An orientation layer 303 is disposed on the electrode array, and a liquid crystal layer 304 is disposed on the orientation layer 303. A second orientation layer 305 is disposed on a planar transparent layer 306, for example made of glass. A single transparent electrode 307, for example made of ITO, is disposed between the transparent layer 306 and the second orientation layer 305.
[0112] Each square electrode 301 together with the covered area of the transparent electrode 307 and the intervening liquid crystal material defines a controllable phase modulation element 308, commonly referred to as a pixel. The effective pixel area or fill factor is the percentage of the total pixel that is optically active, taking into account the space between the pixels 301a. By controlling the voltage applied to each electrode 301 relative to the transparent electrode 307, the properties of the liquid crystal material of the respective phase modulation element can be varied to provide a variable retardation to light incident thereon. The effect is to provide only phase modulation to the wavefront, i.e. no amplitude effects occur.
[0113] The described LCOS SLM outputs spatially modulated light in a reflective manner. A reflective LCOS SLM has the advantage that the signal lines, grating lines and transistors are located below the mirror, which results in a high fill factor (typically greater than 90%) and high resolution. Another advantage of using a reflective LCOS spatial light modulator is that the thickness of the liquid crystal layer can be half the thickness required when using a transmissive device. This greatly increases the switching speed of the liquid crystal (a key advantage for projecting moving video images). However, the teachings of the present disclosure can also be implemented using a transmissive LCOS SLM.
[0114] Head-up display
[0115] In some embodiments, a holographic projection system is provided as part of a heads-up display (or "HUD"). Figure 4 A HUD in a vehicle such as a car is shown. Figure 4 4 shows a windshield 430 and a hood (or engine cover) 435 of a vehicle. The HUD includes a picture generation unit "PGU" 410 and an optical system 420. The PGU 410 and the optical system 420 may be collectively referred to as a holographic projector.
[0116] PGU410 includes a light source, a light receiving surface, and a processor (or computer) arranged to computer-control the image content of a picture. PGU410 is arranged to generate a picture or a series of pictures on the light receiving surface. The light receiving surface can be a screen or a diffuser. In some embodiments, the light receiving surface is plastic (i.e., made of plastic). The light receiving surface is disposed on the primary playback plane. That is, the holographic playback plane of the image is first formed.
[0117] The optical system 420 includes an input port, an output port, a first mirror 421, and a second mirror 422. The first mirror 421 and the second mirror 422 are arranged to guide light from the input port of the optical system to the output port of the optical system. More specifically, the second mirror 442 is arranged to receive light of a picture from the PGU 410, and the first mirror 421 is arranged to receive light of a picture from the second mirror 422. The first mirror 421 is also arranged to reflect the received picture light to the output port. Therefore, the optical path from the input port to the output port includes a first optical path 423 (or a first optical path component) from the input to the second mirror 422 and a second optical path 424 (or a second optical path component) from the second mirror 422 to the first mirror 421. Of course, there is a third optical path (or optical path component) from the first mirror to the output port, but in Figure 4 Due to the shape of the light path, Figure 4 The optical configuration shown may be referred to as a "z-fold" configuration.
[0118] The HUD is configured and positioned within the vehicle so that the light of the picture from the output port of the optical system 420 is incident on the windshield 430 and is at least partially reflected by the windshield 430 to the user 440 of the HUD. Thus, in some embodiments, the optical system is arranged to form a virtual image of each picture in the windshield by reflecting the spatially modulated light from the windshield. The user 440 of the HUD (e.g., the driver of the car) sees a virtual image 450 of the picture in the windshield 430. Thus, in an embodiment, the optical system is arranged to form a virtual image of each picture on the windshield of the vehicle. The virtual image 450 is formed at a distance below the hood 435 of the car. For example, the virtual image may be 1 to 2.5 meters away from the user 440. The output port of the optical system 420 is aligned with a hole in the dashboard of the car so that the light of the picture is directed to the user 440 by the optical system 420 and the windshield 430. In this configuration, the windshield 430 serves as an optical combiner. In some embodiments, the optical system is arranged to form a virtual image of each picture on an additional optical combiner included in the system. The windshield 430 or an additional optical combiner (if included) combines the light from the real world scene with the light of the picture. It can therefore be understood that the HUD can provide an augmented reality including a virtual image of the photo. For example, the augmented reality information may include navigation information or information related to the speed of the motor vehicle. In some embodiments, the light forming the picture is output by being incident on the windshield at a Brewster angle (also called a polarization angle) or within a 5-degree Brewster angle (e.g., within a 2-degree Brewster angle).
[0119] In some embodiments, the first mirror and the second mirror are arranged to fold the optical path from input to output so as to increase the optical path length without unduly increasing the physical size of the HUD.
[0120] The picture formed on the light receiving surface of the PGU410 may be only a few centimeters wide and a few centimeters high. The light receiving surface of the PGU410 can be the display plane of the alignment method. The first mirror 421 and the second mirror 422 provide magnification together or individually. That is, the first mirror and / or the second mirror can have optical power (i.e., diopter or focusing power). The user 440 therefore sees an enlarged virtual image 450 of the picture formed by the PGU. The first mirror 421 and the second mirror 422 can also correct optical distortions, such as those caused by the windshield 430, which typically has a complex curved shape. The folded optical path in the mirror and the optical power together allow the virtual image of the picture to be appropriately magnified.
[0121] The PGU 410 of the present disclosure includes a holographic projector and a light receiving surface such as a screen or a diffuser.
[0122] Distortion Correction
[0123] According to the above disclosure, the holographic projector includes a light source, a spatial light modulator and a hologram processor. The spatial light modulator is arranged to spatially modulate the light according to one or more (usually a series of) holograms presented on the spatial light modulator. The hologram processor is arranged to provide a computer-generated hologram. In some embodiments, the hologram processor calculates and outputs the computer-generated hologram in real time. In some embodiments, each picture formed by PGU410 is a holographic reconstruction on the light receiving surface. That is, in some embodiments, each picture is formed by the interference of spatially modulated light at the light receiving surface.
[0124] Each hologram presented (or displayed) on the spatial modulator may be a hologram of a target picture. A holographic reconstruction is a holographic reconstruction of a picture. Virtual image 450 is a virtual image of the holographic reconstruction. The holographically reconstructed virtual image may be distorted relative to the target picture encoded by the hologram. In other words, the image points or pixels of the holographically reconstructed virtual image may have a different spatial distribution relative to the target picture. In particular, the spacing between adjacent pixels in the holographically reconstructed virtual image may be different from the spacing between corresponding adjacent pixels in the target picture. In other words, the pixels of the holographically reconstructed virtual image may have been shifted relative to the corresponding pixels of the target picture. The distortion on the holographically reconstructed virtual image is generally non-uniform. In other words, the distortion not only causes the holographic reconstruction to be uniformly magnified relative to the target, but also actually distorts / warps the picture. There are multiple reasons for this distortion. The present disclosure is particularly concerned with compensating for distortion associated with the current temperature or temperature changes of the holographic projector. For example, the optical components of the holographic projector typically expand or contract in response to temperature changes. This may cause optical misalignment. In addition, temperature changes of the holographic projector (particularly the light source of the holographic projector) generally result in changes in the wavelength of the light emitted by the light source (in this example, the light source is a coherent light source, such as a laser). Those skilled in the art will understand that the pixel position of the holographic reconstruction will shift in response to the change in the wavelength of the light emitted by the light source.
[0125] If the amount of distortion for each pixel (at a specific temperature and wavelength of light) is known, it is possible to compensate for the distortion in the virtual image 450 by pre-distorting the target picture in a manner that is the opposite of the distortion caused by, for example, optical misalignment in the holographic projector at a specific temperature. In this way, when a hologram of the pre-distorted target picture is computed / (computer) generated and displayed, the holographic reconstruction of the target picture will look as expected. This pre-distortion is schematically shown in Figure 5 middle.
[0126] Figure 5 FIG. 5 shows a process of pre-distorting a target image 500 before calculating a hologram of the target image 500. Figure 5In a representative example, the goal is for a user of the holographic projector to receive a virtual image of a uniform array of dots 502. The array of dots 502 includes Figure 5 The solid black dots 502 in FIG. The spacing between dots 502 is uniform across the array. Figure 5 The array in the example comprises six points by six points. If a hologram of a uniform array of points 502 is calculated and displayed on a spatial light modulator, then the subsequent virtual image 450 projected by the projector will be distorted due to the above-mentioned effects. Therefore, the virtual image 450 (received by the user of the projector) will appear as a non-uniform array. The target image 500 can be pre-distorted using a predetermined distortion map. The predetermined distortion includes a mapping for each pixel of the target image. Each mapping serves as a transformation. Specifically, each mapping transforms a two-dimensional coordinate associated with one or more pixels of the target image to a new (pre-distorted) position. Each mapping compensates for the distortion experienced by the corresponding pixel of the target image in the virtual image 450. The transformation / (pre)distortion of the target image 500 is Figure 5 Specifically, each solid black dot 502 has a hollow dot 504. The position change of each dot 504 is represented by an arrow 506 from the solid black dot 502 to the corresponding hollow dot 504. The pre-distortion amount / the amount of displacement of each solid dot 502 is represented by Figure 5 The length of the arrow 506 is indicated. Figure 5 The length of the arrow 506 is not uniform across the target image 500, and thus the amount of predistortion is not uniform across the target image 500. For example, the required amount of predistortion is smallest at the center 510 of the target image, and is larger for solid points 502 closer to the edge 512 of the target image. In addition, the direction of predistortion is not uniform across the target image 500. In particular, the direction of predistortion is generally substantially parallel to a direction defined from the center 510 of the target image through the corresponding solid points 502. Thus, in Figure 5 In the example of , the solid points 502 on opposite sides of the center 510 are pre-distorted in opposite directions to each other. Once the target image 500 is pre-distorted, a hologram of the pre-distorted target image is calculated, displayed on a spatial light modulator and a projector to form a holographic reconstruction. This is shown in FIG6 .
[0127] Fig. 6A 6 shows a pre-distorted target picture 600 after applying a predetermined distortion map to the target picture 500 . Figure 6B A virtual image 602 representing a holographic reconstruction of the pre-distorted target picture 600. The virtual image 602 appears as a uniform array of solid black dots 604. As described above, this is the picture that is intended to be projected by a holographic projector. Therefore, the pre-distortion of the target picture 500 has successfully compensated for some of the distortion caused by the holographic projector.
[0128] Figure 5The target image in FIG. 5 is an array of black dots 502, which is merely a convenient example to illustrate the process of pre-distorting the target image to compensate for the distortion caused by the holographic projection / current environmental conditions. Readers in the art should be aware that pre-distortion can be applied to any target image (e.g., a target image including a non-uniform pixel distribution).
[0129] Real-time distortion map calculation
[0130] Above about Figure 5 The distortion correction described in and 6 requires a predetermined distortion map. Typically, a lot of experimental and simulation work is required to properly characterize the holographic projector to obtain a predetermined distortion map that satisfactorily pre-distorts the target picture. Even so, the predetermined distortion map will only be accurate for compensating for distortion at a specific temperature and / or light wavelength. Therefore, Figure 5 The pre-distortion of the target image 500 shown in and 6 will accurately compensate for the distortion only when the holographic projector is at a specific temperature and / or a wavelength targeted using a specific predetermined distortion. However, the holographic projector must generally be able to operate over a certain temperature range and provide high-quality (distortion-free) holographic reconstruction. For holographic projectors in vehicles, the required operating temperature range is generally relatively large, for example at least 100 degrees Celsius. Obviously, a single predetermined distortion map is not suitable for use over such a large temperature range. However, the inventors have recognized that it is impractical to characterize a large number of predetermined distortion maps (for different temperatures and / or wavelengths). This is because a) doing so would require a very large amount of validation work, and b) the memory requirements would be very high. Instead, the inventors have developed a fast and computationally efficient method to calculate the distortion map for the current temperature. This method can be performed in real time and on the fly. This means that there is no need to characterize the holographic projector at a large number of temperatures, nor is there a need to store the corresponding maps in a large memory / cumbersome lookup table. Instead, the method can take a single (valid) predetermined map as input and scale / modify the single map for the current temperature. The inventors developed this approach after exhaustive simulations and experiments showing that there is a predictable (linear) relationship between the current temperature and the amount of pre-distortion required to compensate for certain distortions.
[0131] Figure 7 A flow chart representing a method according to the present disclosure is shown.
[0132] Step 702 of the method includes receiving a two-dimensional coordinate array. In some examples, each two-dimensional coordinate represents or corresponds to one or more pixels of a target image (e.g., target image 500) without applying pre-distortion. According to the method of the present disclosure, a distortion map suitable for transforming each two-dimensional coordinate in the array is calculated to compensate for the distortion at the current temperature of the holographic projector. The transformed position of each two-dimensional coordinate is used to pre-distort the target image. In some examples, there is a one-to-one relationship between the number of two-dimensional coordinates in the array and the number of pixels in the target image. In other words, each two-dimensional coordinate can be considered to represent (or even) one pixel of the target image. In such an example, the transformed two-dimensional coordinate array can be used to directly pre-distort / shift each corresponding pixel of the target image. In some other examples, there is a one-to-many relationship between the number of two-dimensional coordinates in the array and the number of pixels in the target image. In other words, each two-dimensional coordinate can be considered to represent or correspond to more than one pixel in the target pixel. In such an example, the calculation of the distortion map may be more efficient because the distortion map includes fewer transformations / mappings (each of which needs to be calculated). However, in such an example, the transformed array of two-dimensional coordinates cannot be used to directly pre-distort / shift each corresponding pixel of the target image. Instead, an interpolation step is required to determine how the position of each transformed two-dimensional coordinate should be used to pre-distort / shift each pixel of the target picture. The skilled reader will be familiar with suitable interpolation methods.
[0133] Step 704 of the method includes receiving a single calibration distortion map M0. The single calibration distortion map includes multiple mappings. Each mapping is used to transform a two-dimensional coordinate in the received two-dimensional coordinate array. The mappings of the single calibration distortion map are mappings that have been determined and verified by previous experiments and / or simulations. In some examples, the steps of determining / verifying these mappings do not constitute part of the method according to the present disclosure. The mappings of the single calibration distortion map are mappings that have been determined by experiments and / or simulations to compensate for the distortion of the target image at a single first predetermined temperature T0.
[0134] Step 706 of the method includes receiving a vector C array. The vector C array includes a vector for each two-dimensional coordinate in the two-dimensional coordinate array. Each vector in the vector C array represents a calibration change of each corresponding two-dimensional coordinate within a predetermined temperature range. In this example, each vector has been determined by simulating how the corresponding two-dimensional coordinate needs to be transformed to calibrate / pre-distort the coordinate at the lowest temperature of the predetermined temperature range and the highest temperature of the predetermined temperature range. The vector represents the change of the two-dimensional coordinate between these two extreme temperature ranges. In an example, determining the vector array includes taking a single calibration distortion map as input and simulating how the distortion map will change with changes in temperature. In particular, it may have been simulated how the distortion map changes at the lowest temperature and the highest temperature. The inventor has found that in an example, the mapping of the distortion map is linear with temperature. In an embodiment, the simulation of the distortion map at the lowest and highest temperatures relies on this linear relationship. For clarity, in most examples, the method does not include a step of determining the vectors in the vector array (i.e., the magnitude and / or direction of the vector). Instead, the method simply includes receiving a vector array that has been individually or predetermined. However, for the sake of completeness, a description of how the vector is determined has been included above.
[0135] Step 708 of the method includes determining a current temperature T of the holographic projector. In this example, step 708 of the method includes measuring the temperature of the holographic projector (in some examples, the temperature of the light source of the holographic projector) using a temperature sensor. In other examples, step 708 of the method includes measuring or determining the wavelength of light emitted by the light source of the holographic projector, and inferring the temperature of the light source based on the measured or determined wavelength in a manner familiar to a skilled reader.
[0136] Step 710 of the method includes determining a scaling factor using the measured or determined current temperature. In one example, the step of determining the scaling factor includes determining the difference between the current temperature and a first predetermined temperature (associated with the calibration distortion map). The difference is then divided by the predetermined temperature range (associated with the vector array). Thus, the scaling factor represents the change in the current (measured or determined) temperature relative to the first predetermined temperature (associated with the calibration distortion map) as a proportion of the predetermined temperature range (associated with the vector array).
[0137] Step 712 of the method includes calculating a modification map for each coordinate in the two-dimensional coordinate array based on the current temperature by: multiplying a vector associated with the corresponding two-dimensional coordinate in the two-dimensional coordinate array by a scaling factor to output a scaling vector; and applying (e.g., adding or subtracting) the scaling vector to a corresponding mapping of the calibration map to output a modification map for transforming the corresponding two-dimensional coordinate to compensate for distortion of the target image when it is projected by the holographic projector at the current temperature.
[0138] Step 714 of the method includes outputting a modification map including the modification mapping array determined in step 712 .
[0139] Figure 7 The method shown is suitable for real-time / on-the-fly execution. In particular, Figure 7 The method shown can be performed when the holographic projector is operating. Figure 7 The method can then be used Figure 7 The step of pre-distorting the target picture according to the mapping determined in the method. A hologram of the pre-distorted target picture can then be calculated. The hologram of the pre-distorted target picture can then be displayed, and then a holographic reconstruction of the target picture is formed. The entire process can then be repeated for a second (different) target picture.
[0140] In one example, the method according to the present disclosure can be represented by the following equation:
[0141]
[0142] In the equation, M(x, y, T) is the modified distortion map. x, y represent corresponding two-dimensional coordinates in a two-dimensional coordinate array (received in step 702 of the method). T is the current temperature (determined in step 708 of the method). M0 is a calibration distortion map (received in step 704 of the method), comprising a plurality of mappings at a first predetermined temperature T0. M0 receives as input the corresponding current two-dimensional coordinates being operated on, and transforms the two-dimensional coordinates according to an appropriate mapping. C(x, y) is a vector array (received in step 706 of the method), and receives as input the corresponding current two-dimensional coordinates being operated on, and outputs a vector of the two-dimensional coordinates. The remaining terms in the equation are is the scaling factor (determined in step 710 of the equation). min and T max are respectively the lowest and highest temperatures of the predetermined temperature range involved by the vector C(x, y). The scaling factor is actually a percentage, which represents the change of the current temperature T from the first temperature T0 to the predetermined temperature range T. max -T min Each vector in the vector array C(x, y) is multiplied by the scaling factor to output a scaled vector array including a plurality of scaled vectors.
[0143] Figure 8 A vector (vector array C (x, y)) scaled based on the current temperature T and the first predetermined temperature T0 is schematically shown. Figure 8 , the vector is represented by arrow 802. Figure 8Also included are two solid black circles 804, 806. These black circles are located at corresponding two-dimensional coordinate positions at two extremes of a predetermined temperature range. Specifically, the position of black circle 804 has been simulated at Tmin, and the position of black circle 806 has been simulated at Tmax. Vector 802 represents the magnitude and direction of the change in position from Tmin to Tmax. In other words, vector 802 represents how the corresponding two-dimensional coordinate will change over the operational temperature range of the holographic projector (when properly calibrated to compensate for temperature distortion). Arrow 808 represents the change in the position of the holographic projector by multiplying the vector by Vector 802 after scaling. The length of scaling vector 808 (i.e., the magnitude of the vector) is less than the length of vector 802, but the direction is substantially the same. As described above, the scaling vector is applied to the transformed two-dimensional coordinates. After the corresponding mapping of the calibration distortion map M0 is applied, the hollow black circle 810 is located at the position of the corresponding two-dimensional coordinate. The hollow black circle 812 is located at the position of the corresponding two-dimensional coordinate after the scaling vector 808 is applied. The model relies on the fact that the vector can be linearly scaled based on the current temperature. After thorough simulations and experiments, the inventors found that a linear model is suitable for simulating this relationship and outputting good results (so that when the scaling vector is applied to the mapping of the calibration distortion map M0, the distortion correction is good).
[0144] Fig. 9 Schematically shows the process of pre-distorting the target image 900 using the modified distortion map M (calculated in real time) before calculating the hologram of the target image. Fig. 9 In, similar to Figure 5 , the purpose is to allow the user of the holographic projector to receive a virtual image of a uniform array of points 902. The array of points 902 includes Fig. 9 Solid black dots 902 in FIG. The spacing between dots 902 is uniform across the array. Fig. 9 The array in the example of comprises six points by six points. Turning to the equation above, the calculation of the modified distortion map actually comprises two terms. The first term corresponds to the calibration distortion map M0. The second term corresponds to the scaling vector array. Fig. 9 These two terms are expressed as separate distortions. In particular, Fig. 9 In FIG. 1 , each point 902 is first distorted using the calibration distortion map M0 and then adjusted using the scaling vector. As a result of the calibration distortion map M0, the pre-distortion component of each solid point 902 is represented by an arrow 904 and a hollow point 906. As a result of the scaling vector, the pre-distortion component of each hollow point 906 is represented by an arrow 908 and a dashed hollow point 910. For illustration purposes only, the two different components are shown in FIG. Fig. 9 In fact, when the modification map M is used to transform / predistort the target picture, each solid point 902 will be directly shifted to the corresponding position represented by the dotted hollow point 910.
[0145] Monochrome channel
[0146] Because of the need to ensure pixel / image feature alignment in holographic reconstructions of different colors, the method according to the present disclosure may have particular application in multi-color holographic projectors. An example of such a holographic projector is described herein.
[0147] Examples of the present disclosure relate to a holographic projector including multiple monochrome channels. Each monochrome channel includes a monochrome holographic projector that forms a monochrome holographic reconstruction (i.e., an image or picture). Multiple monochrome images are formed on a common playback plane. A full-color image can be formed using overlapping red, green, and blue images.
[0148] Fig.10 Red, green and blue channels are shown. The red channel includes a first spatial light modulator 1001r, a first lens 1020r and a first mirror 1027r. The green channel includes a second spatial light modulator 1001g, a second lens 1020g and a second mirror 1017g. The blue channel includes a third spatial light modulator 1001b, a third lens 1020b and a third mirror 1007b. Each monochrome channel forms a monochrome holographic reconstruction (or image) on a playback plane 1050. The first lens 1020r, the second lens 1020g and the third lens 1020b are optional. If each displayed hologram is a Fourier hologram, the first lens 1020r, the second lens 1020g and the third lens 1020b can contribute to the Fourier transform of each corresponding hologram.
[0149] The first spatial light modulator 1001r displays a hologram corresponding to a red image. The first spatial light modulator 1001r is illuminated with red light. The first lens 1020r receives the spatially modulated light from the first spatial light modulator 1001r and forms a red image on a playback plane 1050. The first mirror 1027r is disposed between the first lens 1020r and the playback plane 1050.
[0150] The second spatial light modulator 1001g displays a hologram corresponding to a green image. The second spatial light modulator 1001g is illuminated with green light. The second lens 1020g receives the spatially modulated light from the second spatial light modulator 1001g and forms a green image on the playback plane 1050. The second mirror 1017g is disposed between the second lens 1020g and the playback plane 1050.
[0151] The third spatial light modulator 1001b displays a hologram corresponding to a blue image. The third spatial light modulator 1001b is illuminated with blue light. The third lens 1020b receives the spatially modulated light from the third spatial light modulator 1001b and forms a blue image on the playback plane 1050. The third mirror 1007b is disposed between the third lens 1020b and the playback plane 1050.
[0152] The first mirror 1027r is a first dichroic mirror arranged to reflect red light and transmit green light and blue light. The second mirror 1017g is a second dichroic mirror arranged to reflect green light and transmit blue light. The third mirror 1007b reflects blue light.
[0153] Each monochromatic optical path comprises a first portion from the spatial light modulator to the mirror and a second portion from the mirror to the playback plane. In an embodiment, the first portions of the single channels are spatially offset but substantially parallel. In an embodiment, the second portions of the single channels are substantially collinear.
[0154] The red light path from the first spatial light modulator 1001r to the playback plane 1050 includes reflection from the first mirror 1027r. The green light path from the second spatial light modulator 1001g to the playback plane 1050 includes reflection from the second mirror 1017g, followed by transmission through the first mirror 1027r. The blue light path from the third spatial light modulator 1001b to the playback plane includes reflection from the third mirror 1007b, followed by transmission through the second mirror 1017g, and then transmission through the first mirror 1027r. The playback plane 1050, the first mirror 1027r, the second mirror 1017g and the third mirror 1007b are substantially collinear. The blue path length is greater than the green path length, and the green path length is greater than the red path length. Specifically, in an embodiment, the second portion of the blue light path is longer than the second portion of the green light path, and the second portion of the green light path is longer than the second portion of the red light path. In these embodiments, the lengths of the first portions may be substantially equal.
[0155] Each single color channel can be used to form a holographic reconstruction within the replay field area. The red replay field can contain the red picture content of the picture. The green replay field can contain the green picture content of the picture. The blue replay field can contain the blue picture content of the image. Those skilled in the art will be familiar with the idea of forming a picture by superimposing the red, green and blue picture content using the red, green and blue channels. The alignment of the red, green and blue replay fields is critical to image quality.
[0156] As described above, the holographic reconstruction may be distorted relative to the target image encoded in the corresponding hologram displayed on the spatial light modulator. This can result in the pixels / light points of the holographic reconstruction having a different spatial distribution relative to the corresponding spatial distribution in the target image. It has been described how to calculate the distortion map in real time to pre-distort the target image to compensate for this distortion. However, there is another problem in multi-color holographic projectors, namely that the distortion / shift amount of the corresponding image points / pixels in the holographic reconstructions associated with different channels is typically different. The skilled person will recognize that this is because a) the wavelengths of light in different color channels are different, and b) the misalignment of different channels (for example due to manufacturing tolerances) may be different. This can result in misalignment of the corresponding pixels of different light channels, which can adversely affect the image quality. Such misalignment, such as Fig.11 shown.
[0157] Fig.11 The first green light spot 1102G array formed by the first (green) holographic channel is shown (by Fig.11 ) and a second blue light spot array 1102B formed by a second (blue) holographic channel (represented by Fig.11 The light spots of the first holographic channel are misaligned with respect to the light spots of the second holographic channel.
[0158] As previously described, distortion can be corrected by pre-distorting the target image. However, as a skilled reader will appreciate, because the amount of distortion for different color channels is different, it is not possible to use the same distortion map for red content, green content, and blue content. In other words, the modification map M calculated in the above method may be suitable for correcting distortion in only one channel. But if the same modification map is applied to other channels, this will not result in different color points / pixels being aligned. Therefore, in a multi-color holographic projector, the above method can be repeated for each color channel to determine the modification value for each color channel. There can be a different calibration distortion map for each color channel, and there can be a different vector array for each color channel.
[0159] Additional Features
[0160] The methods and processes described herein may be embodied on a computer-readable medium. The term "computer-readable medium" includes media arranged to store data temporarily or permanently, such as random access memory (RAM), read-only memory (ROM), buffer memory, flash memory, and cache memory. The term "computer-readable medium" should also be considered to include any medium or combination of media capable of storing instructions for execution by a machine, so that when the instructions are executed by one or more processors, the machine performs, in whole or in part, any one or more of the methods described herein.
[0161] The term "computer-readable medium" also encompasses cloud-based storage systems. The term "computer-readable medium" includes, but is not limited to, one or more tangible and non-transitory data storage repositories (e.g., data volumes) in the example form of solid-state memory chips, optical disks, magnetic disks, or any suitable combination thereof. In some example embodiments, instructions for execution may be conveyed by a carrier medium. Examples of such carrier media include transient media (e.g., propagated signals conveying instructions).
[0162] It will be apparent to those skilled in the art that various modifications and variations can be made without departing from the scope of the appended claims. The present disclosure covers all modifications and variations within the scope of the appended claims and their equivalents.
Claims
1. A method for real-time calculation of a graph for distorting a target image to be projected by a holographic projector and compensating for changes in the current temperature of the holographic projector, the method comprising the following steps: receiving a calibration map including a plurality of mappings, each mapping for transforming a corresponding two-dimensional coordinate in a two-dimensional coordinate array to compensate for distortion at a predetermined temperature, each two-dimensional coordinate corresponding to one or more image points of a target image; receiving a vector array including a vector for each two-dimensional coordinate, each vector representing a calibration change for each corresponding two-dimensional coordinate within a predetermined temperature range; Receive the current temperature of the holographic projector; determining a scaling factor based on a difference between a current temperature and a predetermined temperature; For each coordinate in the 2D coordinate array, modify the map based on the current temperature by calculating: multiplying the vectors associated with corresponding coordinates in the two-dimensional coordinate array by the scaling factor to output a scaled vector; as well as Applying the scaling vector to the corresponding map of the calibration image; Output the modified graph.
2. The method of claim 1, wherein: The scaling factor has a linear dependence on temperature.
3. The method of claim 2, wherein: The step of determining the scaling factor comprises determining a difference between the current temperature and a predetermined temperature and dividing the difference by the predetermined temperature range.
4. A method as claimed in any one of the preceding claims, wherein The scaling factor is equal to (T-T0) / (Tmax-Tmin); wherein T is the current temperature, T0 is the predetermined temperature, Tmax is the highest temperature in the predetermined temperature range, and Tmin is the lowest temperature in the predetermined temperature range.
5. A method as claimed in any preceding claim, further comprising the step of receiving the two-dimensional coordinate array.
6. A method as claimed in any preceding claim, further comprising the step of applying the modification map to the two-dimensional coordinate array to output a modified two-dimensional coordinate array.
7. The method of claim 6, further comprising: receiving a target picture including a plurality of image points, wherein each two-dimensional coordinate in the two-dimensional coordinate array corresponds to one or more image points of the target picture; and The target picture is pre-distorted based on the modified two-dimensional coordinate array.
8. The method of claim 7, further comprising calculating a hologram of the pre-distorted target picture.
9. A method as claimed in any one of the preceding claims, wherein: The target image is a first target image, the two-dimensional coordinate array is a first two-dimensional coordinate array, the calibration map is a first calibration map, the vector array is a first vector array, and the modification map is a first modification map; wherein the method further comprises calculating a second map for distorting a second target image to be projected by the holographic projector by: receiving a second calibration map including a plurality of second mappings, each second mapping being used to transform a corresponding two-dimensional coordinate in a second two-dimensional coordinate array to compensate for distortion at a predetermined temperature, each two-dimensional coordinate corresponding to one or more image points of a target image; receiving a second vector array including a vector for each two-dimensional coordinate, each vector of the second array representing a calibration change for each corresponding two-dimensional coordinate over a predetermined temperature range; For each coordinate in the second two-dimensional coordinate array, a second modified map is calculated based on the current temperature by: multiplying the vector associated with the corresponding coordinate in the second two-dimensional coordinate array by the scaling factor to output a scaled vector; and applying the scaling vector to the corresponding mapping of the second calibration map; and Output the second modified map.
10. The method of claim 9, wherein: When a first wavelength is used for holographic projection of the first object picture, a mapping of the first calibration map and the vectors in the first vector array has been determined.
11. The method of claim 10, wherein: When a second wavelength is used for holographic projection of said second object picture, a mapping of said second calibration map and the vectors in said second vector array has been determined.
12. The method of any one of claims 9 to 11, further comprising applying the second modification map to the second two-dimensional coordinate array to output a second modified two-dimensional coordinate array.
13. The method of claim 12, further comprising distorting the second target picture based on the modification map.
14. The method of claim 13, further comprising calculating a second hologram of the distorted second target picture.
15. A holographic projector comprising a display device arranged to display a hologram of a picture and to spatially modulate light incident thereon in accordance with the hologram, wherein the holographic projector is arranged to form a holographic reconstruction of the picture at a playback plane; in, The holographic projector further includes a controller, which includes a memory, in which is stored: a calibration map including a plurality of mappings, each mapping for transforming a corresponding two-dimensional coordinate in the two-dimensional coordinate array to compensate for distortion at a predetermined temperature, each two-dimensional coordinate corresponding to one or more image points of a target image; as well as a vector array including a vector for each two-dimensional coordinate, each vector representing a calibration change for each corresponding two-dimensional coordinate within a predetermined temperature range; The controller is arranged as follows: Determine the current temperature of the holographic projector; determining a scaling factor based on a difference between a current temperature and a predetermined temperature; and For each coordinate in the 2D coordinate array, modify the map based on the current temperature by calculating: multiplying the vectors associated with corresponding coordinates in the two-dimensional coordinate array by the scaling factor to output a scaled vector; and Apply the scaling vector to the corresponding map of the calibration graph.
