Head-up display

By introducing infrared display channels and computer-generated hologram technology into head-up displays, the problems of infrared holographic projectors in vehicle content and aesthetics are solved, and efficient infrared light utilization and image signal-to-noise ratio improvement are achieved.

CN115309021BActive Publication Date: 2024-11-26ENVISICS LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210798002.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2018-07-18
Filing Date
2019-06-12
Publication Date
2024-11-26
Estimated Expiration
2039-06-12

AI Technical Summary

Technical Problem

In existing head-up displays, infrared holographic projectors are difficult to accommodate in vehicles and are aesthetically poor, and are difficult to effectively utilize infrared light to improve image signal-to-noise ratio.

Method used

A head-up display including infrared display channels is designed, using infrared light sources and computer-generated holograms to perform light modulation on a spatial light modulator, and guides infrared light patterns to the eye box area through an optical system to improve energy efficiency and limit infrared. The light irradiation area.

Benefits of technology

Achieves a compact design of infrared holographic projection in the vehicle, improves the signal-to-noise ratio of the driver's image, and responds to driver's movements by dynamically adjusting the infrared light pattern, maintaining uniform illumination, avoiding the waste of infrared light in unseen areas. .

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115309021B_ABST
    Figure CN115309021B_ABST
Patent Text Reader

Abstract

A head-up display having an eyebox is provided, the eyebox comprising a driver monitoring system, a picture generating unit and an optical system. The driver monitoring system is arranged to illuminate and monitor the driver. The driver monitoring system comprises a first display channel. The picture generating unit is arranged to display a picture on a playback plane. The picture generating unit comprises a second display channel. Each display channel comprises a light source, a spatial light modulator and a controller. Each light source is arranged to emit light. Each spatial light modulator is arranged to receive light from a corresponding light source and output spatially modulated light according to a computer-generated hologram displayed on the spatial light modulator to form a corresponding light pattern on the playback plane. Each controller is arranged to output a computer-generated hologram to the spatial light modulator. The optical system is arranged to relay each light pattern from the playback plane. The optical system comprises an input, an output and at least one mirror. The input is arranged to receive light of each light pattern. The output is arranged to output light of each light pattern. At least one mirror is arranged to guide light from the input to the output along an optical path. The light pattern formed by the first display channel is an infrared light pattern. Each light pattern is a holographic reconstruction.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This application is a divisional of the Chinese patent application number 201980034047.0 filed on June 12, 2019, entitled “Head-up Display”. Technical Field

[0002] The present disclosure relates to a holographic projector. More specifically, the present disclosure relates to a head-up display including at least one holographic projection channel. Some embodiments relate to a driver monitoring system for a head-up display. More specifically, some embodiments relate to an infrared holographic projector for a driver monitoring system for a head-up display. Some embodiments relate to an augmented reality head-up display "AR-HUD" that includes eye tracking or head tracking. Other embodiments relate to a picture generation unit for head tracking or eye tracking and an integrated infrared holographic illuminator. Background Art

[0003] Light scattered from an object contains amplitude and phase information. This amplitude and phase information can be captured, for example, on a photographic plate by well-known interference techniques to form a holographic recording or "hologram" comprising interference fringes. The hologram can be reconstructed by illumination with appropriate light to form a two-dimensional or three-dimensional holographic reconstruction or playback image representing the original object.

[0004] Computer generated holography can numerically simulate interference processes. Computer generated holograms can be calculated by techniques based on mathematical transformations such as Fresnel or Fourier transforms. These types of holograms may be referred to as Fresnel / Fourier transform holograms or simply Fresnel / Fourier holograms. Fourier holograms may be considered as Fourier domain / plane representations of objects or frequency domain / plane representations of objects. Computer generated holograms may also be calculated, for example, by coherent ray tracing or point cloud techniques.

[0005] The computer generated hologram may be encoded on a spatial light modulator arranged to modulate the amplitude and / or phase of the incident light. For example, the light modulation may be achieved using electrically addressable liquid crystals, optically addressable liquid crystals or micromirrors.

[0006] A spatial light modulator typically includes a plurality of individually addressable pixels, which may also be referred to as cells or elements. The light modulation scheme may be binary, multi-level, or continuous. Alternatively, the device may be continuous (i.e., not including pixels), so that the light modulation may be continuous across the device. A spatial light modulator may be reflective, meaning that the light is modulated to reflect the output. A spatial light modulator may also be transmissive, meaning that the light is modulated to transmit the output.

[0007] The systems described herein can be used to provide holographic projectors. For example, such projectors have been applied to head-up displays "HUDs" and head-mounted displays "HMDs", including near-eye devices.

[0008] AR-HUDs typically include a driver monitoring system with eye tracking. Such systems require an infrared "IR" emitter to illuminate the driver with infrared light. However, IR emitters are difficult to accommodate within a vehicle and are aesthetically unpleasant. A head-up display is disclosed herein that includes an integrated IR light source for driver monitoring. Summary of the invention

[0009] Aspects of the present disclosure are defined in the accompanying independent claims.

[0010] A head-up display with an eye box is provided. The head-up display includes a driver monitoring system, a picture generation unit, and an optical system. The driver monitoring system is arranged to illuminate and monitor the driver. The driver monitoring system includes an infrared display channel. The infrared display channel includes an infrared light source and a hologram. The infrared light source is arranged to emit infrared light. The hologram is arranged to receive infrared light from the infrared light source and output spatially modulated light according to the hologram to form an infrared light pattern on a playback plane. The picture generation unit is arranged to display a picture on the playback plane. The picture generation unit includes a visible light display channel. The visible light display channel includes a visible light source, a spatial light modulator, and a controller. The visible light source is arranged to emit visible light. The spatial light modulator is arranged to receive visible light from the visible light source and output spatially modulated light according to a computer-generated hologram displayed on the spatial light modulator to form a visible light pattern on the playback plane. The controller is arranged to output a light modulation distribution including a computer-generated hologram to the spatial light modulator. The optical system includes an input, an output, and at least one mirror. The input is arranged to receive an infrared light pattern and a visible light pattern. The output is arranged to output light in an infrared light pattern and light in a visible light pattern. At least one mirror is arranged to direct light from the input to the output along an optical path.

[0011] The light of the infrared light pattern and the light of the visible light pattern output by the optical system may diverge. The light can be received by the optical combiner and directed to the eye box. The virtual image of the visible light pattern can be seen from the eye box area or volume. In normal operation, the viewer (e.g., the driver) is positioned in the eye within their eye box area. The eye box allows the viewer to make some head movements while ensuring that the virtual image is still visible. The optical combiner can be the windshield of the vehicle that houses the head-up display. It can be said that the optical system of the head-up display and the optical combiner of the vehicle jointly form a virtual image of the visible light pattern. The optical system and the optical combiner also jointly guide the infrared light to the eye box. The eye box area or volume is inside the vehicle, and the virtual image appears at a certain distance outside the vehicle. The distance to the virtual image is mainly determined by the optical power of the optical system and / or the optical combiner. Therefore, a head-up display system is provided, which includes a head-up display and an optical combiner, which is arranged to receive the light output by the optical system and form a virtual image of each light pattern. The viewer of course does not see the virtual image of the infrared light pattern, but the infrared light reflected by the optical combiner to the eye box illuminates the viewer.

[0012] The light modulation distribution includes [m×n] arrays of light modulation values. The light modulation distribution may include multiple components, each of which includes [m×n] arrays of light modulation values. The computer-generated hologram may be regarded as the first component of the light modulation distribution. Additional components may be combined with the computer-generated hologram by addition to form the light modulation distribution. That is, corresponding elements of one or more additional [m×n] arrays of light modulation values ​​may be added to form the light modulation distribution. The second component of the light modulation distribution may correspond to a lens function that at least partially determines the distance from the spatial light modulator to the focal plane (i.e., the holographic playback plane) of the light pattern (e.g., the z position in Cartesian coordinates, where the z direction is the direction of the optical axis). The lens function may be a Fresnel function that includes [m×n] arrays of phase modulation values. The third component of the light modulation distribution may correspond to a grating function or a beam steering function that controls the position (e.g., xy position) of the light pattern on the focal plane. The grating function may also include [m×n] arrays of phase modulation values. The grating function may be a phase ramp, such as a modulo 2π phase ramp. That is, the array of phase modulation values ​​of the grating function can increase linearly across the array in one direction (eg, the x-direction) or in two orthogonal directions (eg, the x-direction and the y-direction).

[0013] A computer generated hologram is calculated based on an input image or pattern. The purpose of holographic reconstruction is to faithfully replicate the input image. In some embodiments described herein, the computer generated hologram is a phase hologram. In some embodiments described herein, an iterative phase retrieval algorithm based on the Gerchberg-Saxton algorithm is used to calculate the phase hologram. In some embodiments, the computer generated hologram is calculated in real time. In other embodiments, the computer generated hologram is retrieved from a repository or memory including multiple computer generated holograms. The computer generated hologram is calculated based on an input image. A computer generated hologram is a hologram of an image, wherein the hologram is generated by a computer implemented algorithm. The image is the input to the hologram calculation engine.

[0014] The vehicle's optical system and optical combiner (e.g., a windshield or pop-up dashboard combiner) relay light of the light pattern formed at the playback plane. Optionally, the optical system and / or optical combiner provide amplification of the light pattern. Specifically, the optical system and optical combiner can form an amplified virtual image of the light pattern. However, the relay may not be perfect. That is, the virtual image observed from the eye box may not be a perfect replica of the light pattern. For example, the virtual image may be distorted or warped by the optical system and / or optical combiner. In some embodiments, the input image is processed before a computer-generated hologram is calculated to compensate for the optical system and / or optical combiner. This processing is referred to herein as "pre-processing" to indicate that it occurs before the hologram is calculated. Pre-processing of the image can include changing (e.g., anti-distorting or anti-warping) the image to compensate for the optical response of the optical system and / or optical combiner.

[0015] A head-up display is disclosed herein, which includes multiple single-wavelength (or monochrome) display (or light) channels, each including a respective light source and display component (e.g., a hologram or spatial light modulator). Each display channel forms a respective light pattern. The infrared light pattern and the visible light pattern are formed on the same playback plane. The infrared light pattern and the visible light pattern can substantially overlap or overlap. The visible light pattern can convey useful information (e.g., navigation information) to a viewer (e.g., a driver). The visible light pattern is a picture or a monochrome component of a picture. An infrared light pattern is provided to illuminate the viewer. The picture can only be viewed within an eye box area primarily defined by an optical system. The infrared light and visible light are relayed by the same optical system and optical combiner to an eye box area or volume for the viewer. The inventors recognize that this conveniently limits the infrared light to the eye box area. This is advantageous because it means that infrared light energy is not wasted in an illuminated area where the picture cannot be seen.

[0016] The head-up display is arranged to illuminate the eye box with infrared light. The infrared light is used to increase the signal-to-noise ratio of the image of the viewer (e.g., driver) captured by the image capture device. According to the present disclosure, all infrared light from the infrared display channel can be directed to the eye box. This provides energy efficiency.

[0017] The infrared light pattern formed by the infrared display channel can be a structured light pattern. A structured light pattern is a light pattern with a controlled shape. The structured light pattern can include light and dark areas arranged to provide selective infrared illumination of the eye box. The infrared light pattern can be constructed or shaped in any possible way. For example, it may be advantageous to direct infrared light away from certain areas of the eye box and / or to focus infrared light to other areas of the eye box.

[0018] The infrared intensity of the light region in the eyebox can be substantially uniform. This can be achieved by forming a substantially non-uniform intensity infrared light pattern on the playback plane. The non-uniformity of the infrared intensity of the light region on the playback plane can compensate for spatially varying optical losses in the optical system and / or optical combiner. In embodiments where the hologram of the infrared channel is a computer-generated hologram displayed on a spatial light modulator, the hologram can dynamically change in response to the movement of the viewer to maintain uniform illumination of the light region.

[0019] The infrared display channel can be at least partially integrated with the visible light channel to provide a compact design. For example, at least a portion of the infrared display channel can be substantially colinear with a corresponding portion of the visible light display channel.

[0020] The driver monitoring system may include an image capture device (e.g., a camera, such as an infrared camera) arranged to capture an image (i.e., an infrared image) of a viewer (e.g., a driver). The image capture device may be arranged or positioned behind a mirror in at least one mirror of the optical system. For example, one of the mirrors of the optical system may be transparent to infrared light reflected from the viewer and transmitted back to the optical system. The mirror may reflect as infrared light on a first pass from the playback plane to the viewer and transmit on a second pass from the viewer to the playback plane. This may be achieved in a variety of ways using, for example, polarization and polarization rotation between the first pass and the second pass.

[0021] The hologram may be a computer generated hologram. The infrared display channel may further include a spatial light modulator and a controller. The spatial light modulator is arranged to display the computer generated hologram. The controller is arranged to output a light modulation distribution including the computer generated hologram to the spatial light modulator. Thus, the infrared light pattern is dynamically reconfigurable. For example, the infrared light pattern may be reconfigured according to movement of a viewer within the eye box or changes in ambient light conditions.

[0022] Therefore, a head-up display having an eyebox is provided, the eyebox comprising a driver monitoring system, a picture generating unit and an optical system. The driver monitoring system is arranged to illuminate and monitor the driver. The driver monitoring system comprises a first display channel. The picture generating unit is arranged to display a picture on a playback plane. The picture generating unit comprises a second display channel. Each display channel comprises a light source, a spatial light modulator and a controller. Each light source is arranged to emit light. Each spatial light modulator is arranged to receive light from a corresponding light source and output spatially modulated light according to a computer-generated hologram displayed on the spatial light modulator to form a corresponding light pattern on the playback plane. Each controller is arranged to output a computer-generated hologram to the spatial light modulator. The optical system is arranged to relay each light pattern from the playback plane. The optical system comprises an input, an output and at least one mirror. The input is arranged to receive light of each light pattern. The output is arranged to output light of each light pattern. At least one mirror is arranged to guide light from the input to the output along an optical path. The light pattern formed by the first display channel is an infrared light pattern. Each light pattern is a holographic reconstruction.

[0023] The driver monitoring system may further include an image processor. The image processor is arranged to receive a captured image of a viewer from an image capture device. The image processor is further arranged to provide input to a controller. The controller uses the input to determine a light modulation distribution displayed on a corresponding spatial light modulator. Thus, a feedback system is provided in which the image of the viewer is analyzed and the projected light pattern is changed in response to data extracted from the image of the viewer. For example, the light pattern may be changed in response to the viewer's head or eye movement. The feedback system may be used with any number of display channels, including visible and infrared display channels.

[0024] In some embodiments, correction data (e.g., correction map or correction matrix) is provided for each display channel. That is, infrared correction data is provided for the infrared display channel and visible light correction data is provided for the visible light channel. The correction data compensates for the optical response of the optical system and / or optical combiner. The correction data is data used to pre-process the image. Correction data is provided for each display channel and for each of a plurality of eye positions within the eye box. The most recent correction data may be provided for the eye position detected by the driver monitoring system.

[0025] Therefore, the controller of the infrared display channel is further arranged to receive a first input and a second input. The controller is further arranged to determine a light modulation distribution based on the first input and the second input. The first input comprises an input infrared image for projection using infrared light. The second input comprises infrared correction data.

[0026] The controller of the visible light display channel is also arranged to receive a first input and a second input. The controller is also arranged to determine a light modulation distribution based on the first input and the second input. The first input includes an input visible light image for projection using visible light. The second input includes visible light correction data.

[0027] The first input may be an image for projection. The second input may be a correction map or correction matrix comprising an array of correction values. The controller may be arranged to multiply the image by the correction matrix to form a pre-processed image for hologram calculation. The correction matrix may be retrieved from a memory based on the measured eye position. The correction matrix for each eye position and each wavelength may be determined by an in-situ calibration process or by computer simulation. Each correction matrix may provide translation, rotation and / or magnification. Additionally or alternatively, each correction matrix may change the brightness values ​​of image pixels. In some embodiments, each correction matrix is ​​a brightness gain matrix arranged to compensate for system-wide light losses.

[0028] The hologram of the infrared display channel can be a fixed holographic record in a recording medium. That is, the hologram is not essentially a computer-generated hologram displayed on a spatial light modulator. The holographic reconstruction can be a fixed structured light pattern. The fixed structured light pattern can include regularly spaced horizontal and vertical grid lines.

[0029] The light source for each display channel may be a laser diode configured to emit polarized light. The polarization of the light from the laser diode may be oriented so that reflection from the optical combiner is efficient. For example, the reflection angle from the optical combiner may be approximately equal to the Brewster angle.

[0030] The term "hologram" is used to refer to a record containing amplitude information or phase information or some combination thereof about an object. The term "holographic reconstruction" is used to refer to the optical reconstruction of an object formed by illuminating a hologram. The system disclosed herein is described as a "holographic projector" because the holographic reconstruction is a real image and is spatially separated from the hologram. The term "playback field" is used to refer to the 2D area within which the holographic reconstruction is formed and is fully focused. If the hologram is displayed on a spatial light modulator comprising pixels, the playback field will be repeated in the form of multiple diffraction orders, where each diffraction order is a copy of the zero-order playback field. The zero-order playback field generally corresponds to the preferred or primary playback field because it is the brightest playback field. Unless otherwise explicitly stated, the term "playback field" should be considered to refer to the zero-order playback field. The term "playback plane" is used to refer to a plane in space that contains all playback fields. The terms "playback image" and "image area" refer to the area of ​​the playback field illuminated by the light of the holographic reconstruction. In some embodiments, a "playback image" may include discrete spots of light, which may be referred to as "image spots" or, for convenience only, as "image pixels."

[0031] The terms "encoding", "writing" or "addressing" are used to describe the process of providing a plurality of pixels of the SLM with a corresponding plurality of control values ​​that respectively determine the modulation level of each pixel. It can be said that the pixels of the SLM are configured to "display" a light modulation distribution in response to receiving the plurality of control values. Thus, the SLM can be said to "display" a hologram, and a hologram can be considered to be an array of light modulation values ​​or levels.

[0032] It has been found that a holographic reconstruction of acceptable quality can be formed from a "hologram" that contains only phase information related to the Fourier transform of the original object. Such a holographic recording may be referred to as a pure phase hologram. The embodiments relate to pure phase holograms, but the disclosure is equally applicable to pure amplitude holograms.

[0033] The present disclosure is also equally applicable to forming a holographic reconstruction using amplitude and phase information related to the Fourier transform of the original object. In some embodiments, this is achieved by using complex modulation of a so-called fully complex hologram that contains amplitude and phase information related to the original object. Because the value (gray level) assigned to each pixel of the hologram has an amplitude and a phase component, such a hologram may be referred to as a fully complex hologram. The value (gray level) assigned to each pixel can be represented as a complex number with an amplitude and a phase component. In some embodiments, a fully complex computer-generated hologram is calculated.

[0034] Reference may be made to the phase value, phase component, phase information, or simply phase, of a pixel of a computer-generated hologram or spatial light modulator as shorthand for "phase delay". That is, any phase value described is actually a number (e.g., in the range of 0 to 2π) representing the amount of phase delay provided by that pixel. For example, a pixel of a spatial light modulator described as having a phase value of π / 2 will delay the phase of the received light by π / 2 radians. In some embodiments, each pixel of the spatial light modulator may be operated at one of a plurality of possible modulation values ​​(e.g., phase delay values). The term "grayscale" may be used to refer to a plurality of available modulation levels. For example, the term "grayscale" may be used for convenience to refer to a plurality of available phase levels in a pure phase modulator, even though the different phase levels do not provide different shades of gray. For convenience, the term "grayscale" may also be used to refer to a plurality of available complex modulation levels in a complex modulator.

[0035] Thus, a hologram comprises an array of gray levels, i.e. an array of light modulation values, such as phase delay values ​​or an array of complex modulation values. A hologram is also considered to be a diffraction pattern, since it is a pattern that causes diffraction when displayed on a spatial light modulator and illuminated with light of a wavelength relative to (usually less than) the pixel pitch of the spatial light modulator. Reference is made herein to combining a hologram with other diffraction patterns, such as a diffraction pattern used as a lens or a grating. For example, a diffraction pattern used as a grating may be combined with a hologram to translate the playback field on the playback plane, or a diffraction pattern used as a lens may be combined with a hologram to focus the holographic reconstruction on the playback plane in the near field.

[0036] Throughout this disclosure, reference is made to an "eyebox" of a head-up display. The eyebox is a viewing window for the head-up display image content. The viewer can only view the head-up display image content through this window. The viewer is positioned to view through the eyebox in a real-world scene. The head-up display image content provides augmented reality information to the viewer.

[0037] Reference is also made to a monochromatic or single wavelength display channel. However, as will be appreciated by those skilled in the art, the light sources described may not be perfect, and each display channel may actually include light within a narrow band of wavelengths (e.g., + / - 20 nm of the center wavelength). The single wavelength referred to here is the center wavelength of the narrow band of wavelengths.

[0038] Although different embodiments and embodiment groups may be disclosed separately in the following detailed description, any feature of any embodiment or embodiment group may be combined with any other feature or feature combination of any embodiment or embodiment group. That is, all possible combinations and permutations of the features disclosed in this disclosure are contemplated. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Specific embodiments are described by way of example only with reference to the following drawings:

[0040] Figure 1 is a schematic diagram showing a reflective SLM producing a holographic reconstruction on a screen;

[0041] Figure 2A The first iteration of an example Gerchberg-Saxton type algorithm is shown;

[0042] Figure 2B The second and subsequent iterations of an example Gerchberg-Saxton type algorithm are shown;

[0043] Figure 2C An alternative second and subsequent iteration of an example Gerchberg-Saxton type algorithm is shown;

[0044] Figure 3 is a schematic diagram of a reflective LCOS SLM;

[0045] Figure 4 A heads-up display is shown;

[0046] Figure 5 shows a plurality of display channels according to an embodiment;

[0047] Figure 6 shows components of a picture generation unit and a driver monitoring system according to an embodiment; and

[0048] Fig. 7A and 7B Another example heads-up display including an integrated IR emitter is shown.

[0049] The same reference numbers will be used throughout the drawings to refer to the same or like parts. DETAILED DESCRIPTION

[0050] The present invention is not limited to the embodiments described below, but extends to the full scope of the appended claims. That is, the present invention can be implemented in different forms and should not be construed as limited to the described embodiments, which are set forth for illustrative purposes.

[0051] A structure described as being formed on / under or above / below another structure should be construed to include the case where the structures are in contact with each other and the case where a third structure is provided therebetween.

[0052] When describing a temporal relationship, for example, when the temporal order of events is described as "after," "subsequently," "next," "before," etc., the present disclosure should be considered to include both sequential and non-sequential events unless otherwise specified. For example, the description should be considered to include non-sequential situations unless words such as "only," "immediately," or "directly" are used.

[0053] Although the terms "first", "second", etc. may be used herein to describe various elements, these elements are not limited by these terms. These terms are only used to distinguish between various elements. For example, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element without departing from the scope of the appended claims.

[0054] The features of different embodiments may be coupled or combined with each other in part or in whole, and may interoperate with each other in different ways. Some embodiments may be performed independently of each other, or may be performed together in an interdependent relationship.

[0055] Optical Configuration

[0056] Figure 1An embodiment is shown in which a computer generated hologram is encoded on a single spatial light modulator. The computer generated hologram is a Fourier transform of the object used for reconstruction. It can therefore be said that the hologram is a Fourier domain or frequency domain or spectral domain representation of the object. In this embodiment, the spatial light modulator is a reflective liquid crystal on silicon "LCOS" device. The hologram is encoded on the spatial light modulator and a holographic reconstruction is formed at a playback field, such as a light receiving surface such as a screen or diffuser.

[0057] A light source 110, such as a laser or laser diode, is arranged to illuminate the SLM 140 via a collimating lens 111. The collimating lens causes a substantially planar wavefront of light to be incident on the SLM. Figure 1 In some embodiments, the direction of the wavefront is off-normal (e.g., two or three degrees from being truly perpendicular to the plane of the transparent layer). However, in other embodiments, a substantially planar wavefront is provided at normal incidence, and a beam splitter arrangement is used to separate the input and output optical paths. Figure 1 In the embodiment shown, the arrangement is such that light from the light source reflects from the mirrored rear surface of the SLM and interacts with the light modulating layer to form an exit wavefront 112. The exit wavefront 112 is applied to an optical device including a Fourier transform lens 120, the focus of which is located on a screen 125. More specifically, the Fourier transform lens 120 receives the modulated light beam from the SLM 140 and performs a frequency-space transform to produce a holographic reconstruction on the screen 125.

[0058] It is important to note that in this type of hologram, every pixel of the hologram contributes to the entire reconstruction. There is no one-to-one correlation between a specific point on the playback field (or image pixel) and a specific light modulation element (or hologram pixel). In other words, the modulated light leaving the light modulation layer is distributed over the entire playback field.

[0059] In these embodiments, the position of the holographic reconstruction in space is determined by the power (focus) of the Fourier transform lens. Figure 1 In the illustrated embodiment, the Fourier transform lens is a physical lens. That is, the Fourier transform lens is an optical Fourier transform lens, and the Fourier transform is performed optically. Any lens can act as a Fourier transform lens, but the performance of the lens will limit the accuracy of the Fourier transform it performs. The skilled person understands how to use a lens to perform an optical Fourier transform.

[0060] Holographic computing

[0061] In some embodiments, the computer generated hologram is a Fourier transform hologram, or simply a Fourier hologram or a Fourier based hologram, where the image is reconstructed in the far field by utilizing the Fourier transform properties of a positive lens. The Fourier hologram is calculated by Fourier transforming the desired light field in the playback plane back to the lens plane. The computer generated Fourier hologram can be calculated using the Fourier transform.

[0062] Algorithms such as the Gerchberg-Saxton algorithm can be used to calculate the Fourier transform hologram. In addition, the Gerchberg-Saxton algorithm can be used to calculate a hologram in the Fourier domain (i.e., a Fourier transform hologram) from pure amplitude information in the spatial domain (such as a photograph). Phase information related to the object is effectively "retrieved" from the pure amplitude information in the spatial domain. In some embodiments, a computer-generated hologram is calculated from pure amplitude information using the Gerchberg-Saxton algorithm or a variation thereof.

[0063] The Gerchberg-Saxton algorithm considers that when the intensity cross-section I of the beam in planes A and B is known, A (x,y) and I B (x,y) and I A (x,y) and I B (x,y) is related by a single Fourier transform. For a given intensity cross section, the phase distribution in planes A and B is approximated as Ψ A (x,y) and Ψ B (x, y). The Gerchberg-Saxton algorithm finds a solution to this problem by following an iterative process. More specifically, the Gerchberg-Saxton algorithm iteratively applies spatial and spectral constraints while repeatedly transferring representations of I between the spatial domain and the Fourier (spectral or frequency) domain. A (x,y) and I B A data set (amplitude and phase) of (x,y). A corresponding computer generated hologram in the spectral domain is obtained by at least one iteration of the algorithm. The algorithm is convergent and arranged to produce a hologram representing the input image. The hologram can be a pure amplitude hologram, a pure phase hologram or a full complex hologram.

[0064] In some embodiments, a phase-only hologram is calculated using an algorithm based on the Gerchberg-Saxton algorithm, such as described in UK Patents 2498170 or 2501112, the entire contents of which are incorporated herein by reference. However, the embodiments disclosed herein describe the calculation of pure phase holograms by way of example only. In these embodiments, the Gerchberg-Saxton algorithm retrieves the phase information Ψ[u,v] of the Fourier transform of the data set, which produces known amplitude information T[x,y], where the amplitude information T[x,y] represents the target image (e.g., a photograph). Since amplitude and phase are inherently combined in the Fourier transform, the transformed amplitude and phase contain useful information about the accuracy of the calculated data set. Therefore, the algorithm can be used iteratively with feedback of the amplitude and phase information. However, in these embodiments, the pure phase information Ψ[u,v] is used as a hologram to form a holographic representation of the target image on the image plane. The hologram is a data set (e.g., a 2D array) of phase values.

[0065] In other embodiments, an algorithm based on the Gerchberg-Saxton algorithm is used to calculate a full complex hologram. A full complex hologram is a hologram having an amplitude component and a phase component. A hologram is a data set (e.g., a 2D array) comprising an array of complex data values, where each complex data value comprises an amplitude component and a phase component.

[0066] In some embodiments, the algorithm processes complex data and the Fourier transform is a complex Fourier transform. The complex data can be considered to include (i) a real component and an imaginary component, or (ii) an amplitude component and a phase component. In some embodiments, the two components of the complex data are processed differently at various stages of the algorithm.

[0067] Figure 2A The first iteration of an algorithm for calculating a pure phase hologram according to some embodiments is shown. The input to the algorithm is an input image 210 comprising a 2D array of pixels or data values, each of which is an amplitude or amplitude value. That is, each pixel or data value of the input image 210 does not have a phase component. Therefore, the input image 210 can be regarded as an amplitude-only or pure amplitude or intensity-only distribution. An example of such an input image 210 is a frame of a photograph or a video comprising a time sequence of frames. The first iteration of the algorithm begins with a data formation step 202A, which includes assigning a random phase value to each pixel of the input image using a random phase distribution (or random phase seed) 230 to form a starting complex data set, wherein each data element of the data set includes an amplitude and a phase. It can be said that the starting complex data set represents the input image in the spatial domain.

[0068] The first processing block 250 receives the starting complex data set and performs a complex Fourier transform to form a Fourier transformed complex data set. The second processing block 253 receives the Fourier transformed complex data set and outputs a hologram 280A. In some embodiments, the hologram 280A is a pure phase hologram. In these embodiments, the second processing block 253 quantizes each phase value and sets each amplitude value to 1 in order to form the hologram 280A. Each phase value is quantized according to the phase level that can be represented on the pixel of the spatial light modulator, which will be used to "display" the pure phase hologram. For example, if each pixel of the spatial light modulator provides 256 different phase levels, each phase value of the hologram is quantized to one of the 256 possible phase levels. The hologram 280A is a pure phase Fourier hologram representing the input image. In other embodiments, the hologram 280A is a fully complex hologram comprising an array of complex data values ​​(each comprising an amplitude component and a phase component) derived from a received Fourier transformed complex data set. In some embodiments, the second processing block 253 constrains each complex data value to one of a plurality of allowable complex modulation levels to form the hologram 280A. The constraining step may include setting each complex data value to the closest allowable complex modulation level in the complex plane. It can be said that the hologram 280A represents the input image in the spectral or Fourier or frequency domain. In some embodiments, the algorithm stops at this point.

[0069] However, in other embodiments, the algorithm continues as Figure 2A In other words, following Figure 2A The steps indicated by dashed arrows are optional (ie, not essential to all embodiments).

[0070] The third processing block 256 receives the modified complex data set from the second processing block 253 and performs an inverse Fourier transform to form an inverse Fourier transformed complex data set. The inverse Fourier transformed complex data set can be said to represent the input image in the spatial domain.

[0071] The fourth processing block 259 receives the inverse Fourier transformed complex data set and extracts the distribution of amplitude values ​​211A and the distribution of phase values ​​213A. Optionally, the fourth processing block 259 evaluates the distribution of amplitude values ​​211A. Specifically, the fourth processing block 259 can compare the distribution of amplitude values ​​211A of the inverse Fourier transformed complex data set with the input image 510, which itself is of course a distribution of amplitude values. If the difference between the distribution of amplitude values ​​211A and the input image 210 is small enough, the fourth processing block 259 can determine that the hologram 280A is acceptable. That is, if the difference between the distribution of amplitude values ​​211A and the input image 210 is small enough, the fourth processing block 259 can determine that the hologram 280A is a sufficiently accurate representation of the input image 210. In some embodiments, for comparison purposes, the distribution of phase values ​​213A of the inverse Fourier transformed complex data set is ignored. It will be appreciated that any number of different methods may be employed to compare the distribution of amplitude values ​​211A and the input image 210, and the present disclosure is not limited to any particular method. In some embodiments, a mean square error is calculated, and if the mean square error is less than a threshold, the hologram 280A is deemed acceptable. If the fourth processing block 259 determines that the hologram 280A is unacceptable, further iterations of the algorithm may be performed. However, this comparison step is not required, and in other embodiments, the number of iterations of the algorithm performed is predetermined or preset or user defined.

[0072] Figure 2B 202B. The second iteration of the algorithm and any further iterations of the algorithm are represented. The distribution of phase values ​​213A of the previous iteration is fed back through the processing blocks of the algorithm. The distribution of amplitude values ​​211A is rejected in favor of the distribution of amplitude values ​​of the input image 210. In the first iteration, the data forming step 202A forms a first complex data set by combining the distribution of amplitude values ​​of the input image 210 with the random phase distribution 230. However, in the second and subsequent iterations, the data forming step 202B includes forming a complex data set by combining (i) the distribution of phase values ​​213A from the previous iteration of the algorithm with (ii) the distribution of amplitude values ​​of the input image 210.

[0073] Then, refer to Figure 2A The same method as described is handled by Figure 2BThe complex data set formed in the data forming step 202B is used to form a second iterative hologram 280B. Therefore, the description of the process will not be repeated here. When the second iterative hologram 280B has been calculated, the algorithm can stop. However, any number of further iterations of the algorithm can be performed. It will be understood that the third processing block 256 is only required when the fourth processing block 259 is required or further iterations are required. The output hologram 280B generally gets better with each iteration. However, in practice, a point is often reached where no measurable improvement can be observed, or the positive benefits of performing further iterations are offset by the negative effects of the additional processing time. Therefore, the algorithm is described as iterative and convergent.

[0074] Figure 2C 213A of the previous iteration is fed back through a processing block of the algorithm. The distribution of amplitude values ​​211A is rejected in favor of an alternative distribution of amplitude values. In this alternative embodiment, the alternative distribution of amplitude values ​​is derived from the distribution of amplitude values ​​211 of the previous iteration. Specifically, processing block 258 subtracts the distribution of amplitude values ​​of input image 210 from the distribution of amplitude values ​​211 of the previous iteration, scales the difference by a gain factor α, and subtracts the scaled difference from input image 210. This is expressed mathematically by the following equation, where the subscript text and number represent the number of iterations:

[0075] R n+1 [x,y]=F'{exp(iψ n [u,v])}

[0076] ψ n [u,v]=∠F{η·exp(i∠R n [x,y])}

[0077] η=T[x,y]-α(|R n [x,y]|-T[x,y])

[0078] in:

[0079] F' is the inverse Fourier transform;

[0080] F is the forward Fourier transform;

[0081] R[x,y] is the complex data set output by the third processing block 256;

[0082] T[x,y] is the input or target image;

[0083] ∠ is the phase component;

[0084] Ψ is the pure phase hologram 280B;

[0085] η is the new distribution of amplitude values ​​211B; and

[0086] α is the gain factor.

[0087] The gain factor α may be fixed or variable. In some embodiments, the gain factor α is determined based on the size and rate of the input target image data. In some embodiments, the gain factor α depends on the number of iterations. In some embodiments, the gain factor α is only a function of the number of iterations.

[0088] In all other respects, Figure 2C Examples and Figure 2A and Figure 2B It can be said that the pure phase hologram Ψ(u,v) comprises a phase distribution in the frequency domain or the Fourier domain.

[0089] In some embodiments, a Fourier transform is performed using a spatial light modulator. Specifically, the hologram data is combined with second data that provides optical power. That is, the data written to the spatial light modulation includes hologram data representing the object and lens data representing the lens. When displayed on the spatial light modulator and illuminated with light, the lens data simulates a physical lens, that is, it focuses light in the same way as a corresponding physical optical element. Therefore, the lens data provides light or focusing power. In these embodiments, the hologram data can be omitted. Figure 1 A physical Fourier transform lens 120 of FIG. 1 is known in the art of computer generated holography how to calculate data representing a lens. The data representation of a lens may be referred to as a software lens. For example, a pure phase lens may be formed by calculating the phase delay caused by each point of the lens due to its refractive index and the spatially varying optical path length. For example, the optical path length at the center of a convex lens is greater than the optical path length at the edge of the lens. A pure amplitude lens may be formed by a Fresnel zone plate. It is also known in the art of computer generated holography how to combine data representing a lens with a hologram so that a Fourier transform of the hologram may be performed without the need for a physical Fourier lens. In some embodiments, the lensing data is combined with the hologram by a simple addition, such as a simple vector addition. In some embodiments, a physical lens is used in combination with a software lens to perform a Fourier transform. Alternatively, in other embodiments, the Fourier transform lens is omitted entirely so that the holographic reconstruction occurs in the far field. In further embodiments, the hologram may be combined with grating data in the same manner, i.e., data arranged to perform a grating function, such as beam steering. Again, it is known in the art of computer generated holography how to calculate such data. For example, a pure phase grating can be formed by modeling the phase delay caused by each point on the surface of a blazed grating. A pure amplitude grating can simply be superimposed with a pure amplitude hologram to provide angular steering of the holographic reconstruction.

[0090] In some embodiments, the Fourier transform is performed jointly by a physical Fourier transform lens and a software lens. That is, some of the optical power that contributes to the Fourier transform is provided by the software lens, while the remaining optical power that contributes to the Fourier transform is provided by one or more physical optical devices.

[0091] In some embodiments, a real-time engine is provided that is arranged to receive image data and calculate a hologram in real time using an algorithm. In some embodiments, the image data is a video comprising a sequence of image frames. In other embodiments, the hologram is pre-calculated, stored in a computer memory and called up as needed for display on the SLM. That is, in some embodiments, a repository of predetermined holograms is provided.

[0092] The embodiments relate to Fourier holography and Gerchberg-Saxton type algorithms by way of example only. The present disclosure is equally applicable to Fresnel holography and holograms computed by other techniques such as techniques based on point cloud methods.

[0093] Light Modulation

[0094] A spatial light modulator can be used to display diffraction patterns including computer generated holograms. If the hologram is a pure phase hologram, a spatial light modulator that modulates the phase is required. If the hologram is a full complex hologram, a spatial light modulator that modulates both phase and amplitude can be used, or a first spatial light modulator that modulates the phase and a second spatial light modulator that modulates the amplitude can be used.

[0095] In some embodiments, the light modulation elements (i.e., pixels) of the spatial light modulator are cells comprising liquid crystals. That is, in some embodiments, the spatial light modulator is a liquid crystal device in which the optically active component is a liquid crystal. Each liquid crystal cell is configured to selectively provide multiple light modulation levels. That is, each liquid crystal cell is configured to operate at a light modulation level selected from a plurality of possible light modulation levels at any one time. Each liquid crystal cell can be dynamically reconfigured to a light modulation level different from the plurality of light modulation levels. In some embodiments, the spatial light modulator is a reflective liquid crystal on silicon (LCOS) spatial light modulator, but the present disclosure is not limited to this type of spatial light modulator.

[0096] LCOS devices provide a dense array of light modulating elements or pixels within a small aperture (e.g., a few centimeters wide). Pixels are typically about 10 microns or smaller, which results in a diffraction angle of a few degrees, meaning that the optical system can be compact. It is much easier to fully illuminate the small aperture of an LCOS SLM than the large apertures of other liquid crystal devices. LCOS devices are typically reflective, which means that the circuitry that drives the LCOS SLM pixels can be buried under the reflective surface. The result is a higher aperture ratio. In other words, the pixels are tightly packed, which means there are almost no dead spaces between pixels. This is advantageous because it reduces optical noise in the playback field. LCOS SLMs use a silicon backplane, which has the advantage that the pixels are optically flat. This is particularly important for phase modulation devices.

[0097] The following is just an example, Figure 3 A suitable LCOS SLM is described below. An LCOS device is formed using a single crystal silicon substrate 302. It has a 2D array of square planar aluminum electrodes 301, which are spaced apart by gaps 301a and arranged on the upper surface of the substrate. Each electrode 301 can be addressed by a circuit 302a buried in the substrate 302. Each electrode forms a respective plane mirror. An orientation layer 303 is disposed on the electrode array, and a liquid crystal layer 304 is disposed on the orientation layer 303. A second orientation layer 305 is disposed on a planar transparent layer 306, for example made of glass. A single transparent electrode 307, for example made of ITO, is disposed between the transparent layer 306 and the second orientation layer 305.

[0098] Each square electrode 301 together with the covered area of ​​the transparent electrode 307 and the intervening liquid crystal material defines a controllable phase modulation element 308, commonly referred to as a pixel. The effective pixel area or fill factor is the percentage of the total pixel that is optically active, taking into account the space between the pixels 301a. By controlling the voltage applied to each electrode 301 relative to the transparent electrode 307, the properties of the liquid crystal material of the individual phase modulation elements can be varied to provide a variable retardation to light incident thereon. The effect is to provide a pure phase modulation to the wavefront, i.e. no amplitude effects occur.

[0099] The described LCOS SLM outputs spatially modulated light in a reflective manner. A reflective LCOS SLM has the advantage that the signal lines, grating lines and transistors are located below the mirror, which results in a high fill factor (typically greater than 90%) and high resolution. Another advantage of using a reflective LCOS spatial light modulator is that the thickness of the liquid crystal layer can be half that required when using a transmissive device. This greatly increases the switching speed of the liquid crystal (a key advantage for projecting moving video images). However, the teachings of the present disclosure can be equally implemented using a transmissive LCOS SLM.

[0100] Head-up display

[0101] Figure 4 FIG. 4 shows a HUD in a vehicle such as a car. Figure 4 4 shows a windshield 430 and an engine hood (or bonnet) 435 of the vehicle. The HUD includes a picture generation unit “PGU” 410 and an optical system 420.

[0102] The PGU 410 includes a light source, a light receiving surface 460, and a processor (or computer) arranged to computer control the image content of the picture. The PGU 410 is arranged to generate a picture or a sequence of pictures on the light receiving surface 460. The light receiving surface 460 can be a screen or a diffuser. In some embodiments, the light receiving surface is plastic (i.e., made of plastic). The light receiving surface is disposed on the playback plane. However, the light receiving surface is optional, and the holographic reconstruction can be formed in free space. The holographic reconstructions formed according to the present disclosure are all formed on a common playback plane. In some embodiments, multiple holographic reconstructions are superimposed on the playback plane.

[0103] The optical system 420 includes an input port, an output port, a first mirror 421, and a second mirror 422. The first mirror 421 and the second mirror 422 are arranged to guide light from the input port of the optical system to the output port of the optical system. More specifically, the second mirror 422 is arranged to receive light of a picture from the PGU 410, and the first mirror 421 is arranged to receive light of a picture from the second mirror 422. The first mirror 421 is further arranged to reflect the received light of the picture to the output port. Therefore, the optical path from the input port to the output port includes a first optical path 423 (or a first optical path component) from the input to the second mirror 422 and a second optical path 424 (or a second optical path component) from the second mirror 422 to the first mirror 421. Of course, there is a third optical path (or optical path component) from the first mirror to the output port, but in Figure 4 No reference numerals are assigned to the figures. Figure 4 The optical configuration shown in can be referred to as a "z-folded" configuration due to the shape of the optical path.

[0104] The HUD is configured and positioned within the vehicle so that the light of the pictures from the output port of the optical system 420 is incident on the windshield 430 and is at least partially reflected by the windshield 430 to the user 440 of the HUD. Thus, in some embodiments, the optical system is arranged to form a virtual image of each picture in the windshield by reflecting the spatially modulated light from the windshield. The user 440 of the HUD (e.g., the driver of the car) sees a virtual image 450 of the picture in the windshield 430. Thus, in an embodiment, the optical system is arranged to form a virtual image of each picture on the windshield of the vehicle. The virtual image 450 is formed at a distance below the hood 435 of the car. For example, the virtual image may be 1 to 2.5 meters away from the user 440. The output port of the optical system 420 is aligned with a hole in the dashboard of the car so that the light of the picture is directed to the user 440 by the optical system 420 and the windshield 430. In this configuration, the windshield 430 acts as an optical combiner. In some embodiments, the optical system is arranged to form a virtual image of each picture on an additional optical combiner included in the system. The windshield 430 or the additional optical combiner (if included) combines the light from the real world scene with the light of the picture. Therefore, it can be understood that the HUD can provide an augmented reality including a virtual image of the picture. In some embodiments, the picture includes a plurality of superimposed light patterns. For example, the augmented reality information may include navigation information or information related to the speed of the motor vehicle. In some embodiments, the light forming the picture is output by incident on the windshield at a Brewster angle (also referred to as a polarization angle) or at a Brewster angle within 5 degrees (e.g., a Brewster angle within 2 degrees).

[0105] In some embodiments, the first mirror and the second mirror are arranged to fold the optical path from input to output to increase the optical path length without unduly increasing the physical size of the HUD.

[0106] The width and height of the picture formed on the light receiving surface 460 of the PGU 410 may be only a few centimeters. Therefore, the first mirror 421 and the second mirror 422 provide magnification together or individually. That is, the first mirror and / or the second mirror can have optical power (i.e., diopter or focusing power). Therefore, the user 440 sees an enlarged virtual image 450 of the picture formed by the PGU. In some embodiments, the first mirror 421 and the second mirror 422 can also at least partially correct optical distortions that typically have complex curved shapes, such as caused by the windshield 430. The folded optical paths in the mirrors and the optical power together can appropriately magnify the virtual image of the picture.

[0107] The PGU 410 of the present invention includes at least one holographic projection channel. Each holographic projection channel of the picture generation unit is referred to as a display channel in this article. According to the above disclosure, each holographic projection channel includes a light source, a spatial light modulator and a controller. The spatial light modulator is arranged to spatially modulate the light according to the hologram displayed on the spatial light modulator. The controller provides light modulation data (i.e., light modulation distribution or light modulation value array, such as phase delay value array) including a computer-generated hologram. The light modulation data may also include other functions, such as the beam steering function described elsewhere in this disclosure. In some embodiments, the controller selects a computer-generated hologram to output from a repository (e.g., a memory) including a plurality of computer-generated holograms. In other embodiments, the controller calculates and outputs a computer-generated hologram in real time. Each picture formed by the PGU 410 is a holographic reconstruction. That is, each picture is formed by the interference of spatially modulated light on the playback plane.

[0108] Display Channels

[0109] The present disclosure relates to a system including multiple single wavelength display channels. In some embodiments, each single wavelength display channel includes a single wavelength holographic projector that forms a single wavelength holographic reconstruction (i.e., a light pattern or picture). Multiple single wavelength light patterns are formed on a common playback plane. A full-color visible image can be formed using consistent red, green, and blue holographic reconstructions. In some embodiments, the hologram is a Fourier hologram. In other embodiments, the hologram is a Fresnel hologram.

[0110] Figure 5 Red, green and blue display channels are shown. The red display channel includes a first spatial light modulator 511, a first lens 521 and a first mirror 531. The green display channel includes a second spatial light modulator 512, a second lens 522 and a second mirror 532. The blue display channel includes a third spatial light modulator 513, a third lens 523 and a third mirror 533. Each monochrome display channel forms a monochrome holographic reconstruction (or picture) on the playback plane 560. Multiple monochrome holographic reconstructions can be superimposed.

[0111] The first spatial light modulator 511 displays a hologram corresponding to a red image. The first spatial light modulator 511 is illuminated with red light. The first lens 521 receives the spatially modulated light from the first spatial light modulator 511 and forms a red image on a playback plane 560. The first mirror 531 is disposed between the first lens 521 and the playback plane 560.

[0112] The second spatial light modulator 512 displays a hologram corresponding to a green image. The second spatial light modulator 512 is illuminated with green light. The second lens 522 receives the spatially modulated light from the second spatial light modulator 512 and forms a green image on the playback plane 560. The second mirror 532 is disposed between the second lens 522 and the playback plane 560.

[0113] The third spatial light modulator 513 displays a hologram corresponding to a blue image. The third spatial light modulator 513 is illuminated with blue light. The third lens 523 receives the spatially modulated light from the third spatial light modulator 513 and forms a blue image on the playback plane 560. The third mirror 533 is disposed between the third lens 523 and the playback plane 560.

[0114] The first mirror 531 is a first dichroic mirror arranged to reflect red light and transmit green light and blue light. The second mirror 532 is a second dichroic mirror arranged to reflect green light and transmit blue light. The third mirror 533 reflects blue light.

[0115] Each monochromatic optical path includes a first portion from the spatial light modulator to the mirror and a second portion from the mirror to the playback plane. In an embodiment, the first portions of the respective monochromatic display channels are spatially offset but substantially parallel. In an embodiment, the respective second portions of the monochromatic display channels are substantially collinear.

[0116] The red light path from the first spatial light modulator 511 to the playback plane 560 includes reflection from the first mirror 531. The green light path from the second spatial light modulator 512 to the playback plane 560 includes reflection from the second mirror 532, followed by transmission through the first mirror 531. The blue light path from the third spatial light modulator 513 to the playback plane 560 includes reflection from the third mirror 533, followed by transmission through the second mirror 532, and then transmission through the first mirror 531. The playback plane 560, the first mirror 531, the second mirror 532 and the third mirror 533 are substantially collinear. The blue path length is greater than the green path length, and the green path length is greater than the red path length. Specifically, in an embodiment, the second portion of the blue light path is longer than the second portion of the green light path, and the second portion of the green light path is longer than the second portion of the red light path. In these embodiments, the lengths of the first portions may be substantially equal.

[0117] The red, green and blue display channels are part of a picture generation unit of a head-up display. The red, green and blue display channels are described only by way of example. The picture generation unit may include any number of single wavelength visible light display channels.

[0118] Figure 5Also shown is an infrared display channel including a fourth spatial light modulator 514, a fourth lens 524, and a fourth mirror 534. The infrared display channel also forms a holographic reconstruction on a playback plane 560. The holographic reconstruction formed by the infrared display channel is an infrared light pattern. The infrared light pattern may have a superimposed picture. That is, the infrared light pattern may at least partially overlap with the monochromatic light pattern formed by the visible light display channel.

[0119] The fourth spatial light modulator 514 displays a hologram corresponding to the infrared light pattern. The fourth spatial light modulator 514 is illuminated with infrared light. The fourth lens 524 receives the spatially modulated light from the fourth spatial light modulator 514 and forms the infrared light pattern on the playback plane 560. The fourth mirror 534 is disposed between the first lens 524 and the playback plane 560.

[0120] The infrared light path from the fourth spatial light modulator 514 to the playback plane 560 includes reflection from the fourth mirror 524, followed by transmission through the third mirror 533, then transmission through the second mirror 532, and finally transmission through the first mirror 531. Therefore, the third mirror 533 is also a dichroic mirror that transmits infrared light. The fourth mirror 534 is substantially collinear with the playback plane 560, the first mirror 531, the second mirror 532, and the third mirror 533. It will be apparent to those skilled in the art that the display channels in the plurality of display channels may be ordered in any manner. Those skilled in the art will be able to select a suitable dichroic mirror for any order of the display channels. The infrared display channel is illustrated only by way of example. Figure 5 Structurally and optically, the infrared display channel is integrated with the visible light display channel. However, functionally, the infrared display channel is not part of the picture generation unit, as will be better understood below.

[0121] Figure 6 Key components of a heads-up display according to the present disclosure are shown. Notably, components of the infrared display channel are part of the driver monitoring system 650 .

[0122] Specifically, Figure 6 A picture generation unit 600 is shown comprising a plurality of monochromatic visible light channels. The picture generation unit 600 is arranged to receive an input image 605 for projection. Each visible light channel comprises a separate controller 610, 620, 630 arranged to ensure that the image received for projection is faithfully projected. Figure 6 In the embodiment of the present invention, a red controller 610, a green controller 620 and a blue controller 630 are provided.

[0123] The red controller 610 controls a red light display channel arranged to form a red light pattern on the playback plane by holographic projection. The wavelength of the red light can be 640nm+ / -25nm. The green controller 620 controls a green light display channel arranged to form a green light pattern on the playback plane by holographic projection. The wavelength of the green light can be 530nm+ / -25nm. The blue controller 630 controls a blue light display channel arranged to form a blue light pattern on the playback plane by holographic projection. The wavelength of the blue light can be 450nm+ / -25nm. The three monochromatic light patterns can be superimposed on the playback plane to form a full-color reconstruction of each input image.

[0124] The picture generation unit 600 receives a plurality of different inputs for ensuring faithful reproduction of the input image. Each controller 610, 620, 630 coordinates a corresponding laser diode driver 612, 622, 632 and a spatial light modulator driver 614, 624, 634. Each laser diode driver 612, 622, 632 drives a corresponding laser diode 616, 626, 636. Each spatial light modulator driver 614, 624, 634 drives a corresponding spatial light modulator 618, 628, 638. Each controller coordinates the spatial light modulator and the laser diode pair so as to display and illuminate the desired light modulation pattern. In particular, each controller 610, 620, 630 ensures that the corresponding spatial light modulator 618, 628, 638 is synchronized with the corresponding laser diode 616, 626, 636.

[0125] The red controller 610 determines a computer generated hologram that, when displayed and illuminated with red light, will reconstruct the red image content of the input image 605. The red controller 610 may compute the computer generated hologram based on the red image content, or may retrieve a pre-computed computer generated hologram from a memory. The red controller 610 may combine the computer generated hologram with other functions, such as beam steering functions or lens functions, as described elsewhere in this disclosure, so that the light modulation pattern including the computer generated hologram is appropriately displayed on each spatial light modulator. The red controller 610 may distort the red image content of each input image before computing the hologram in order to compensate for measured non-uniformities in an optical device such as an optical combiner.

[0126] The green controller 620 determines a computer generated hologram that, when displayed and illuminated with red light, will reconstruct the green image content of the input image 605. The green controller 620 can compute the computer generated hologram based on the green image content, or can retrieve a pre-computed computer generated hologram from a memory. The green controller 620 can combine the computer generated hologram with other functions such as beam steering functions or lens functions, as described elsewhere in this disclosure, so that the light modulation pattern including the computer generated hologram is appropriately displayed on each spatial light modulator. The green controller 620 can distort the green image content of each input image before computing the hologram to compensate for measured non-uniformities in an optical device such as an optical combiner.

[0127] The blue controller 630 determines a computer generated hologram that, when displayed and illuminated with red light, reconstructs the blue image content of the input image 605. The blue controller 630 can calculate the computer generated hologram based on the blue image content, or can retrieve a pre-calculated computer generated hologram from a memory. The blue controller 630 can combine the computer generated hologram with other functions such as beam steering functions or lens functions, as described elsewhere in this disclosure, so that the light modulation pattern including the computer generated hologram is appropriately displayed on each spatial light modulator. The blue controller 630 can distort the blue image content of each input image before calculating the hologram to compensate for measured non-uniformities in optical devices such as optical combiners.

[0128] Figure 6Also shown is a driver monitoring system 650, which includes a camera 660 and an IR controller 670. The camera 660 is arranged to capture an infrared image of the driver. The driver monitoring system 650 also includes an image processor 662, which is arranged to process the captured image of the driver and determine the absolute position of the driver or the position change of the driver. For example, the image processor 662 can track the eyes or the head. The driver monitoring system 650 outputs data representing the driver's position. The IR controller 670 includes a laser diode driver 672, a laser diode 676, a spatial light modulator driver 674 and a spatial light modulator 678, which are arranged to irradiate the driver with infrared light to improve image capture. The IR wavelength can be 1200 to 1800nm, such as 1500 to 1600nm. The IR controller 670 determines a computer-generated hologram that will form an infrared light pattern. The infrared light pattern can be dynamically changed by dynamically changing the light modulation pattern displayed. For example, the infrared light pattern can be linearly shifted in response to the driver's head movement by changing the beam steering (e.g., grating) function combined with the computer-generated hologram. The infrared light pattern can be changed to increase the intensity of the infrared light in low reflectivity areas. The IR controller can receive positional information about the driver from the image processor 662 and determine the infrared light pattern based on the positional information. For example, the IR controller can be configured to ensure that the driver's face is evenly illuminated with IR light. Any desired infrared light pattern can be formed, and the skilled person will understand how the described feedback system can cooperate with the infrared display channel to provide a dynamically changeable IR illumination system for the driver monitoring system 650.

[0129] In some embodiments, the head-up display further comprises a memory. The memory may contain a repository of pre-computed holograms. The memory may contain correction data, such as an infrared correction map and / or a visible light correction map. In some embodiments, the correction map is provided to one or more display channel controllers. The infrared correction map may be provided to a controller of an infrared display channel. The visible light correction map may be provided to a controller of a visible light display channel. Each correction map represents a system-wide optical response of the head-up display. Each correction map is wavelength-specific. The infrared correction map represents the system-wide optical response of the head-up display to infrared light of the infrared display channel. The visible light correction map represents the system-wide optical response of the head-up display to visible light of the visible light display channel. The light pattern formed at the playback plane may not be perfectly relayed to the eye box. That is, some unwanted changes may occur to the light pattern. For example, the change may be a distortion of the spatial variation of the light pattern or a change in the spatial variation of the brightness of the light pattern. Likewise, the change may be wavelength-dependent.

[0130] In some embodiments, the optical combiner is a windshield of a vehicle. In some embodiments, a light pattern is formed on a screen (e.g., a diffuser) at a playback plane. Variations in the light pattern may be caused by optical components of the head-up display, such as the optical combiner or the screen. In some embodiments, the correction map compensates for the spatially varying optical power of the windshield. In some embodiments, the correction map compensates for the non-uniform diffusivity of the screen. The correction map provides a system-wide correction that ensures that the light distribution within the eye box faithfully reconstructs the input image. According to the present disclosure, a light pattern is formed by holographic projection from a pixelated display device (i.e., a spatial light modulator). The pixelated structure forms a holographic playback field that may have non-uniform brightness. This non-uniformity can be corrected using the correction map described herein.

[0131] In some embodiments, correction maps are provided for multiple different head or eye positions of a viewer within the eye box. More specifically, a unique correction map is provided for each channel and each of the multiple head or eye positions of a viewer within the eye box. This is because the image formed at each viewpoint is formed by a specific bundle of light rays, each of which follows a specific light path through the system. Light rays can pass through different parts of components (such as lenses) and can be reflected from different parts of other components (such as optical combiners). Therefore, head or eye tracking information is used to determine an infrared correction map output to a controller for an infrared display channel and a visible light correction map output to a controller for a visible light display channel. Each correction map compensates for the system-wide optical response of the head-up display so that each light distribution relayed to the eye box is faithful to the input (or source) light pattern, regardless of the eye position within the eye box.

[0132] Each correction map can be determined from different positions within the eye box using ray tracing. Alternatively, each correction map can be determined during a calibration process. The calibration process can include capturing images from the eye box using a camera. That is, the calibration process includes positioning the camera at different positions in the eye box and capturing images of the relayed light pattern. The captured image can be compared with the input (or source) image. The difference between each captured image and the input image can be measured at multiple points within the eye box. For example, an array of light spots can be projected into the eye box. The relative brightness of each light spot can be used to form a brightness correction map for each wavelength. Those skilled in the art will understand how such measurements can be used to form a brightness correction map. The brightness correction map can be considered as an array of brightness gain values. Before determining the corresponding hologram, each input image or pattern for projection can be pre-processed using the brightness correction map.

[0133] In some embodiments, a picture generation unit receives a color image for projection. The picture generation unit separates the color image into red, green, and blue image components and sends each to a corresponding monochrome display channel. The controller of each monochrome display channel preprocesses the received monochrome image using a corresponding monochrome correction map (e.g., multiplying by the inverse of a brightness correction map) and then calculates a hologram corresponding to the monochrome image. In some embodiments, each hologram is calculated in real time using the above-described Fourier hologram calculation method. Similarly, the driver monitoring system can determine (e.g., calculate) a hologram corresponding to a desired infrared light pattern in real time after preprocessing the image data using an infrared correction map, such as an infrared brightness correction map.

[0134] It will be understood that in some embodiments, the picture generation unit and / or the driver monitoring system uses feedback based on the head or eye tracking system to change the hologram (or light modulation pattern) and thus change the projected light pattern. As described above, the infrared light pattern can be moved or shaped to compensate for head movement. In addition, in some embodiments, feedback from head or eye tracking is used to determine (e.g., select) a correction map from a plurality of correction maps. This is because the correction map for each wavelength depends on the viewpoint within the eye box. Therefore, if the viewer moves, a different correction map should be used for each display channel. The described feedback system is used to identify a correction map for the infrared display channel and / or a correction map for the visible light display channel. It will be understood that correction maps are obtained for multiple eye box positions, and identifying an appropriate correction map may include identifying a correction map for the nearest measured viewpoint.

[0135] It is noteworthy that the use of computer-generated holograms can dynamically reconfigure the overall shape of the infrared light pattern that illuminates the viewer in the eye box area. Therefore, it is not necessary to flood the interior of the vehicle with infrared light extensively to ensure that head or eye tracking data can be obtained. In addition, the infrared display channel is relayed by the same optical system as the picture of the picture generation unit. Therefore, the infrared light field is advantageously confined to the eye box area. That is, there is no need to illuminate the area outside the eye box with infrared light. In some embodiments, the optical combiner includes an optical coating configured to increase reflectivity at infrared wavelengths. In some embodiments, the camera of the driver monitoring system is disposed behind one of the mirrors of the optical system, and the mirror is adapted to transmit at infrared wavelengths. In other embodiments, the camera is located behind a dichroic mirror (e.g., a first mirror 531) of one of the visible light display channels (e.g., a red light display channel).

[0136] Other Examples

[0137] Fig. 7A and 7B An example heads-up display system is shown that includes a driver monitoring system including an infrared illumination system and an image capture device.

[0138] Fig. 7A A cross section of a head up display system is shown. The head up display comprises a freeform mirror 706 arranged to relay light of a picture from a picture generating unit (not shown) to a viewer 710 in an eye box. The head up display comprises a glare catcher 704. Fig. 7A An infrared light path 714 is shown from a viewer 710 to an infrared image capture device 712. The viewer 710 is illuminated with infrared light, which will be referred to below. Figure 7B 708. The infrared light path 714 includes reflections from the inner surface of the vehicle windshield 702 and reflections from the beam splitter 708. The beam splitter 708 is reflective to infrared light and transmissive to visible light. Therefore, the beam splitter 708 does not disrupt the light of one or more visible light patterns. It is noteworthy that the infrared image capture device 712 is located within the volume of the head-up display. The infrared camera is located below the protective glass. The glare capturer 704 is transmissive to infrared light.

[0139] Figure 7B A top view of the glare catcher 704 is shown. An array of infrared light sources, such as infrared light source 718, is packaged next to the glare catcher. Optionally, the infrared light source is packaged with a camouflage film that is opaque to visible light and transparent to infrared light. The infrared light source is arranged to emit infrared light. All emitted infrared light is directed to the eye box.

[0140] Therefore, a head-up display is provided, comprising: a picture generating unit, an optical system comprising at least one mirror, a protective glass and a beam splitter which is reflective to infrared light and transmissive to visible light, wherein the head-up display is characterized in that it also comprises an infrared image capturing device arranged within the volume of the optical system and a plurality of infrared light sources packaged on top of the glare capturer, wherein the infrared light sources are arranged to illuminate only the eye box area.

[0141] Additional Features

[0142] The embodiments relate to electrically activated LCOS spatial light modulators by way of example only. The teachings of the present disclosure may equally be implemented on any spatial light modulator capable of displaying a computer-generated hologram according to the present disclosure, such as any electrically activated SLM, optically activated SLM, digital micromirror device, or micro-electromechanical device.

[0143] In an embodiment, the holographic reconstruction is in color. In some embodiments, a method known as spatially separated colors "SSC" is used to provide color holographic reconstruction. The SSC method uses three spatially separated spatial light modulators for three monochrome holograms. An advantage of the SSC method is that the image can be very bright because all three holographic reconstructions can be formed simultaneously.

[0144] The methods and processes described herein may be embodied on a computer-readable medium. The term "computer-readable medium" includes media arranged to store data temporarily or permanently, such as random access memory (RAM), read-only memory (ROM), buffer memory, flash memory, and cache memory. The term "computer-readable medium" should also be considered to include any medium or combination of media capable of storing instructions for execution by a machine, so that when the instructions are executed by one or more processors, the machine performs, in whole or in part, any one or more of the methods described herein.

[0145] The term "computer-readable medium" also encompasses cloud-based storage systems. The term "computer-readable medium" includes, but is not limited to, one or more tangible and non-transitory data storage repositories (e.g., data volumes) in the example form of solid-state memory chips, optical disks, magnetic disks, or any suitable combination thereof. In some example embodiments, instructions for execution may be conveyed by a carrier medium. Examples of such carrier media include transient media (e.g., propagated signals conveying instructions).

[0146] It will be apparent to those skilled in the art that various modifications and variations can be made without departing from the scope of the appended claims. The present disclosure covers all modifications and variations within the scope of the appended claims and their equivalents.

Claims

1. A head-up display having a glare catcher and an eye box, wherein the head-up display comprises: Image generation unit (600); An optical system comprising at least one mirror, a protective glass, and a beam splitter (708), wherein the beam splitter (708) is substantially reflective to infrared light and transmissive to visible light; an infrared image capture device (712), the infrared image capture device (712) being disposed within the volume of the optical system; as well as a plurality of infrared light sources (718), the plurality of infrared light sources (718) being encapsulated with the glare capturer (704), wherein the infrared light sources (718) are arranged to illuminate the eye box, wherein the glare capturer (704) is transmissive to infrared light, wherein a plurality of infrared light sources (718) are packaged next to the glare capturer (704), and all emitted infrared light is directed to the eye box, and wherein infrared light reflected from a viewer (710) at the eye box has an infrared light path (714) through the optical system to the infrared image capture device, the glare capturer (704) is in the infrared light path (714), and wherein the infrared light path (714) has a path through the glare capturer (704) and also includes a path reflected from the beam splitter (708) to the infrared image capture device (712).

2. The head-up display according to claim 1, wherein: The plurality of infrared light sources (718) are packaged using a camouflage film.

3. The head-up display according to claim 1, wherein: The plurality of infrared light sources (718) are packaged on top of the glare capturer (704).

4. The head-up display according to claim 1, wherein: The plurality of infrared light sources (718) is an array of infrared light sources arranged to emit infrared light.

5. The head-up display according to claim 1, wherein: The at least one mirror comprises a free-form surface mirror (706) arranged to relay light of a picture generated by a picture generation unit (600) to the eye box.

6. The head-up display according to claim 1, wherein: The beam splitter (708) is non-destructive to the light of the picture generated by the picture generation unit (600).

7. The head-up display according to claim 1, wherein: The picture generation unit (600) includes at least one visible light channel.

8. The head-up display according to claim 7, wherein: Each visible light channel includes: a visible light source (616, 626, 636) arranged to emit visible light; and A spatial light modulator (618, 628, 638) is arranged to receive said visible light from said visible light source (616, 626, 636) and to output spatially modulated light in accordance with a computer generated hologram displayed on the spatial light modulator (618, 628, 638) in use to form a corresponding light pattern on a playback plane in use.

9. The head-up display according to claim 8, wherein: Each visible light channel further comprises a controller (610, 620, 630) arranged, in use, to output a light modulation profile comprising a computer generated hologram to the spatial light modulator (618, 628, 638).

10. The head-up display according to claim 1, wherein: The infrared image capture device (712) and the plurality of infrared light sources (718) are part of a driver monitoring system, the driver monitoring system further comprising: An image processor (662) is arranged, in use, to receive a captured image of a viewer (710) and provide an input which, in use, determines a light modulation profile output by the at least one controller and thereby determines a corresponding light pattern formed on a playback plane.

11. A heads up display as claimed in claim 1 , wherein the heads up display is arranged to direct light of the picture to the eye box using an optical combiner in use.