Holographic display calibration by holographic phase modulation

Abstract

A method for holographic display calibration using phase modulation includes projecting an initial pattern on a windshield of a vehicle, capturing an image of the initial pattern using a camera in the vehicle, determining a loss function value between the captured image of the initial pattern by the camera and a target pattern, modulating a phase of a light beam generating the initial pattern with the loss function value to generate an updated pattern, and displaying the updated pattern on the windshield of the vehicle.

Description

Technical Field

[0001] This disclosure relates to a system and method for calibrating holographic displays using holographic phase modulation. Background Technology

[0002] This introduction provides an overview of the background of this disclosure. To the extent described in this introduction, the work of the currently named inventors, and aspects of this description that may not be considered prior art at the time of application, are neither explicitly nor implicitly regarded as prior art to this disclosure.

[0003] A holographic head-up display (HUD) projects an image onto a windshield. In a holographic HUD, a light source emits a coherent beam of light onto a spatial light modulator (SLM). The image on the SLM is then projected onto the windshield. The beam carrying the image information is then reflected off the windshield to create an image within the eye ellipse. However, the reconstructed image may be blurry due to systematic generation errors caused by the SLM, the pupil replicator, and the roughness and / or inhomogeneity of the windshield surface. Therefore, there is a need to develop a calibration method and system to minimize these systematic generation errors. Summary of the Invention

[0004] This disclosure describes a method and system for calibrating a holographic display using holographic phase modulation. The currently disclosed system includes an in-vehicle camera to capture images generated by a holographic HUD. In the method performed by this system, an image is projected onto the windshield using an initial hologram, and the camera captures an image of the initial hologram. An optimization algorithm is used to correct the initial hologram, thereby removing system-generated noise and enhancing contrast in the final output (i.e., the final image). After the algorithm completes, an updated optimized hologram is created. The difference between the initial hologram and the optimized hologram is stored as a lookup table. When the vehicle is in operation and the HUD is in use, the difference is applied to the hologram for each frame. Multiple lookup tables are created for multiple eye ellipse positions, and an eye tracker on the vehicle helps determine which lookup table to apply based on the driver's eye position.

[0005] In one aspect of this disclosure, a method for holographic display calibration using phase modulation includes: projecting an initial pattern onto a vehicle's windshield; capturing an image of the initial pattern within an eye ellipse using a camera inside the vehicle; determining a loss function value between the captured image of the initial pattern captured by the camera and a known target pattern; and modulating the phase of a beam that generates the initial pattern using the loss function value to generate an updated pattern. The method also includes displaying the updated pattern on the vehicle's windshield.

[0006] In one aspect of this disclosure, determining the loss function value includes determining that the loss function value is ideally zero or less than a threshold. The method also includes avoiding repeated capture, determination, modulation, and display steps in response to determining that the loss function value is less than the threshold. As a non-limiting example, the threshold could be 2%.

[0007] In one aspect of this disclosure, the value of the loss function is determined using the following formula:

[0008] in: It is the value of the loss function; M is the total number of pixels in the x-direction of the image captured by the camera; N is the total number of pixels in the y-direction of the image captured by the camera; m is one of the total number of pixels M in the x-direction; n is one of the total number of pixels N in the y direction; a target It is the target graphic; and a cap These are images captured by a camera.

[0009] In one aspect of this disclosure, modulating the phase of the beam includes modulating the phase of the beam from a first phase value to a second phase value. The method also includes determining a loss function value between an image of an initial pattern having the first phase value and a target pattern. The method further includes determining a loss function value between an image of an updated pattern generated by the beam having the second phase value and the target pattern.

[0010] In one aspect of this disclosure, the loss function value between the image of the initial pattern having a first phase value and the target pattern is a first loss function value. The loss function value between the image of the updated pattern generated by the beam having a second phase value and the target pattern is a second loss function value. The method further includes determining which of the first and second loss function values ​​has a minimum value. The method further includes storing the minimum value.

[0011] In one aspect of this disclosure, the method further includes determining an average contrast value for the first pixel of the hologram from an image of an initial graphic captured by a camera.

[0012] In one aspect of this disclosure, the average contrast value of the image of the initial graphic captured by the camera is determined using the following equation:

[0013] in: It is the average intensity of the white portion of the image captured by the camera; It is the average intensity of the background portion of the image captured by the camera; and C is the average contrast value.

[0014] In one aspect of this disclosure, the image is one of multiple images located at different eye ellipse positions. Capturing images of the initial pattern using an in-vehicle camera involves capturing multiple images at different eye ellipse positions.

[0015] This disclosure also describes a vehicle having a system for holographic calibration via holographic phase modulation. In one aspect of this disclosure, the vehicle includes a windshield and a holographic head-up display (HUD). The holographic HUD includes a spatial light modulator (SLM) and a light source. The light source is configured to emit a beam of light towards the SLM. The SLM is configured to modulate the beam of light. The holographic HUD is configured to project an initial pattern onto the windshield. The vehicle also includes a camera and a controller. The camera is positioned to capture an image of the initial pattern. The controller communicates with the holographic HUD and the camera. The controller is programmed to perform the methods described above.

[0016] Other applicable areas of this disclosure will become apparent from the detailed description provided below. It should be understood that the detailed description and specific examples are for illustrative purposes only and are not intended to limit the scope of this disclosure.

[0017] The above-described features and advantages, as well as other features and advantages, of the currently disclosed systems and methods will become apparent from the detailed embodiments (including the claims) and exemplary embodiments taken in conjunction with the accompanying drawings. Attached Figure Description

[0018] This disclosure will be more fully understood through detailed description and accompanying drawings, wherein: Figure 1 It is a block diagram of a vehicle including a system for holographic display calibration; Figure 2 This is a flowchart of a method for calibrating holographic displays using holographic phase modulation; Figure 3 This is a flowchart of a method for minimizing pattern errors using holographic phase modulation; and Figure 4 This is a flowchart of a method for enhancing contrast in graphics. Detailed Implementation

[0019] Several examples of this disclosure will now be shown in detail in the accompanying drawings. Wherever possible, the same or similar reference numerals are used in the drawings and description to refer to the same or similar parts or steps.

[0020] refer to Figure 1The vehicle 10 includes a windshield 12 and a holographic head-up display (HUD) 14 configured to project one or more graphics 24 onto the windshield 12. The holographic HUD 14 can be configured as an augmented reality (AR) HUD and includes a light source 16 (e.g., a coherent light source), a spatial light modulator (SLM) 18, and a lens 20 for directing the graphics 24 toward the windshield 12. In the holographic HUD 14, a coherent beam emitted by the light source 16 is used to project the graphics by illuminating the hologram displayed by the SLM 18. The beam carrying the graphic information is then reflected off the windshield 12 to create the graphics 24 in an eye ellipse 22. The term "eye ellipse" refers to a statistical representation of the driver's eye position within the vehicle 10.

[0021] Continue to refer to Figure 1 Vehicle 10 includes a system 26 for calibrating a holographic display 14 via holographic phase modulation. System 26 includes a controller 34 and one or more cameras 28 communicating with the controller 34. The cameras 28 are disposed within vehicle 10 and configured and positioned to capture one or more images of a pattern 24 projected toward an eye ellipse 22 within vehicle 10. The controller 34 includes at least one processor 44 and a non-transitory computer-readable storage device or medium 46. The processor 44 may be a custom or commercially available processor, a central processing unit (CPU), a graphics processing unit (GPU), an auxiliary processor among several processors associated with the controller 34, a semiconductor-based microprocessor (in the form of a microchip or chipset), a macroprocessor, a combination thereof, or a device generally used for executing instructions. For example, the computer-readable storage device or medium 46 may include volatile and non-volatile memory in the form of read-only memory (ROM), random access memory (RAM), and keep-alive memory (KAM). KAM is persistent or non-volatile memory that can be used to store various operational variables when the processor 44 is powered off. The computer-readable storage device or medium 46 can be implemented using various storage devices, such as a programmable read-only memory (PROM), an electrically programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), flash memory or other electrical storage devices capable of storing data, magnetic storage devices, optical storage devices, or combinations thereof, some of which represent executable instructions used by the controller 34 in controlling the vehicle 10. The controller 34 of the vehicle 10 may be referred to as a vehicle controller and can be programmed to execute methods 100, 200, and 300, which are described in detail below. The controller 34 also communicates with the light source 16 and the SLM 18. Therefore, the controller 34 is programmed to control the operation of the light source 16 and the SLM 18.

[0022] Figure 2This is a flowchart of method 100 for calibrating a holographic HUD 14 using phase modulation. The holographic HUD 14 may produce graphic errors that result in poor image quality, and method 100 uses optimization-based calibration to correct these errors, thereby improving image quality. Specifically, method 100 adaptively modifies the holographic phase and creates a lookup table for minimizing errors, including those from the windshield 12 and the pupil replicator. Method 100 begins at block 102. Then, method 100 proceeds to block 104. At block 104, camera 28 is moved (if necessary) such that the field of view of camera 28 points to eye ellipse 22. As mentioned above, the term "eye ellipse" refers to a statistical representation of the driver's eye position within vehicle 10. At this point, camera 28 can capture an image of the graphic generated by the holographic HUD 14, as perceived by the vehicle driver. It is conceivable that camera 28 can be moved within vehicle 10. For this purpose, controller 34 is programmed to command camera 28 to move until the field of view of camera 28 points to eye ellipse 22. Then, method 100 continues to box 106.

[0023] At block 106, controller 34 commands holographic HUD 14 to project an initial image onto the windshield 12 of vehicle 10. Next, method 100 proceeds to block 108. At block 108, once the initial image on the windshield 12 of vehicle 10 is projected onto the windshield 12, controller 34 commands camera 28 to capture an image of the initial image generated by holographic HUD 14. Upon receiving this command, camera 28 captures an image of the initial image. Then, method 100 continues to block 110. At block 110, controller 34 executes method 200 described below. Figure 3 ) and / or method 300 ( Figure 4The method 100 proceeds to block 112. At block 112, the controller 34 uses the iterative optimization algorithm executed at block 110 to adaptively modularize the phase of the beam of the pattern 24, thereby improving the quality of the pattern 24. In other words, the phase of the hologram displayed on the SLM is modulated. Next, method 100 proceeds to block 114. At block 114, the controller 34 uses the iterative optimization algorithm executed at block 110 to generate an updated optimized pattern. Then, method 100 proceeds to block 114. At block 114, the controller 34 creates multiple lookups with phase values ​​that produce the lowest loss function value between the image of the initial pattern 24 captured by the camera 28 and the target pattern, which can be stored on a computer-readable medium 46. The phase values ​​can be stored, for example, on a computer-readable storage device or medium 46. The camera 28 can be used as an eye tracker to track the head and eye position of the vehicle driver. Therefore, the controller 34 can create multiple lookup tables for different eye ellipse positions. The controller 34 is programmed to select and apply an appropriate lookup table based on the vehicle driver's head and / or position. Method 100 then proceeds to block 116. At block 116, method 100 ends.

[0024] Figure 3 This is a flowchart of method 200 for minimizing errors in graph 24. Method 200 begins at box 202. Then, method 200 proceeds to box 204. At box 204, camera 28 is moved (if necessary) so that the field of view of camera 28 points to eye ellipse 22. Next, method 200 proceeds to box 206.

[0025] At frame 206, controller 34 commands holographic HUD 14 to project an initial graphic onto the windshield 12 of vehicle 10. The graphic is reflected from the windshield 12 onto the eye ellipse. Next, method 200 proceeds to frame 208. At frame 208, once the initial, noisy graphic 24 has been projected onto the windshield 12 of vehicle 10, controller 34 commands camera 28 to capture an image of the initial graphic generated by holographic HUD 14 within the eye ellipse. Upon receiving this command, camera 28 captures an image of the initial graphic 24 within the eye ellipse. Then, method 200 continues to frame 210.

[0026] At box 210, controller 34 determines (e.g., calculates) the loss function value between the initial graphic 24 and the target graphic (which are stored on a non-transitory computer-readable storage device or medium 46) for each pixel. For this purpose, controller 34 can use the following formula:

[0027] in: It is the loss function value for a single pixel; M is the total number of pixels in the x-direction of the image captured by camera 28; N is the total number of pixels in the y-direction of the image captured by camera 28; m is one of the total number of pixels M in the x-direction; n is one of the total number of pixels N in the y direction; a target It is the target graphic; and a cap These are images captured by camera 28.

[0028] The x-direction can be horizontal, and the y-direction can be vertical. After box 210, method 200 continues to box 212. At box 212, controller 34 selects the first pixel of the initial or subsequently updated graphic 24. Then, method 200 continues to box 214. At box 214, controller 34 commands SLM 18 to operate in small predetermined steps (± The method iterates k times at a selected pixel (e.g., the first pixel), thereby changing the phase of the beam. After changing the phase value of the beam at the selected pixel (e.g., the first pixel) of the initial pattern 24 (or subsequently updated pattern 24), the controller 34 commands the camera 28 to capture the updated pattern 24 in k iterations. Upon receiving the command from the controller 34, after changing the phase value of the beam at the selected pixel (e.g., the first pixel) of the pattern 24, the camera 28 captures an image of the updated pattern 24. Then, method 200 proceeds to box 216. At box 216, the controller 34 uses the formula above to calculate the loss function value between the initial (or subsequently updated) pattern 24 and the target pattern in each of the k iterations. Then, method 200 proceeds to box 218. At box 218, the controller 34 saves and maintains the phase value of the loss function of the pattern 24 at the selected pixel (e.g., the first pixel) with the minimum value. Next, method 200 proceeds to box 220.

[0029] At box 220, controller 34 determines whether the actions described in boxes 214, 216, and 218 have been performed on all pixels of graphic 24. If not all pixels of graphic 24 have been analyzed according to boxes 212, 214, 216, and 218, method 200 proceeds to box 222. At box 222, controller 34 selects another pixel of graphic 24. Then, method 200 proceeds to box 214.

[0030] At box 220, if controller 34 determines that the actions described in boxes 214, 216, and 218 have been performed on all pixels of graphic 24, method 200 proceeds to box 224. At box 224, controller 43 determines that the actions in boxes 212, 214, 216, and 218 should be repeated for all pixels of SLM 18. Therefore, after box 224, method 200 returns to box 212. Method 200 terminates when the total loss function value between the latest updated graphic 24 and the target graphic is less than a predetermined value (e.g., 2%). In other words, when the total loss function value between the latest updated graphic 24 and the target graphic is less than a predetermined value (e.g., 2%), controller 34 does not perform any further actions of method 200.

[0031] Figure 4 This is a flowchart of method 300 for enhancing contrast in graphic 24. Method 300 begins at box 302. Then, method 300 proceeds to box 304. At box 304, camera 28 is moved (if necessary) so that the field of view of camera 28 is directed toward eye ellipse 22. Next, method 300 proceeds to box 306.

[0032] At block 306, controller 34 commands holographic HUD 14 to project an initial graphic onto the windshield 12 of vehicle 10. Next, method 300 proceeds to block 308. At block 308, once the initial, noisy graphic on the windshield 12 of vehicle 10 has been projected onto the windshield 12, controller 34 commands camera 28 to capture an image of the initial graphic generated by holographic HUD 14. Upon receiving this command, camera 28 captures an image of the initial graphic. Then, method 300 continues to block 310.

[0033] At box 310, controller 34 determines (e.g., calculates) the average contrast value of the image captured by camera 28 for each pixel of the image pattern captured by camera 28. For this purpose, controller 34 can use the following formula:

[0034] in: It is the average intensity of the white portion of the image captured by camera 28; It is the average intensity of the background portion of the image captured by camera 28; and C is the average contrast value.

[0035] Following box 310, method 300 proceeds to box 312. At box 312, controller 34 selects the first pixel of the initial or subsequently updated graphic 24. Then, method 300 proceeds to box 314. At box 314, controller 34 commands SLM 18 to operate in small predetermined steps (±... The method iterates k times at a selected pixel (e.g., the first pixel) to change the phase of the light beam. After changing the phase value of the light beam at the selected pixel (e.g., the first pixel) of the initial pattern 24 (or the subsequently updated pattern 24), the controller 34 commands the camera 28 to capture the updated pattern 24 in k iterations. Upon receiving the command from the controller 34, after changing the phase value of the light beam at the selected pixel (e.g., the first pixel) of the pattern 24, the camera 28 captures an image of the updated pattern 24. Then, method 300 proceeds to box 316. At box 316, the controller 34 calculates the average contrast value for each of the k iterations using the formula above. Then, method 300 proceeds to box 318. At box 318, the controller 34 saves and holds the phase value of the average contrast value of the pattern 24 that has the highest value at the selected pixel (e.g., the first pixel). Next, method 300 proceeds to box 320.

[0036] At box 320, controller 34 determines whether the actions described in boxes 314, 316, and 318 have been performed on all pixels of graphic 24. If not all pixels of graphic 24 have been analyzed according to boxes 312, 314, 316, and 318, method 300 proceeds to box 322. At box 322, controller 34 selects another pixel of graphic 24. Then, method 300 proceeds to box 314. Therefore, after box 322, method 300 returns to box 314.

[0037] At box 320, if controller 34 determines that the actions described in boxes 314, 316, and 318 have been performed on all pixels of pattern 24, method 300 continues to box 324. At box 324, controller 43 determines that boxes 312, 314, 316, and 318 should be repeated for all pixels of SLM 18. Method 300 terminates when the overall average contrast value between the most recently updated pattern 24 and the target hologram increases by less than a predetermined contrast threshold (e.g., 1%). In other words, when the overall average contrast value between the most recently updated pattern 24 and the target hologram increases by less than a predetermined contrast threshold (e.g., 1%), controller 34 does not perform any further actions of method 300.

[0038] Although exemplary embodiments have been described above, these embodiments are not intended to describe all possible forms of the invention. Rather, the wording used in the specification is illustrative rather than limiting, and it should be understood that various changes can be made without departing from the spirit and scope of the invention. As described above, features of various embodiments can be combined to form further embodiments of the currently disclosed systems and methods, which are not described in detail or shown. While various embodiments may be described as providing advantages or superiority over other embodiments or prior art implementations for one or more desired characteristics, those skilled in the art will recognize that one or more features or characteristics can be compromised depending on the specific application and implementation to achieve the desired overall system properties. These characteristics may include, but are not limited to, cost, strength, durability, life cycle cost, merchantability, appearance, packaging, size, serviceability, weight, manufacturability, ease of assembly, etc. Therefore, embodiments described as less desirable in one or more features than other embodiments or prior art implementations are not outside the scope of this disclosure and may be ideal for a particular application.

[0039] The accompanying drawings are simplified and not drawn to exact scale. For convenience and clarity only, directional terms such as top, bottom, left, right, upward, above, above, below, under, behind, and front may be used with reference to the drawings. These and similar directional terms should not be construed as limiting the scope of this disclosure in any way.

[0040] This document describes embodiments of the present disclosure. However, it should be understood that the disclosed embodiments are merely examples and other embodiments may take various and alternative forms. The drawings are not necessarily drawn to scale, and some features may be enlarged or minimized to show details of particular components. Therefore, the specific structural and functional details disclosed herein should not be construed as limiting, but merely as a representative basis for teaching those skilled in the art to use the currently disclosed systems and methods in different ways. As will be understood by those skilled in the art, the various features shown and described with reference to any of the drawings may be combined with features shown in one or more other drawings to produce embodiments not explicitly shown or described. The combinations of features shown provide representative embodiments for typical applications. However, for a particular application or implementation, various combinations and modifications of features consistent with the teachings of the invention are desirable.

[0041] Embodiments of this disclosure are described herein in accordance with functional and / or logical block components and various processing steps. It should be understood that such block components can be implemented by multiple hardware, software, and / or firmware components configured to perform specific functions. For example, embodiments of this disclosure may employ various integrated circuit elements, such as memory elements, digital signal processing elements, logic elements, or lookup tables, which can perform multiple functions under the control of one or more microprocessors or other control devices. Furthermore, those skilled in the art will understand that embodiments of this disclosure can be practiced in conjunction with multiple systems, and that the systems described herein are merely exemplary embodiments of this disclosure.

[0042] For the sake of brevity, techniques relating to signal processing, data fusion, signaling, control, and other functional aspects of the system (as well as individual operating components of the system) may not be described in detail herein. Furthermore, the connecting lines shown in the various figures included herein are intended to illustrate exemplary functional relationships and / or physical couplings between various elements. It should be noted that many alternative or other functional relationships or physical connections may exist in the embodiments of this disclosure.

[0043] This specification is merely exemplary in nature and is not intended to limit the scope of this disclosure or its application or use. The broad teachings of this disclosure can be implemented in many forms. Therefore, while this disclosure includes specific examples, its true scope should not be limited thereto, as other modifications will become apparent upon examination of the drawings, specification, and appended claims.

Claims

1. A method for holographic display calibration using phase modulation, comprising: Project the initial graphic onto the vehicle's windshield; Using a camera inside the vehicle, an image of the initial pattern is captured within the eye ellipse; The phase value of the beam that generates the initial image is iteratively changed through predetermined steps to generate multiple updated images until the loss function value between each of the multiple updated images and the target image is less than a predetermined value. Iteratively determine the loss function value between each of the plurality of updated graphics and the target graphics, until the loss function value is less than the predetermined value; as well as The latest of the plurality of updated graphics is displayed on the windshield of the vehicle; The loss function value is determined using the following formula: in: It is the value of the loss function; M is the total number of pixels in the image captured by the camera in the x-direction; N is the total number of pixels in the image captured by the camera in the y-direction; m is one of the total number of pixels M in the x direction; n is one of the total number N pixels in the y direction; a target It is the target graphic; and a cap The image is captured by the camera.

2. The method according to claim 1 further includes storing the phase value corresponding to the minimum loss function value.

3. The method of claim 1, further comprising determining, for a first pixel of the hologram displayed on the spatial light modulator SLM, an average contrast value of the image of the initial pattern captured by the camera.

4. The method according to claim 3, wherein, The average contrast value of the image of the initial graphic captured by the camera is determined using the following formula: in: It is the average intensity of the white portion of the image captured by the camera; It is the average intensity of the background portion of the image captured by the camera; and C is the average contrast value.

5. The method according to claim 4, wherein, The image is one of a plurality of images at different eye ellipse positions, and the image of the initial pattern captured using the camera in the vehicle includes the plurality of images captured at different eye ellipse positions.

6. The method of claim 5 further includes creating multiple lookup tables for phase values ​​at different eye ellipse positions.

Images