Image projection system and image projection method, content mapping module and content mapping method, and highlight projection system

Abstract

An image projection system and method are provided. The image projection system has a hybrid projector architecture that combines steered light with non-steered light, where the steered light is narrowband light and the non-steered light is broadband light. The division of the steered and non-steered light is determined based on brightness level. Effect: Non-steering light contributes the majority of the light for brightness levels up to a threshold. Steering light contributes an increasing portion of the light as the brightness level rises above the threshold.

Description

[Technical Field]

[0001] Cross-reference of related applications This application is based on the priority of U.S. application No. 62 / 505827, filed on May 12, 2017. This application asserts that, for the purposes of the United States, this application is incorporated herein by reference for all purposes. A US patent application filed on May 12, 2017, entitled "System and Method for High-Brightness Projection." Claiming benefits under Section 119 of the United States Patent Act in National Patent Application No. 62 / 505827 .

[0002] This invention relates to a digital image projector, such as a movie projector. [Background technology]

[0003] There is a need for a projector that can give viewers a realistic and / or engaging viewing experience. Yes, the viewing experience can be improved when high dynamic range (HDR) images are projected. Providing high maximum brightness can also improve the viewing experience. However, Even the latest movie projectors currently available still have in-scene contrast. It lacks strike, and more importantly, it lacks high peak brightness. Both of these are, These are important perceptual attributes that make an image appear realistic.

[0004] Various research concepts from this field have been incorporated into commercial display products. On the other hand, practical innovations in computer displays, mainly large-screen projectors... This has been pioneered. To show realistic images, although there is room for debate, displays The most important visual characteristics of the system are the range and number of light levels and colors that can be displayed. Mentally, significantly increasing this range in a projector is prohibitively expensive. Pea ak brightness is directly proportional to the power of the display and the cost of the light source, while the brightness perception of the brightness value is , almost logarithmic.

[0005] In some cinemas, the projection light is dispersed across a large screen. Even when the projector has a high light output, due to the large screen area, the achievable maximum brightness (full-screen white) is not that bright.

Summary of the Invention

[0006] Further aspects and exemplary embodiments are shown in the accompanying drawings and / or described in the following description.

Brief Description of the Drawings

[0007] The accompanying drawings illustrate embodiments of non-limiting examples of the present invention.

[0008] [Figure 1] The standardized movie SDR brightness range (yellow) on a logarithmic scale is compared to an example of the HDR brightness range achievable by a light steering projector architecture (green) to visualize the effect of approximate perceived brightness. [Figure 2] The theoretical and measured steering efficiency for a test pattern with varying average brightness is compared to a traditional projection system that forms images using amplitude modulation. [Figure 3] It is a fully light steering architecture with an overall system efficiency of 15 - 25%. [Figure 4] It is a hybrid architecture that increases the overall system light efficiency to reduce costs while achieving image quality comparable to a fully light steering projector. In this example, the light efficiency of the steering optical path is 13.5% and the light efficiency of the non-steering optical path is 31.5%. c [Figure 5]Figure 5A is a straight line showing the possible separation of input signals (black) between optical steering projectors (green) and non-optical steering projectors (red). Figure 5B is a logarithmic plot showing the possible separation of input signals (black) between optical steering projectors (green) and non-optical steering projectors (red). [Figure 6] Figures 6A to 6D are example images from the image set. [Figure 7] This represents the relative power required to reproduce each of the 104 HDR images on three virtual projectors. Two optically controlled projectors with peak brightness of 48 cd / m² (blue) and 480 cd / m² (green) are compared to a traditional light-stopping movie projector with a peak brightness of 48 cd / m² (red). The average power required to achieve the ideal peak brightness (48 cd / m²) is on the order of 13% of that of the traditional projector. Importantly, all images except the extremely brightest (approximately 9% of all tested images) can be reproduced up to a peak brightness of 480 cd / m² using the same or less power. [Figure 8] This is a photograph of the completed RGB prototype in Dark Labs. [Figure 9] This is a chromaticity diagram comparing the chromaticity of one example projector with several common color standards. [Figure 10] These are typical relative rise time (left) and fall time (right) for a phase modulator between 0 and 2π. [Figure 11] Figure 11A shows the subframe phase response at drive level 65 out of 255. Figure 11B shows the subframe phase response at drive level 75 out of 255. Figure 11C shows the subframe phase response at drive level 100 out of 255. [Figure 12] This demonstrates the high-level operating principle of DMD timing. [Figure 13] Figure 13A is a relative timing diagram showing the first frame of a pulsed light source operated by a DMD using slow asynchronous optical pulses. Figure 13B is a relative timing diagram showing the second frame of a pulsed light source operated by a DMD using slow asynchronous optical pulses. [Figure 14]Figure 14A is a relative timing diagram showing the first frame of a pulsed light source operated by a DMD using high-speed asynchronous optical pulses. Figure 14B is a relative timing diagram showing the second frame of a pulsed light source operated by a DMD using high-speed asynchronous optical pulses. [Figure 15] This is a relative timing diagram of a pulsed light source operated by a DMD using slow-speed synchronous optical pulses. [Figure 16] This is a relative timing diagram showing the combination of phase LCoS, DMD, and laser pulse. [Figure 17] This is a photograph of a prototype with a phase LCoS and a DMD amplitude modulator. [Figure 18] This flowchart illustrates the high-level optical blocks within the optical path of a prototype projector, from the laser light source to the projection lens. [Figure 19] This is an on-screen image of a laser light component that is not steered by a phase modulator. [Figure 20] This is an image on a screen with a full-screen white pattern. [Figure 21] This shows a high-level algorithm block for controlling the projector architecture. [Figure 22] This shows an algorithm block where the input conversion block acquires PQ-XYZ input, linearizes it, and converts it to the optical steering projector color space. [Figure 23] The content mapping algorithm block is shown. [Figure 24] This graph shows the steering and non-steering divisions as a function of the target brightness in cd / m2 units. [Figure 25] The forward model algorithm block is shown. [Figure 26] The point spread function applied to the test pattern is shown. [Figure 27] The phase pattern calculation block is shown. [Figure 28] The amplitude pattern generation block is shown. [Figure 29]Figure 29A shows the first frame from a movie displayed on the optical steering prototype (left) and on a conventional projector with the same power from the lens (right). Figure 29B shows the first frame from a movie displayed on the optical steering prototype (left) and on a conventional projector with the same power from the lens (right). [Modes for carrying out the invention]

[0009] Throughout the following description, specific details are provided to give a more thorough understanding of the present invention. However, the present invention can also be implemented without such details. In other examples... To avoid unnecessarily obscuring the present invention, well-known elements are either omitted or described in detail. Not included. Therefore, this specification and the drawings are illustrative rather than restrictive. That's what we should consider.

[0010] The present invention relates to an architecture, components, and method applicable to a digital projector. Regarding this, this detailed explanation is incorporated herein by reference for all purposes of the United States. MTT Innovations, Inc., whose patent application was published as Patent Application Publication No. 2017 / 0085846 The overall projector architecture described in U.S. Patent Application No. 15 / 368,021, which is owned by the company. Various aspects of the present invention will be explained in the context of the present invention. The technology described herein is HDR (High Dynamic Range). This is particularly beneficial for applications in (analytic range) projection systems. The technologies described are those in International Publication Nos. 2017 / 059537 and 2016 / 015163. Apparatus and techniques described in publications 2015 / 184549 and 2015054797 These may be used together. All of these are owned by MTT Innovations, Inc. All of these are incorporated here by reference for all purposes. These references The reference provides an example of a method of optical steering that may be applied in the context of this application.

[0011] In conventional (light-attenuating) projection systems, the illumination power is adjusted to achieve the desired peak brightness on the screen. While it is linearly scaled, the luminance value of the human visual system (HVS) Perception becomes nonlinear (close to logarithmic). U.S. Patent Application Publication No. 2017 / 0085846 The projector architecture described herein uses full-screen white (FSW) to display the selected image feature area. Light steering is used to enable reproduction at a significantly higher level than the luminance level. Figure 1 shows the cinema The current standardized cinema SDR luminance range is 10A (Digital Cinema Initiative). Refer to the DCI specification. Peak brightness white is 14 fL = 48 cd / m². 2 (as defined in) and One of the optical steering brightness ranges 10B achievable in a projector architecture using optical steering. Compare the examples using a logarithmic scale.

[0012] Image statistics and power requirements

[0013] Some embodiments of the disclosed technology use feature portions of projected image content, The goal is to reproduce high-brightness and high-contrast HDR content, and power and cost (light) This approach utilizes the idea that it is possible to achieve a trade-off with minimizing the source. Optical simulations and cost models of several projector architectures (here, optical (This will be referred to as a Light Budget Estimator (LBE)). Experiments were conducted using this in combination with statistical analysis of various representative theater HDR content. It has been done.

[0014] Optical steering efficiency

[0015] In an ideal optical projector, all available light is used to form an image. In such an architecture, the power required to form the image is the target image. Equivalent to the mean brightness or power of the image. In certain implementation examples, very small brightness The ability to steer light towards specific features is, for example, the point spread function of the system (point s It is limited by the pre-function (PSF). Figure 2 shows the source signal. In this context, optical steering projection has the same minimum and maximum brightness but different average brightness (power). Test patterns reproduced in the machine and in a traditional amplitude-attenuating projector The selection is shown. The solid curve represents the theoretically possible maximum peak brightness in an ideal optical steering projector. Degrees, that is,

number

[0016] Curve 11A shows the theoretically achievable luminance as a function of the non-black pattern area. Line 11B shows actual measurement data for the early optical steering prototype. What we can learn However, steering efficiency is not 100%. Non-steering light partially spills out into areas outside the projector screen. And it is scattered. Some of the non-steered light is present on the screen and increases the black level. The dashed line is, This shows the maximum brightness of a traditional projection system that uses amplitude modulation to form an image.

[0017] Component efficiency

[0018] Each component in the optical path of a projection system generally reduces the total optical throughput. There is a tendency for this to happen. For example, the total optical throughput achievable in a particular projector is phase The reflectivity of the modulator (often in the range of 65% to 80%), and any regularity in the optical path Loss due to higher-order diffraction effects from the pixel grid appearing on the constructed surface (40%~5 (Often within the 0% loss range), and also the relay lens, diffuser, and optical path are folded. Folding broadband mirrors, and dichroic mirrors that combine or split color channels, etc. It is affected by additional optical elements.

[0019] Optical steering requires additional optical components not found in conventional projectors. The presence of certain components can reduce light efficiency. (Prototype optical steering projector) The total optical efficiency was measured at approximately 5% from the source light to the screen. This inefficiency is partial Specifically, it balances out the large gain in peak brightness caused by optical steering, but especially high average image Images with image level (APL) are generally more expensive. Optimized system In the case of custom coatings for light source wavelengths, the overall efficiency is 15% to 25%. The height can be as high as 1%, which is approximately 35% of the light source relative to the full-image white test pattern. This is still low compared to traditional projector architectures where approximately 45% of the image can reach the screen.

[0020] Narrowband light source

[0021] In the new HDR projector architecture, optical steering efficiency is its highest peak brightness and Furthermore, it performs best for well collimated narrowband light. The laser diode is this It is well suitable for the application. However, when using a laser diode, how many There is a pitfall. This includes the following: • Cost: LEDs, illuminators, and other light sources such as laser + phosphor systems are currently more expensive. In that respect, it is cost-effective in terms of output lumens per dollar. • Observer metamerism: The same measurable chromaticity coordinates composed of narrowband light sources differ in their appearance. It is perceived differently by different observers. This effect is less pronounced for non-narrowband light. It's inconspicuous. • Screen speckle: Small-scale interference patterns from the laser light appear as disturbances on the projected screen. It can cause small spatial intensity fluctuations. This effect is invisible to broadband light. be.

[0022] Some of these negative effects associated with using a laser light source are due to some of the laser's properties. To break the structure (for example, by introducing a large angular diversity), or to change the light from the laser This can be addressed by either combining it with a non-laser light source. Interesting. In particular, many of the overall bright scenes are adapted to the brighter parts of the screen by the human visual system. This does not necessarily mean that a low black level is required.

[0023] Full optical steering architecture

[0024] Figure 3 is a representation of the specifications described in U.S. Patent Application Publication No. 2017 / 0085846. This document demonstrates a high-level system architecture for a tactile projector. Full optical steering architecture 20 In this configuration, the control firmware 22 controls the optical steering module 24 and the projector head 26. Control. The optical steering module controls the projector head 26 to generate the final HDR output image 30. This generates a steering light image 28 that is projected onto the screen. The control firmware 22, for example, The optical steering module 24 may include an image processing unit and a system control unit. For example, it may include one or more RGB laser banks, in free space or fiber architecture It can operate using a choke. The projector head 26 is often used in cinema or consumer projectors. In particular, detail layers including 4K resolution, HFR, and 3D effects may be added. This full optical operation The rudder architecture uses currently available optical components to achieve approximately 15% to 25% composite It has measurable system efficiency.

[0025] Hybrid optical steering architecture

[0026] Figure 4 shows a high-level illustration of an example hybrid projector architecture 32. Projector 32 In order to further reduce costs and mitigate certain image artifacts, Bandband light for steering is combined with broadband non-steering light (which uniformly illuminates the amplitude modulator). This forms a single system. This hybrid architecture allows for a full optical steering projector. While achieving comparable image quality, the increased overall system light efficiency resulted in reduced performance. It can be obtained through strikes.

[0027] In the hybrid projector 32, the control firmware 22 controls the base attenuation light source 34, The optical steering module 24 and the projector head 26 are controlled. The optical steering module 24 controls the steering light. Image 28 is generated, and the steering light image 28 is the base attenuation light image generated by the base attenuation light source 34. It can be combined with 36.

[0028] In one example embodiment, the resulting image is composed of 30% steering light and 70% base light. It may include attenuated light. In one example embodiment, the optical steering module 24 has an efficiency η = 0.4 The projector head has an efficiency η=0.45. The combined image is projected to the projector. The final HDR output image 30 is produced by projection by the head 26. In this example, The optical efficiency of the steering light path is 13.5%, while the optical efficiency of the non-steering light path is 31.5%.

[0029] Hardware parameters include the division of source light between the steering and non-steering parts of the system. This may include the type of light source, and associated costs and spectral characteristics.

[0030] The software parameters drive the system between steering and non-steering stages. This may include multiple aspects of the algorithm used to allocate light. Figures 5A and 5B are shown below. This shows an example of splitting a linear input signal between a classical projector optical path and an optical steering path. Figure 5 shows Possible separation between the optical steering projector 40B and the non-optical steering projector 40C of the input signal 40A. A linear plot is shown. Figure 5B shows the optical steering projector 42B and non-optical steering projector with input signal 42A. A logarithmic plot of similar possible separations with respect to 42C is shown.

[0031] By applying perceptually meaningful image quality metrics, the desired color and intensity of the target image are determined. It is possible to determine how faithfully it has been reproduced.

[0032] In one exemplary embodiment, the hybrid projector includes a non-steered projector and an optically steered projector. A two-projector system may be used. A non-steering projector may use, for example, a commercially available projector that uses amplitude modulation. A projection device may be included. The optical steering projector may have, for example, a phase / amplitude modulation design.

[0033] This section describes non-theater HDR image data (mapped to an approximate brightness range). Based on the data and a simplified system model of the full-light steering projector, the image statistics were collected. He states.

[0034] Average brightness of HDR images in cinema

[0035] Publicly available high-brightness H-rated displays with color grading for theater viewing environments. DR video content is almost nonexistent. This is partly due to the large screen with sufficient capabilities. This is due to the fact that no projection system currently exists. In this section, we It uses color-graded HDR still images to achieve the current peak brightness in cinema. Estimating the relative power required to reproduce HDR brightness levels up to 10 times higher An attempt was made. Analysis of 104 HDR images was performed, and in the proposed architecture, such as Power requirements for the optical steering projector were estimated. In this theoretical exercise, traditional Light-controlled projectors with lower power than cinema projectors were among those surveyed, with a power output of 48 cd / m². 2 Images up to this point All, and 480 cd / m² 2 Almost all HDR images up to this point have additional tone compression. It was found that it could be reproduced directly without the need for [something else]. The results are summarized in Table 1. Mel. [Table 1]

[0036] methodology

[0037] 104 Scene Reference HDR Images of Mark Fairchild (Mark D Fairchild. The e hdr photographic survey. In Color and Imaging Conference, pages 233-238. Socie The study (from *ty for Imaging Science and Technology*, 2007) was analyzed. Figures 6A-6D are from this paper. These are example images from the image set used in the survey. These images have a dynamic range. Less than 1000:1 to 10 9 :Differs by up to 1. Most images are outdoor scenes. The image data represents the measured scene-reference HDR (actual scene brightness level), and the cinema It is not intended for viewing on a projector, and the initial rendering is appropriate for cinema. The period estimate is that APL will approximately match the estimated audience adaptation level in cinema. This can be established by shifting the image intensity. Simple linear scaling G operator S adaptation However, this was determined manually for each image. The image was 48c d / m 2 Calibrated 27 inches set to peak white brightness (D65 white point) Inch reference monitor (using Photo Research's PR-650 spectroradiometer) Calibration was confirmed in Dell's U2713HMt, and in a dark viewing environment, Dynamic tuning was performed. While adjusting the intensity, the image was viewed from a distance of approximately 3-5 screen heights. It was heard.

[0038] Once the appropriate luminance scaling factor is determined, it becomes 10 times the FSW, i.e., 480 cd / m 2 Brightness levels exceeding this were clipped. Next, the proposed projector architecture Steering efficiency is approximated by the system PSF (in this case, the actual number of horizontal image pixels out of 1920). (Qualitatively, it is explained via a somewhat conservative, large Gaussian kernel spanning 81 pixels.) has been clarified. The average intensity across all pixels of the obtained luminance profile serves as an approximate metric for the power requirements of the light steering projector. plays a role as an approximate metric for the power requirements of the light steering projector.

[0039] <00003९9>Calculation steps

[0040]

Number

[0041] Results

[0042] Figure 7 shows the estimation of the power required to reproduce each HDR image in a light steering projector with the same peak luminance of 48 cd / m m 2 as a traditional cinema projector and a light steering projector with a peak luminance one order of magnitude greater than the cinema reference system, at 480 cd / m m 2 In the light steering architecture, only a small portion (13%) of the power of a traditional projector is used to reproduce all images. Importantly, almost all images can be reproduced at up to 480 cd / m 2 (10 times higher peak luminance) using less power than a traditional projector. In Figure 7, the power required for a traditional projector to generate up to 48 cd / m is shown by line 44A of the traditional projector, the power required for the steering projector to generate up to 48 cd / m m 2 is shown by line 44B of the low - luminance steering projector, and the power required for the steering projector to generate up to 480 cd / m

[0043] In Figure 7, the power required for a traditional projector to generate up to 48 cd / m 2 is shown by line 44A of the traditional projector, the power required for the steering projector to generate up to 48 cd / m is shown by line 44B of the low - luminance steering projector, and the power required for the steering projector to generate up to 480 cd / m 2 is shown by line 44C of the low - luminance steering projector. In Figure 7, the power required for a traditional projector to generate up to is shown by line 44A of the traditional projector, the power required for the steering projector to generate up to 48 cd / m 2 is shown by line 44B of the low - luminance steering projector, and the power required for the steering projector to generate up to 480 cd / m is shown by line 44C of the low - luminance steering projector.

[0044] Using the scaling and clipping operations described above, further artistic color correction can be performed. If not available, the mean image level (APL) of the dataset is used for cinema-ready high-luminance HDR content. It appears higher than what could be predicted from the content. Certain HDR content is significantly lower. It may have an APR (for example, approximately 3% or less). Projecting HDR content with a low APR. The power requirements for the optical steering projector architecture to be used are as follows: It can also be considerably lower (or have high peak brightness and contrast).

[0045] Improved RGB projector prototype

[0046] The improved full-color RGB projector prototype incorporates the features listed in Table 2. These features can be applied individually or in any combination. The details of the features are not intended to be limiting. Each of the listed features is subject to various possible substitutions. It can be implemented using any of the components. For example, the prototype described In the project, LCoS-based SLMs are used as amplitude SLMs, while for the purpose of amplitude modulation... as transmissive or reflective SLM, liquid crystal display panel or digital micromirror device ( Other SLMs such as DMD can also be used. Examples of techniques suitable for phase modulation or amplitude modulation are: “Rolf R Hainich and Oliver Bimber. Displays: fundamentals & applications.CRC pr ess, 2016; and David Armitage, Ian Underwood, and Shin-Tson Wu. Introduction to This is described in "microdisplays, volume 11. John Wiley & Sons, 2006." It is incorporated herein by reference for any purpose. [Table 2]

[0047] Architecture of an RGB projector prototype

[0048] Figure 8 is a photograph of the completed RGB prototype projector that implements the features from Table 2. Mounted on top is a mechanical vision camera for color calibration. And spot spectroradiometer.

[0049] Color

[0050] Using a light source that includes red, green, and blue laser diodes, we have installed a full-color projector. It can be measured. The prototype embodiment is 462nm, 520nm and 638nm It uses a native laser diode that emits light of wavelength m. Prototype implementation. The system has a parallel architecture. Here, the light source, phase modulator and optical components are used. Three monochromatic light paths, including one, are combined to form a white beam, which is then sent to an amplitude-modulated projection head. It is relayed. In other embodiments, a field sequential system (for example, red, A system in which green and blue light fields are displayed sequentially may be used. Further implementations In this state, parallel or sequential architectures use more or fewer optical wavelengths than this. It is acceptable. In some embodiments, light of four or more wavelengths is generated in parallel, and These will be displayed sequentially. In addition to narrowband light sources, the architecture of several embodiments is broadband. And / or may include a full-spectrum light source.

[0051] The achievable color range of this prototype is compared to other common displays and cinemas. This can be seen in the chromaticity diagram of Figure 9 for primary colors. u' and v' are used as chromaticity coordinates. CIE1976 Uniform Chromaticity Scale le(UCS))46A gives a perceptually uniform relationship between individual colors (i.e., mutually). (The perceived difference between colors at equidistant from each other is of the same magnitude.) Starred data point 46B This represents the D65 white point, which is common in cinema and home display systems. The proposed chromaticity space 46B, which can be reproduced using the typeface, is the DCI P3 space 46C It is significantly larger than both Rec and includes both. This is Rec.2020 color sky. It almost completely includes the interval 46E and exceeds its area.

[0052] Light power

[0053] The light power of the prototype proof-of-concept projector was measured as 10 lumens outside the lens. They used a projector with a high light source power to increase the brightness outside the lens, thereby increasing the brightness of the steering light. It can be used as a target for new prototypes, with a range of 100-200 lumens. The steering light represents a meaningful foreshadowing. However, the required light source power is not available in existing radar systems. This cannot be easily achieved with the diode. The three laser diode characteristics are It is noteworthy that the total power of the laser diodes, the dimensions of the light-emitting element, and the amount of light emitted are all important. This is optical divergence. It involves beam expansion and collimation of multiple laser diodes in a light source. The optics and tilt can be adjusted by individual components, some with 6 degrees of freedom. It can be implemented using mechanical design.

[0054] Speckle

[0055] Similar to displays based on narrowband or monochromatic light sources (such as LEDs or lasers), To manage inter-observer metamerism and undesirable characteristics such as speckle Caution is needed. Typically, this reduces the visible speckle contrast for the observer. There are three measures to achieve this. Namely, making the polarization random and in the optical path This involves increasing the angular diversity of light and expanding the light source spectrum. Although not the main focus of this book, elements of one or all of these three methods are described here. This can be applied to projectors.

[0056] The projection head used in the RGB prototype is an LC that requires linear polarization input. Based on OS technology, this polarization is used to form the final image either in front of or behind the projection lens. It is possible in principle to randomize the data after it has been processed. Similarly, optical steering is possible for DMD systems. When coupled to the camcorder head, the polarization follows the phase modulation and precedes the DMD amplitude modulation. It can be randomized within the projector's optical path. DMD requires linear polarization in the input. Because it does not. The light in our optical steering system is ideally sufficiently collimated. This is converted to an optical system with a high f-number (f#), and such a limited angular die It involves varsity. Such advanced collimation is necessary for the optical steering portion of the system. While it's important to retain this optical property, it's no longer necessary after the intermediate image has been formed. .

[0057] Subsequently, f# is reduced, for example, using an optical shaping diffuser, and the optical instrument It can be adjusted to the input angle tolerance. This results in high angular diversity of the beam. The visible spectrum decreases as the light is moved. Because the angle of expansion is randomized over time, visible speckle contrast is further reduced. This can be achieved, for example, by rotating a diffuser disc inside the projector, or By linearly displacing the optical elements in or near the diffuser, It can be achieved.

[0058] A second effective way to reduce speckle contrast is to introduce motion into the projected image. Therefore, slight continuous displacement of the screen surface means that the light from the projector lens is not on the (non-flat) screen surface. As it is reflected, it has the effect of being averaged over many angles. One effective technique, though not very practical in a cinematic setting, is the movement of the observer.

[0059] Binning and calibration including laser diodes with different central wavelengths for each color By using a modified laser diode, the effective spectral bandwidth of the light source can be widened. This allows for a reduction in visible speckle contrast. By spreading the light source... Furthermore, after light manipulation, a superimposed set of slightly enlarged and reduced images is applied to each color channel. This results in a blurred effect. This can be modeled by a small blur kernel, so it is not necessarily required. It's not that it's undesirable.

[0060] synchronization

[0061] Phase SLM and amplitude SLM, as well as pulse width modulation (PWM) dimmable laser light sources, Ideally, synchronization should be required at the frame or subframe level. The amplitude modulator originates from consumer projectors and is located within the built-in image processing block that we will describe. This exhibits undesirable latency in multiframes. In cinema, high projector light output is required. In order to handle and comply with prescribed standards (e.g., DCI), binary DMD is primarily It is used as amplitude SLM. The following section on temporary considerations describes the expected temporary A new drive ski designed to reduce artifacts (e.g., flicker). Let's discuss timing.

[0062] Time considerations in HDR projection displays

[0063] Some of the prototypes were built on consumer hardware, but customer By using the development kit together with the modified light source control electronics, pulses from the light source can be controlled. Digital pulse codes (subf) are necessary to deal with laser intensity modulation (S) and phase modulators. The bit plane within the frame, and, for example, the projector head Good synchronization of the binary primary amplitude modulator (DMD mirror state) is permitted. This section explores different solutions for synchronous drive schemes. This section describes synchronization options for the projector architecture.

[0064] LCoS-based phase modulator

[0065] The phase modulator we selected for the implementation of the optical steering projector is currently digital. LCoS microdisplay with backplane and no input or output polarizing filter Use this. The backplane is updated in a manner that scrolls vertically up and down. Once Either one or two lines are updated. The relatively slow response of the liquid crystal material allows for efficient near-analog phase response of the display. This makes it possible to achieve this. This is advantageous. High speed of micromirror projector heads Synchronization to a binary state is necessary for truly binary high-speed LCoS devices (e.g., ferroelectric devices). This is because it has fewer problems compared to (S).

[0066] The following are important for our application: • Overall refresh rate of the phase modulator (the refresh rate is the overall refresh rate of the projector) (This must be compatible with the required video frame rate.) • Phase accuracy over a certain frame period (the phase modulator operates based on the corresponding drive level) (To what extent can the given phase value be reproduced?) • Phase stability within a given frame (overall phase response (phase flicker), To what extent can individual digital pulses from the cuplene be measured, and is this light How does this affect steering? • Phase drift within subframes between each line update.

[0067] In a calibrated LCoS device, the liquid crystal for each pixel is subjected to an electric field. When it is added or removed, it drifts toward 2π or 0π, respectively. The response time is This can be considered as the time from when the electric field is applied to when the liquid crystal undergoes a 2π phase shift. This is possible. The response time can be tuned according to the chemical formula of the liquid crystal.

[0068] Figure 10 shows the rise and fall characteristics of a phase-only microdisplay. An example is shown. Figure 10 shows a typical relative rise time (left) for a phase modulator from 0 to 2π. This shows the falling (right) time. If the target phase response is between 0 and 2π, the intermediate phase delay is reached. The desired voltage can be periodically applied and removed.

[0069] The circuit that generates the periodic voltage enables the spatial resolution and bit precision of the microdisplay's phase control. It has a frequency governed by the degree. The liquid crystal response time is determined by the update frequency given from the drive circuit. It is tuned to provide the most stable image possible when available.

[0070] The drive circuit used in the current prototype has a drive on / off voltage at 7kHz. This FPGA is capable of changing its state. In this case, the pixel array starts from the top. This is updated line by line. Therefore, it is a single bitplane across the entire frame. Updating takes 1 / 7,000 of a second (i.e., approximately 145,000 nanoseconds).

[0071] An example of the variability in phase stability and phase drift of the current system is shown in Figures 11A to 11C. This can be seen in the measurement. Here, the liquid crystal alignment is determined by the application of a periodic square wave voltage. This allows it to be balanced in a half-rotation state.

[0072] Figures 11A, 11B, and 11C show drive levels of 65 (left) and 75 (center) out of 255. The subframe phase response at ) and 100 (right) is shown. We have shown the temporal response on a small scale. Ripple is referred to as phase flicker, and deviation around the mean is referred to as phase stability. Let's assume that.

[0073] The total rise response time is 8.7 milliseconds, and the total fall response time is currently It is 21 milliseconds. The cell thickness of a particular panel under test is suitable for the application. It is not stabilized, and the entire light spectrum, including the infrared (IR) portion, is sufficient. This introduces a phase delay. Therefore, the cell thickness is thicker than necessary. For this particular panel example... For blue laser diodes, the maximum possible phase delay is 6 nanometers, and for red lasers... It can approach 3 nanometers. Fast response time reduces cell thickness and maximizes the required position per wavelength. This can be achieved by ensuring that the phase delay is never exceeded. The function that provides a steer effect allows for a simple adjustment of the effective refresh rate of the phase modulator. It can and should be explained through a model. A new framework arrives. When this happens, the drive state of all pixels is updated in the next refresh cycle.

[0074] Fast response time and high phase stability, when viewed along one shared dimension of time control, are These are somewhat conflicting goals. This is because, within the duration of a video frame, a continuous electric field is By applying it initially, it is permissible for the liquid crystal to move quickly to the correct position (after that, The electric field is removed or sometimes reduced to just pulses, while the phase stability across the entire video frame is maintained. Qualitative analysis compares the on and off states of the electric field over the entire duration of the video frame. This is because it is best achieved by spreading it evenly to the target. Both goals are, In this regard, it can be explained in the lens effect algorithm, and the phase code word The digital digits that form the basis of pulse code modulation (PCM), which is used to map to the optical phase response. This can also be explained in terms of a barrel renewal scheme.

[0075] The functionality provided by the FPGA in the current prototype system is also a dedicated AS This can be provided by an IC. An ASIC can provide a high update rate (e.g., 15kHz). This can be done. In the phase stability plot in Figure 11, twice the number of peaks (P) can be observed within the same time period. A quay and a valley can be seen.

[0076] In addition, to enable faster response times (e.g., a twofold improvement), the current phase modulators... There are options to modify or redefine the birefringence properties of the liquid crystal (LC) materials used. do.

[0077] Characteristics of DLP technology

[0078] DLP technology uses micromirrors for each pixel to flip diagonally back and forth. It utilizes an Inari modulator. Each mirror is either on, directing light rays towards the screen, or off, at any given time. It can be either in an off state, or directed towards a surface area, the so-called light damping area. - generates grayscale by rapidly flipping back and forth. For example, video Over the entire process of rendering, more bright pixels are rendered when the setting is on. Time is spent on this, and for dark pixels, more time is spent in the off state.

[0079] Typically, each pixel is per video frame (usually 60 frames per second (fps)). It is controlled by an 8-bit (or more) grayscale drive value. Figure 12 shows this Here is an example of how such grayscale can be converted to a mirror flip. Figure 12 shows the high-level operating principle (conceptual bit in one frame for 8-bit color). DMD as a representation of a partition (reproduced from the T1 DLP documentation) This is a diagram illustrating the timing.

[0080] Whether a bit is set to 0 or 1 determines whether the mirror flips to the ON position. Determines whether to be switched on or flipped to the off position. The bit position determines whether the mirror is in that state. Determines the relative duration that remains constant. Mirror's typically achievable flips per second. The maximum number is just below 10kHz (see Figure 12). Professional applications (For this, up to 80kHz has been reported), therefore, regarding this estimate, We set the shortest duration of the Ra state to 0.1 milliseconds, or 100,000 nanoseconds. This is referred to as the mirror-flip period, which is equal to the period of b0 in the diagram in Figure 12. Let's do it this way.

[0081] Asynchronous light pulse

[0082] Where pulsed light sources are used (for example, to generate light at 50 percent of the maximum level) For example, due to low pulse frequencies in the light state, the off and on pulses cause mirror frizz. If the program is asynchronous and the off and on periods differ significantly for each frame of the still image, Flickering occurs.

[0083] Figures 13A and 13B show DMD operation using a pulsed light source with slow asynchronous optical pulses. This is a relative timing diagram of the two frames. In Figure 13A, if the signals are asynchronous Due to this fact, how does the light turn on for 2 / 5 of the time in frame 1? Note whether the light is on for 3 / 5 of the time in frame 2 of Figure 13B. And if the on-light period is shorter than the mirror flip period, the off period between still frames The difference between the on-time and off-time should be dramatically reduced and imperceptible to the human eye.

[0084] Figures 14A and 14B show that the light source is modulated significantly faster than the shortest possible mirror flip time. This shows the relative timing diagram of two frames of DMD operation using a pulsed light source.

[0085] In Figures 14A and 14B, only one minimum width mirror flip with a drive value of 1 is shown. Please note the following point. The light state is similar to the PWM clock described below. In this example, the light is on 27 / 54 in frame 1 and 28 / 5 in frame 2. Please also note that it is set to 4 (on).

[0086] Synchronized light pulse

[0087] The light source off period and the light source on period are synchronized with the mirror flip (as shown in Figure 15). If this is the case, there should be virtually no difference in intensity between still frames, so the light source pulse The device only needs to operate during the mirror flip period, and consequently, the control solution Requirements and considerations for electromagnetic interference (EMI) can be dramatically reduced. Figure 15 shows low-speed synchronization. Relative timing diagram for one frame of DMD operation using a pulsed light source that uses optical pulses. That is the case.

[0088] When a new frame arrives, the mirror-flip logic for all pixels is: Via a double buffering scheme (or, if desired, in a block from top to bottom) (They can be updated simultaneously.)

[0089] Laser control solutions

[0090] We are using iC-Haus iC-HG to directly drive laser diodes with high current. iC Haus GmbH. ic-haus homepage - product: ic-hg. http: / / ichaus.de / HG, 20 17) was selected, and the option to operate pulses at ultra-high frequencies was chosen. This device It has switching capability up to 200MHz from differential pair inputs. 500mW (optical power 638nm laser diode (power 650mA and 2V per diode) Synchronized switching of the array up to 100MHz was confirmed. Constant power supply to iC-HG The current limit for the ON state is set by the input voltage. We are a high-speed digital-to-analog converter. A constant voltage and current input was driven by a (DAC).

[0091] A combination of LCoS phase modulator, binary DMD-based SLM, and pulsed laser.

[0092] Figure 16 shows the relative timing diagrams for combinations of phase LCoS, DMD, and laser pulses. Yes. This figure shows the shortest possible time for a DMD mirror flip, and for illustrative purposes, it is different from the actual time. Includes reduced light source pulses and expected phase drift of LCoS-based phase modulators. The following assumptions were used for the first estimation of the system time response: • Approximately 100 optical states and PWM clock pulses exist for the shortest mirror flip duration. (Shown in smaller quantities in Figure 16 for clarity), the LCoS phase error is, in this example, It drifts by an average of 1.5 times between 0.1π and -0.1π. • These visibilitys are in line with the high-speed asynchronous PWM clock drive scheme introduced as described above. The upgrade includes a (compared to the prototype) twice as fast phase LCoS SLM and a high-speed controller chip. ASIC was used.

[0093] Table 3 shows typical update rates for different modulation elements within the projector and the resulting performance. This indicates the duration of the effect. [Table 3]

[0094] An error (drift) of 0.1π in phase modulation is the maximum possible phase delay exceeding 2π. Relatively low compared to the quantity, the high-speed laser light source is the fastest DMD mirror flip number. It also has no effect. Many alternative implementations include continuous wave (CW) or always-on laser drivers. This is possible. Here, excess light is directed away from the active image area. .

[0095] DMD System Experimental Prototype

[0096] An LCoS-based phase modulator, a PWM-dimmable laser light source (on the order of kHz), and a DMD are used. An experimental prototype, which combines elements according to an asynchronous driving scheme, was built for demonstration purposes. Although direct timing measurements were not performed, visible time artifacts were noticeable enough. It did not become dominant. The relatively low amplitude of the phase error, and the maximum of the phase flicker. A lower amplitude relative to 2π helps mask potentially present time artifacts. It happened.

[0097] Figure 17 is a photograph of a prototype with a phase LCoS and DMD amplitude modulator. The ether is coupled to the transport modulator via fiber (red, left side of the enclosure), and The intensity is adjusted via a PWM drive scheme.

[0098] Projector colorimetric calibration

[0099] In a full-color system, colorimetric calibration is particularly important for systems including PSF. This requires both marking and precise modeling of the light source and optical path. Because it depends on [something], it may potentially also depend on the position or the size of the image feature area.

[0100] Optical steering image formation model

[0101] In the operation from the light source to the screen, the collimated laser light is transmitted through a series of optical instruments. The signal is then relayed from the light source to the reflection phase modulator. From the phase modulator for a given color channel The departing light is combined with the light from the remaining channels and filtered through the diffuser. The light is relayed to a prism or RGB prism. The prism separates the light and outputs its multiple components. The component color is then modulated by an amplitude modulator and then prism. They are recombined internally and directed towards the primary projection lens.

[0102] Figure 18 shows the high-level optical path of the prototype projector 50 from the laser light source to the projection lens. A light block is drawn. Lasers 52A, 52B, and 52C generate laser light, and that laser The light is directed to pass through phase light modulators 54A, 54B, and 54C, respectively. (Example) In this embodiment, lasers 52A, 52B and / or 52C are provided by laser diodes It is acceptable to give it that way. For example, laser 52A is a laser diode that produces a wavelength of 462 nm. Laser 52B is a laser diode that generates a wavelength of 520 nm, and laser 5 2C can be used as a laser diode that generates a wavelength of 638 nm.

[0103] After passing through the phase light modulators 54A, 54B, and 54C, the light from the three optical paths becomes combined light. After being coupled in the equipment block 56, it is diffused in the diffuser 60. The light passes through the RGB prism 60, and the RGB prism 60 divides the light, It uses multiple component wavelengths. Each separate optical path uses a corresponding spatial light modulator (SLM62). It is directed towards A, 62B, and 62C respectively. After modulation by the spatial light modulator, the light is directed towards R The signals are recombined in the GB prism, directed into lens 64, and then projected onto screen 66. The combination of RGB light to produce white light was chosen for convenience, and prior to Intended for compatibility with commercially available projection hardware with aligned SLMs. Separate RGB A well-designed projector that implements an optical path can be expected to have good contrast performance. .

[0104] According to this design, the phase modulator 54 can introduce a phase shift to the incident wavefront. Therefore, it is possible to generate local intensity that is well above typical full-screen intensity. The phase modulator 54 is a programmable lens that responds to a software-driven phase pattern. It can function as follows: The phase pattern is derived from the input illumination profile to the target light profile. It is calculated to attempt to redistribute light to the file, and ideally in the target image below. The upper envelope of intensity is selected to approximate the dark area. Due to the identification of a specific type of image, light tends to be redistributed into brighter areas that have a wide, dimly lit region. With its placement and small, bright highlights, it achieves a level of 10 that can only be achieved with conventional projectors. This results in a significant degree of focus from the preliminary light, which can reach more than double the level of focus.

[0105] Because phase modulators introduce a certain number of artifacts, diffusers are incorporated into optical designs. It is incorporated to reduce speckle and also acts as a low-pass filter for the light irradiation field. It works. This artifact is due to the fixed texture and the unsteering lighting component. , including diffraction artifacts. The diffuser includes diffraction artifacts and fixed textures. While effective in removing this, it generally cannot compensate for un-steered glare (see Figure 19). ).

[0106] Figure 19 shows the unsteering component of the prototype laser beam. The component consists of laser light that is not steered by a phase modulator. The amount of light from the Nent is determined by the illumination that enters the phase modulator after filtering by the diffuser. Related. Typically, about 10% of the light incident on the phase modulator is directed to the unoperated component. Yes.

[0107] As a result, the un-steered components become an important contributor to subsequent image formation. Measurement of unsteering components involves steering all available light off-screen using phase shifting. This is achieved by designing the turn. The remainder are the unsteering components (Figure 19). .

[0108] For laser diode systems like the RGB prototype, the unsteering component The Nent shows the individual diode beams reflected from the phase modulator. This is shown in Figure 19. Here, vertical stripes (red) and horizontal stripes (green, blue) are shown. The difference in orientation is in the diode. This is due to the different polarization orientations. Since the beams from the light source are polarized in the same direction, all The stripes are oriented similarly according to the design. In the case of fiber-coupled lasers, the unsteering component The pattern becomes significantly more uniform. Figure 20 shows the resulting full-screen white pattern. show.

[0109] Optical model

[0110] This section describes the algorithms used to drive the system. We start with the high-level algorithm block for typical algorithms. The main block is This is further described in a specialized subsection. Each rectangular block corresponds to a set of operations. The parallelograms indicate data passed between blocks. The solid arrows indicate known phases between blocks. While interactions are shown, dashed arrows indicate optional relationships.

[0111] High-level algorithms

[0112] Figure 21 is a flowchart of the high-level method 70. Method 70 takes in the input image 72. The input image 72 is input to the transformation block 74 which generates a linear input 76. Force 76 is manipulated by content mapping algorithm block 78. The content mapping algorithm embodied by K78 is based on the target light illumination field 80, and the eye This generates the target image 82 and the input for the power control 98.

[0113] The target light irradiation field 80 is determined by the forward model algorithm 84 and the phase pattern calculation block. It is affected by the action of 88. The forward model algorithm 84 predicts the light irradiation field 86. is generated. The predicted light irradiation field 86 aims at recalculating the target light irradiation field 80 and the target image 82, and is fed back as an input to the content mapping algorithm 78 in an iterative process. .

[0114] Finally, the predicted light irradiation field 86 is passed to the amplitude pattern generation block 90, and the amplitude pattern generation block 90 further takes in the target image 82 and generates the amplitude pattern LS94 and the amplitude pattern NS96. The phase pattern calculation block 88 generates the phase pattern 92. In the RGB prototype system, this input includes values in the CIE1931 XYZ color space by perceptual quantization ( PQ) encoding for each channel.

[0115] In the input conversion block, the image data content is linearized and converted into the working color space of the system. The output of the input conversion is currently a linear image represented in linear RGB laser primaries. <00​​​​​​​​​​​​​​​​​​​​It is used as an input to the phase modulator. This is necessary to affect the optical redistribution by the phase modulator. The required drive parameters are calculated. In addition, the target light irradiation field is determined by the forward model. It is also used by Gorism Block. This is the feedfor of the optical steering image formation model. Implement word simulation. In practice, the phase modulator and the subsequent optical path are arbitrary. This is because it is not possible to accurately reproduce the target light field. The forward model procedure is Combined with the target image, the predicted light illumination field is used by the amplitude pattern generation block. An image is generated. This allows the necessary amplitude patterns to be generated for both optical steering and non-optical steering blocks. The line is decided.

[0118] Input conversion

[0119] The input conversion block primarily converts the input image to the color space in which the input is represented (e.g., XYZ color space). It converts the PQ encoded image (in this context) to a color space defined by laser primary colors. The precise conversion used depends on the format of the input image data.

[0120] Figure 22 shows that the input conversion block 74 acquires the PQ-XYZ input image 72 and linearizes it. In step 102, the algorithm linearizes the image and generates a linear XYZ output 104. The color space is then displayed. Subsequently, the linear XYZ output is converted to the color space of the optical steering projector by color space conversion 106. In between, it is converted to a linear output 76.

[0121] The forward and inverse transforms for PQ encoding are given by the following equations.

number

[0122] The primary colors and white point (D65) of the RGB projector for color conversion are shown in Table 4.

Table 4

[0123] To obtain an RGB image in the laser primary colors from these, one of them can be converted to the primary color of the RGB projector (MTTP3). This conversion is selected to maintain the luminance of each channel, and the following relationship is obtained between the XYZ images.

Equation

[0124] The luminance value per channel is used to define the conversion M from MTTP3 to XYZ. This transformation can be defined as follows:

number

number

[0125] Regarding the chromaticity and white point mentioned above, this gives the following results for M.

number

[0126] Similarly, the reverse mapping from XYZ to MTTP3 can be expressed as follows:

number

[0127] Content Mapping

[0128] Content mapping block 78 takes a linear input image as input and performs optical steering projection. Determine the division between the machine and the non-optical steering projector, and the required power level.

[0129] Figure 23 shows an example of an application that may be used to implement content mapping block 78. The details of algorithm 78A are shown below. Algorithm 78A first obtains a linear input 76 and then... Check whether the input image is feasible under the stem power budget conditions. This is the current This is done using a power heuristic. If No, the input is tone. Mapping 114 is performed. Tone mapping 114 includes, for example, simple scaling. Hmm. However, more sophisticated tone mapping algorithms may also be applied. As a result... The resulting target image 82 (which is either passed through or tone-mapped) is then... It will continue to be the target light field for the hand stage. For feasible input content Therefore, the linear input image and the target image are identical. The target image 82 is then the target light illumination field. It is used to generate image 80. The target image 82 is between the non-steering system and the steering system. It is divided in between. The division function 116 provides information for the target light irradiation field 80 and power control 98. Generate information.

[0130] The division between steering and non-steering is, for example, raising the target image to an index (γ>1) for optical steering. This can be achieved by determining the rudder image. This involves correcting the optical steering projector data. It doesn't exactly reproduce it. The content is destroyed in order to emphasize the highlights. Therefore, accurately (and optically efficient) dividing is essential for most image formation. Lighting level (e.g., 48 cd / m²) 2 ) Until then, non-steered projectors were used, and the lighting level increased. This is achieved by gradually fading in the optical steering projector. Figure 24 shows an example of the division between steering and non-steering, in cd / m 2 As a function of the target brightness of the unit This is the graph. Note that it uses a logarithmic-logarithmic scale.

[0131] In the example embodiment shown in Figure 24, the splitting function 116 initiates the handover to the steering projector. The point is 47 cd / m². 2 For 90% of the image formation up to this point, non-steered projectors were used. Attempting to use. Bright image with a painted-on appearance. To avoid highlighting specific areas, a steering projector is used for a portion of the image at every pixel. It is desirable to do so. The partition is 1D and may be implemented as a single function or as a LUT.

[0132] Forward Model

[0133] Figure 25 shows a possible application for implementing the forward model algorithm block 84. This is a diagram of algorithm 84A. The forward model block 84 takes the input as... The target light field 80 is obtained from the content mapping block 78 and used for light mapping. The output of the stem is predicted. This is referred to as the predicted light irradiation field 86.

[0134] Calculating the predicted light field of view 86 is advantageous. If the entire target light field of view is achievable... This is not the case. The predicted light irradiation field 86 is used to calculate the accurate amplitude pattern. It can be used. The predicted light irradiation field 86 is compared to the target light irradiation field 80 with the actual light irradiation field. It is possible to determine how much the field differs from the target light illumination field. Algorithm 84A is , Calibrated system PSF122 and actual light field (steering) 124, And the unsteering component 126, while taking care to explain the overall power level. This includes applying it to the output image.

[0135] The forward model takes the target light field 80 as input and blurs it with a diffuser. To predict the actual light field results afterward, the PSF122 system was set to a target light field of 80. Apply. An example of a PSF for an RGB projector system is shown in Figure 26, arranged in a 4x4 pattern. The PSF for the red, green, and blue channels is of different sizes. and / or may have a shape.

[0136] In this case, the effect of the unsteering component 126 is added to the resulting light irradiation field. This is added after blurring. The measurement of this image was taken after passing through the diffuser. This is because it can only be achieved once that happens. In the current system, the fixed pattern is highly non-uniform. Therefore, in fiber-coupled systems, it can be approximated by a Gaussian distribution.

[0137] Phase pattern calculation

[0138] The phase pattern generation block 88 generates the phase pattern necessary to achieve the target light irradiation field. Calculate. Figure 27 shows an example of an application that may be used to give a phase pattern calculation block 88. This shows Lugorhythm 88A.

[0139] To accurately frame the image on the amplitude modulator and separate high diffraction orders, target illumination It is desirable to preprocess the 80th channel. This is intended to align the three channels. Includes warping 132 with the calibrated strain shown in the figure. For example, dimension W × Each point [x,y] in the target image of H is expressed by a 2D cubic polynomial, and the point [x m ,y m It can be mapped to ].

number

[0140] The source image then corresponds to [x] for each target pixel x,y. m ,ym In ] linear It can be pumped. Target coordinates ([x n ,y n By normalizing the coordinates, the source image and the eye Mapping can be calculated even with resolution mismatches with the target image. 10×1 FIT β parameter vector x and β y However, it is obtained from calibration. Once When sampling and resampling are performed, the resulting image will be (depending on the optical configuration) (Then) a cyclic shift 134 is performed, and the appropriate range (such as [0.001, 1000.0] is selected. The ramp 136 is applied. At this point, the phase calculation algorithm 142 is applied. Then, The final phase pattern is mapped to the output range of the phase panel.

[0141] Amplitude pattern analysis

[0142] The amplitude pattern generation block 90 uses the target image and the predicted light field as input. This determines the amplitude patterns for the steering projector and the non-steering projector. Figure 28 shows the vibration This shows an example algorithm 90A given for implementing the width pattern generation block 90. .

[0143] Algorithm 90A first adds the non-steering illumination 152 to the predicted light field 86. This is, This represents the total amount of light available on the screen. The target image 82 is used to illustrate the predicted light field 86. The common amplitude pattern 160 is then adjusted to both the optical steering and non-optical steering amplitude modulators. The calculation is performed in relation to the effective range of transmittance (e.g., [0,1]). Clamped 156 (or tone map to preserve texture in out-of-range areas) (This may be applied.) Any necessary LUT can then be applied to describe the response of the amplitude SLM. 58 The pattern is then sent directly to the projection head. Steering projector and non-steering projector To spatially align the shadow machine, the alignment warping 162 is calibrated. It can be used based on the pre-aligned pixel-compatible section. Block 162 can use the same cubic warping function as the phase pattern generation block. This results in optical steering (LS) amplitude pattern 94 and non-steering (NS) amplitude pattern 9 You get 6.

[0144] result

[0145] Figures 29A and 29B show a side-by-side comparison of the cinema projector and the prototype optical steering projector. The photograph shows that both are introduced in this section by the same light power outside the lens. Play back the video processed using the Lugorism framework. Figures 29A and 29B These are, respectively, a prototype of optical steering using the same light power outside the lens (left) and a traditional projector. (Right) shows a frame from the movie displayed. The optical steering projector (left) is a comparative projector. The contrast and peak brightness are significantly exceeded by approximately 20 times (Optical steering projector: 1, 000 cd / m 2 Right projector: 48 cd / m² 2 ).

[0146] Interpretation of terms

[0147] Throughout this specification and the claims, it is not required that the context is clearly otherwise. as long as • "Include," "equip," etc., in contrast to their exclusive or exhaustive meanings, have a comprehensive meaning. This should be interpreted as meaning "includes, but is not limited to," • "Connected," "joined," or any variation thereof refers to a direct or interlocking relationship between two or more elements. This means any indirect connection or link, where the connection or link between the elements is physical or logical. Often, as a target, or a combination thereof, • The words "here," "above," "below," and similar terms are used in this specification. When used, it should refer to the entire Specified Specification, and not to any specific Specified Specification. It should not be mentioned in part, • When referring to a list of two or more items, "or" and "or else" are used in the following ways: All of the interpretations, that is, any item in the list, all items in the list, and the It covers any combination of items in the list, The singular forms "one," "one," and "that," "this," and "that" are also appropriate. It also includes the meaning of the plural form.

[0148] As used herein and in any appended claims, “vertical,” “horizontal,” and “horizontal” are used in this specification and in any appended claims. "Upward", "Downward", "Front", "Forward", "Backward", "Backward", "Inward" "Outside", "vertical", "crossing", "left", "right", "front", "back", "peak", "bottom", Words indicating direction, such as "down," "up," and "down," (if they exist) are indicated and illustrated. It depends on the specific orientation of the device. The subject matter described herein may assume various alternative orientations. Therefore, these terms are not strictly defined and should be interpreted narrowly. isn't it.

[0149] Embodiments of the present invention include specifically designed hardware, configurable hardware, and data Software that can run on the processor (optionally may include "firmware"). A programmable data processor provided by, or as described in detail herein. One or more steps in the method and / or a combination of two or more of these steps A dedicated computer that is specifically programmed, configured, or constructed This can be implemented using a data processor. Specifically designed hardware Examples include logic circuits, application-specific integrated circuits ("ASICs"), and large-scale integrated circuits ("LSIs"). Examples of configurable hardware include, Programmable Array Logic ("PAL"), Programmable Logic Array ("PLA") ), one or more programmed fields such as Field Programmable Gate Arrays ("FPGAs") It is a programmable logic device. An example of a programmable data processor is a microprocessor. S, digital signal processors ("DSPs"), embedded processors, graphics processors Mathematical coprocessor, general-purpose computer, server computer, cloud computer These include mainframe computers, computer workstations, etc. For example, One or more data processors in the projector's control circuit are accessible to the processor The method described herein is implemented by executing software instructions in program memory. It is possible.

[0150] Processing may be centralized or distributed. If processing is distributed, the software and / or data Information containing data may be centrally stored or distributed. Such information may be stored in a local area network. Network (LAN), Wide Area Network (WAN), or Internet Uncommunication networks, wired or wireless data links, electromagnetic signals, or other data communications It can be exchanged between different functional units via a communication channel.

[0151] For example, a process or block is presented in a given order, but alternative examples are presented in a different order. A routine with steps can be performed, or a system with blocks can be used. Some processes or blocks are removed to provide alternatives or subcombinations. They can be moved, added, subdivided, combined, and / or modified. These processes or Each block can be implemented in various different ways. Also, processes or Blocks are sometimes shown to occur in series, but these processes or blocks Alternatively, the actions can be performed in parallel or at different times.

[0152] In addition, elements may sometimes be shown to be executed sequentially, but instead, simultaneously Alternatively, they may be performed in a different order. Therefore, the following claims are intended to be within the scope of the intended scope. It is intended to be interpreted as including all such variations within the enclosed text.

[0153] The software and other modules are for servers, workstations, and personal computers. Computers, tablet computers, image data encoders, image data decoders, video Projectors, audiovisual receivers, displays (like televisions), digital cinema It resides in projectors, media players, and other devices suitable for the purposes described herein. That's fine.

[0154] The present invention may also be given in the form of a program product. The program product is a data product. When executed by a losser, the computer causes the data processor to perform the method of the present invention. The present invention may include any non-temporary medium that carries a set of readable instructions. RAM products can take any of the many different forms. Program products, for example, are floppy disks. Magnetic data storage media including P-diskettes, hard disk drives, and CD-ROMs Includes optical data storage media, DVDs, ROMs, and flash RAM. , EPROM, wired or pre-programmed chip (e.g., EEPROM semiconductor chip), This may include non-temporary media such as technology memory. The data-readable signal may optionally be compressed or encrypted.

[0155] In some embodiments, the present invention may be implemented in software. And, “software” includes any instructions executed on the processor, firmware, This may include, but is not limited to, resident software, microcode, etc. Processing hardware Both the air and the software are, as is known to those skilled in the art, collected in whole or in part. It may be centralized or distributed (or a combination thereof). For example, software and other modules. The `resource` is used via local memory, via networks, and in the context of distributed computing. via a browser or other application, or other suitable for the purposes described above. It can be accessed through various means.

[0156] Components (e.g., software modules, processors, assemblies, optical modulators) If lenses, projector heads, prisms, devices, circuits, etc. are referred to above, then so Unless otherwise indicated, references to that component (including references to “means”) , as an equivalent of that component, any component that performs the function of the described component It should be interpreted as including components (i.e., functionally equivalent). Figure of the present invention The structure of the disclosure performing the function in the typical embodiment shown is not structurally equivalent. It also includes components.

[0157] Specific examples of systems, methods, and apparatus have been described herein for illustrative purposes only. These are multiple These are merely examples. The techniques presented here are not suitable for systems other than those described in the examples above. It can also be used. Within the implementation of the present invention, many substitutions, modifications, additions, omissions, and replacements are possible. The present invention includes modifications of the embodiments described. These modifications are for those skilled in the art. It is clear that the features, elements and / or processes are equivalent to the features, elements and / or processes. Substitution, mixing and matching features, elements and / or processes from different embodiments To modify the features, elements, and / or processes from the embodiments described herein, other technologies Combining with the features, elements and / or processes, and / or from the embodiments described Includes variations obtained by omitting and combining the features, elements, and / or processes. nothing.

[0158] Various features are described here as being present in "several embodiments." These features are not essential and are not present in all embodiments. Embodiments of the present invention are This may include zero, any one, or any combination of two or more such features. This is limited only to the extent that a particular feature of such a feature becomes incompatible with other features of such a feature. It will be done. A person skilled in the art will be able to construct a practical embodiment that combines such incompatible features. This means that it is impossible to do so. Therefore, "some embodiments" possess feature A. Furthermore, the statement that "some embodiments" possess feature B is an inaccuracy of the inventor (the statement Unless it is stated that this is not the case, or unless it is stated that feature A and feature B are not fundamentally incompatible, (To the extent that) this can be interpreted as an explicit indication that embodiments combining Feature A and Feature B are also intended. It should be done.

[0159] Therefore, the claims attached below and any claims introduced thereafter are: All such modifications, substitutions, additions, omissions and subcombinations that can be reasonably inferred It is intended to be interpreted as including the claims. The claims are preferably described in multiple examples. It should not be limited by any particular embodiment, but rather the broadest interpretation that matches the description as a whole. It should be given.

Claims

1. 1. A hybrid light steering projector, comprising: a baseline path including a base light source operable to generate a base light; a highlight path including a light steering module operable to generate a steering light image with a steering light; a projector head operable to combine the base light and the steered light to illuminate a first spatial light modulator to generate an output image; a controller operable to process image data including pixel values ​​defining target illumination levels for pixels of a target image to be displayed, and to generate control signals for the first spatial light modulator and the light steering module; Including, the processing includes determining an illumination level for the baseline path and a per-pixel illumination level for the highlight path and dividing light output between the baseline path and the highlight path such that the base light provides a majority of the light for the pixels of the target image up to a first threshold illumination level and the steering light provides an additional portion of the light above the first threshold illumination level for each of the pixels of the target image; a hybrid light steering projector, wherein the processing further includes dividing the target image into a light steered image portion for illumination with the steering light and a light non-steered image portion for illumination with the base light, the division of the target image being based on a function or a lookup table.

2. The controller generating a modified target image by a process including determining the pixel values ​​defining the target image as γ powers of the pixel values, γ>1; basing a control signal for the light steering module on the corrected target image; 10. The hybrid light steering projector of claim 1 configured to:

3. 3. The hybrid light steering projector of claim 2, wherein the controller is configured to maintain a fixed ratio of the luminance of the steering light to the luminance of the base light for pixels having a target illumination level below the first threshold illumination level.

4. 4. The hybrid light steering projector of claim 3, wherein the controller is configured to control the highlight path such that, for pixels having a target illumination level that exceeds the first threshold illumination level, a ratio of the steered light to the base light exceeds the fixed ratio.

5. 5. The hybrid light steering projector of claim 3 or 4, wherein the fixed ratio is less than 20%.

6. 5. The hybrid light steering projector of claim 1, wherein the maximum achievable illumination level by the highlight path for any pixel of the steering light is at least 10 times greater than the maximum achievable illumination level by the baseline path for any pixel of the image.

7. the controller includes the lookup table; 5. The hybrid light steering projector of claim 1, wherein the controller is configured to divide the light output based on an output from the look-up table.

8. the controller is configured to check to determine whether display of the target image is feasible; 5. The hybrid light steering projector of claim 1, wherein if the check determines that displaying the target image is not feasible, the controller is configured to tone map the image data to obtain tone mapped image data and continue processing with the tone mapped image data.

9. The hybrid light steering projector of claim 1 , wherein the highlight path includes a narrowband light source.

10. 10. The hybrid light steering projector of claim 9, wherein the narrowband light source comprises one or more laser diodes.

11. The hybrid light steering projector of claim 10 , wherein the narrowband light source includes a red light emitting diode, a green light emitting diode, and a blue light emitting diode.

12. 12. The hybrid light steering projector of claim 11, wherein the light steering module includes a plurality of phase modulators, and the light emitted by the red light emitting diode, the green light emitting diode, and the blue light emitting diode is steered by a first phase modulator, a second phase modulator, and a third phase modulator of the plurality of phase modulators, respectively.

13. the controller implements a forward model that calculates a predicted light field for light steered by the light steering module; 13. The hybrid light steering projector of claim 11 or 12, wherein the forward model includes separate point spread functions for light emitted by the red, green, and blue light emitting diodes.

14. The hybrid light steering projector of claim 13 , wherein the forward model is iterated.

15. The hybrid light steering projector of claim 13 , wherein the forward model models a steered component of light from the light steering module and an unsteered component of light from the light steering module.

16. 1. A method for projecting an image, comprising: generating a base light using a baseline path including a base light source; generating a steering light image with a steering light using a highlight path including a light steering module; combining the base light and the steering light to illuminate a first spatial light modulator to generate an output image; processing image data including pixel values ​​defining target illumination levels for pixels of a target image to be displayed, said processing including determining illumination levels for said baseline path and per-pixel illumination levels for said highlight path and dividing light output between said baseline path and said highlight path such that said base light provides a majority of light for said pixels of said target image up to a first threshold illumination level and said steering light provides an additional portion of said light above said first threshold illumination level for each of said pixels of said target image; generating a control signal for the light steering module based on the determined amount of steering light for a pixel of the target image; Including, the processing further includes dividing the target image into a light-steered image portion for illumination with the steering light and a light-unsteered image portion for illumination with the base light, the division of the target image being based on a function or a lookup table.

17. generating a modified target image by a process including determining the pixel values ​​defining the target image as γ powers of the pixel values, γ>1; basing a control signal for the light steering module of the highlight path on the modified target image; The image projection method of claim 16, comprising:

18. 17. The method of claim 16, wherein the segmentation is performed pixel by pixel based on the target illumination level.

19. 20. The method of claim 17, further comprising: for pixels having a target illumination level below the first threshold illumination level, maintaining a fixed ratio of the steering light luminance to the base light luminance.

20. 20. The method of claim 19, further comprising controlling the highlight path so that for pixels having a target illumination level that exceeds the first threshold illumination level, a ratio of the steering light to the base light exceeds the fixed ratio.

21. 21. The image projection method according to claim 19 or 20, wherein the fixed ratio is less than 20%.

22. 21. The image projection method of claim 16, wherein the maximum achievable illumination level by the highlight path for any pixel of the steering light is at least 10 times greater than the maximum achievable illumination level by the baseline path for any pixel of the image.

23. 21. The method of any one of claims 16 to 20, further comprising dividing the light output based on an output from the look-up table.

24. 21. A method for projecting an image according to any one of claims 16 to 20, comprising: performing a check to determine whether displaying the target image is feasible; and if the check determines that displaying the target image is not feasible, tone mapping the image data to obtain tone mapped image data; and continuing processing with the tone mapped image data.

25. 21. The method of any one of claims 16 to 20, wherein the highlight path includes a narrowband light source.

26. the narrowband light source includes one or more laser diodes; 26. The method of claim 25, comprising operating the laser diode to illuminate one or more phase modulators of the light steering module.

27. 27. The image projection method of claim 26, wherein the narrowband light source includes a red light emitting diode, a green light emitting diode, and a blue light emitting diode.

28. 28. The image projection method of claim 27, comprising steering light emitted by the red light emitting diode, the green light emitting diode, and the blue light emitting diode using a first phase modulator, a second phase modulator, and a third phase modulator, respectively, of the plurality of phase modulators of the light steering module.

29. implementing a forward model to calculate a predicted light field for light being steered by the light steering module; 29. The method of claim 27 or 28, wherein the forward model includes separate point spread functions for light emitted by the red, green and blue light emitting diodes.

30. 30. The image projection method of claim 29, wherein the forward model is a recursive model.

31. 30. The image projection method of claim 29, wherein the forward model models a steered component of light from the light steering module and an unsteered component of light from the light steering module.

32. a content mapping module, a controller configured to receive image data defining a target image as an input image, and determine a light power allocation between steering light and non-steering light based on the input image and corresponding power levels of the steering light and the non-steering light for the target image to be displayed, the controller configured to generate control signals for a light steering module of a projector based on the light power allocation; the controller is configured to allocate the light output such that the unsteered light provides a majority of the light for pixels of the target image up to a threshold illumination level, and the steered light provides a further portion of the light above the threshold illumination level for each of the pixels of the target image.

33. 33. The content mapping module of claim 32, wherein the controller is configured to determine whether it is feasible to provide a desired amount of steering light and non-steering light based on the input image, and to tone map the input image if it is not feasible to provide the desired amount of steering light and non-steering light.

34. The controller generating a modified target image by a process including determining pixel values ​​defining said target image as γ powers of said pixel values, γ>1; basing said steering light splitting on said corrected target image; 33. The content mapping module of claim 32 configured to:

35. 33. The content mapping module of claim 32, wherein the controller is configured to make the allocation on a pixel-by-pixel basis of the target image.

36. 33. The content mapping module of claim 32, wherein the controller is configured to output a target light field and information for a power controller.

37. 33. The content mapping module of claim 32, wherein the input image is a linearized image.

38. 1. A method of content mapping, comprising: determining an allocation of light power between steering lights and non-steering lights based on input image data defining a target image and corresponding power levels of the steering lights and non-steering lights for the target image to be displayed; allocating the steering light and the non-steering light such that the non-steering light provides a majority of the light for pixels of the target image up to a threshold illumination level, and the steering light further provides a portion of the light above the threshold illumination level for each of the pixels of the target image; Including, A portion of the steering light illuminates each of the pixels of the target image.

39. 39. The method of claim 38, comprising determining whether it is feasible to provide a desired amount of steering light and non-steering light based on the input image, and tone mapping the input image if it is not feasible to provide the desired amount of steering light and non-steering light.

40. generating a modified target image by determining pixel values ​​defining said target image as γ powers of said pixel values, γ>1; basing said steering light splitting on said corrected target image; 39. The method of claim 38, comprising:

41. 39. The method of claim 38, comprising performing the allocation on a pixel-by-pixel basis of the target image.

42. 40. The method of claim 38, including outputting the target light field and information for a power controller.

43. 39. The method of claim 38, comprising linearizing the input image.

44. 39. The method of claim 38, wherein the input image is a linearized image.

45. 1. A highlight projection system, comprising: a highlight projector including a light steering module operable to project a steering light onto a screen; Controller and Including, The controller processing image data including pixel values ​​defining target illumination levels for pixels of the target image; determining a target contribution to the target illumination level for the highlight projector such that splitting light output between the steering light and non-steering light, the non-steering light providing a majority of the light for the pixels of the target image up to a threshold illumination level, and the steering light further providing a portion of the light above the threshold illumination level for each of the pixels of the target image; generating a control signal for the light steering module based on the target contribution; and and The target contribution includes a light contribution to each of the pixels of the target image.

46. a content mapping module including a controller, The controller processing image data including pixel values ​​defining target illumination levels for pixels of the target image; dividing light output between steering light and non-steering light such that the non-steering light provides a majority of the light for the pixels of the target image up to a threshold illumination level, and the steering light also provides a portion of the light above the threshold illumination level for each of the pixels of the target image; and determining a target contribution to the target illumination level for the steering light; generating a control signal for a light steering module based on the target contribution; and and The target contribution includes a light contribution to each of the pixels of the target image.

Images