Projector System

Abstract

To display an image with better contrast and a better dynamic range in a multiple projector system. A light projection system and method includes combining light from two or more projectors, each of which is controlled so that the combined light output of the projectors matches a target for the projected light, and optimization is performed to generate image data and control signals for each of the projectors.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority from U.S. Application No. 61 / 994002, filed May 15, 2014, and U.S. Application No. 62 / 148041, filed April 15, 2015. For U.S. purposes, this application claims the benefit under 35 U.S.C. § 119 to U.S. Application No. 61 / 994002, entitled "BRIGHTNESS BOOSTER FOR MULTIPLE-STAGE PROJECTORS," filed May 15, 2014, and U.S. Application No. 62 / 148041, entitled "OPTIMIZING DRIVE SCHEMES FOR MULTIPLE PROJECTOR SYSTEMS," filed April 15, 2015, both of which are hereby incorporated by reference herein for all purposes. [Field]

[0002] IMAGE PROJECTOR AND METHOD FOR PROJECTING IMAGES The present invention has applications in, for example, cinema projection, projection television, advertising displays, general lighting such as spatially adaptive automobile headlights, and the like. [Background technology]

[0003] Many optical projectors have a light source that uniformly illuminates an imaging chip such as a DMD, LCoS, LCD, or reflective LCD (or film) that subtractively modulates the incident light to generate a target image. Such projectors typically 1) cannot exceed a peak brightness set by the optical power of the light source, the size of the projected image, and the reflectivity of the image screen, and 2) have a dynamic range or contrast limited by the imaging device, e.g., film, or a digital device such as an LCD, LCoS, or DMD imaging chip.

[0004] Light projectors differ in their ability to generate target images with specific luminance and chromaticity values. The range of capabilities comes from technical limitations related to the maximum peak luminance (light output of the light source) to the lowest black level and therefore contrast (contrast of the imaging technology involved), color purity and color gamut (controlled by filters applied to either broadband light sources or, for example, the wavelengths of laser light sources), as well as uniformity and noise specifications. Some projectors may generate a light output with limited contrast, for example reaching a peak luminance of 100 cd / m2 and a black level of 1 cd / m2, and thus a contrast of 100:1. Other projectors may reach brighter highlights (by increasing the light source power) and / or deeper black levels (with higher contrast imaging techniques). In some systems, very deep black levels may be achieved by modulating the image twice ("double modulation"). The contrast or dynamic range of the projector may be dynamically adjusted by inserting a stop or aperture in the light path, the light blocking of which may be triggered in response to the image content.

[0005] The type and requirements of the image or video content reproduced on the projector may change considerably over time during the course of the presentation of the image or video content, including, for example, the presentation of a movie in a cinema, a live performance using a projector, or the projection of light by adaptive (image-) projector headlights while driving in different conditions in a vehicle. For example, a movie may start with a dark, high-contrast, black and white scene, and later include bright and low-contrast scenes with multiple pure colors. While driving at night, adaptive car headlights may need to project a uniform, low-contrast light field on an empty road outside a city, but within the city they may need to generate very high-contrast and bright images to highlight stop signs, illuminate approaching cars (casting shadows in the area), or avoid signaling information on the road.

[0006] High brightness, high dynamic range projectors are often more expensive than standard lower dynamic range projectors for similar average light (power) output. One reason for this is that achieving better black levels often requires more elements in the system (e.g., dual modulation designs with cascaded light attenuation elements). Another reason is that achieving higher peak brightness on the same screen requires more light source power in the projector.

[0007] As in the above example, there remains a need for good methods of controlling projection systems to reproduce image content that has multiple characteristics that vary significantly over time (e.g., characteristics such as dynamic range, black level, maximum brightness, color saturation, etc.) Such methods would beneficially provide advantages such as reducing power requirements, providing good black levels, and / or providing bright highlights.

[0008] There remains a need for optical projection systems that provide higher image quality and / or better cost-effectiveness.

[0009] There remains a need for practical, cost-effective projection systems suitable for projecting patterns such as images, desired lamp illumination patterns, etc. There is a particular need for such systems that are capable of accurately displaying content that has characteristics that change significantly over time (e.g., a system that can be called upon to display a bright, low-contrast image at some times and a dark image with bright highlights at other times). Summary of the Invention

[0010] The present invention has a number of aspects. One aspect provides a projector system that combines multiple projectors. The projectors may have different performance characteristics. The projectors may be separate devices or may share certain components, such as control electronics or certain optics. Another aspect provides a control hardware device useful for coordinating the operation of two or more projectors to display an image. Another aspect provides a method for splitting an incoming image signal into multiple separate images.

[0011] Multiple image generating devices may be used to form a composite image. Each device has a set of operating specifications (which may include, for example, specifications for peak brightness, resolution, black level, contrast, color range or color gamut, etc.). The defined mathematical functions provide image quality and cost metrics in a mathematical framework that allows optimization to achieve goals such as improved image quality or lower cost. The optimization results in separate image data for each image generating device.

[0012] This concept can be applied to a projector where two or more systems with similar or different capabilities generate a composite image according to image data.

[0013] In the presence of a low dynamic range projector in an installation, or if it is not possible to find a high dynamic range projector with suitable maximum output power, it may be desirable to combine two or more projectors with similar or different capabilities to produce a single image with high peak brightness and low black level. Examples of such configurations include a low dynamic range projector and a high dynamic range projector that produce a single image with high peak brightness and low black level.

[0014] Further aspects and example embodiments are illustrated in the accompanying drawings and / or described in the following description. [Brief description of the drawings]

[0015] The accompanying drawings illustrate several non-limiting exemplary embodiments of the present invention.

[0016] [Figure 1] FIG. 1 is a block diagram illustrating a projection system in accordance with an exemplary embodiment.

[0017] [Figure 2A] 1 is an exemplary image. [Figure 2B] 2B is an image projected by an LDR projector that may be synthesized to reproduce the image of FIG. 2A. [Figure 2C] 2B is an image projected by an HDR projector that may be synthesized to recreate the image of FIG. 2A.

[0018] [Figure 3A] 1 is another example image. [Figure 3B] 3B is an image projected by an LDR projector that may be synthesized to recreate the image of FIG. 3A. [Figure 3C] 3B are images projected by an HDR projector that may be combined to recreate the image of FIG. 3A.

[0019] [Figure 4A] 1 is another example image. [Figure 4B] 4B is an image projected by an LDR projector that may be synthesized to reproduce the image of FIG. 4A. [Figure 4C] 4B is an image projected by an HDR projector that may be synthesized to recreate the image of FIG. 4A.

[0020] [Figure 5A] 1 is another example image. [Figure 5B]5B is an image projected by an LDR projector that may be synthesized to reproduce the image of FIG. 5A. [Figure 5C] 5B is an image projected by an HDR projector that may be combined to recreate the image of FIG. 5A.

[0021] [Figure 6A] 1 is another example image. [Figure 6B] 6B is an image projected by an LDR projector that may be synthesized to recreate the image of FIG. 6A. [Figure 6C] 6B is an image projected by an HDR projector that may be combined to recreate the image of FIG. 6A.

[0022] [Figure 7] 1 is a schematic illustration of an abstract concept of a display;

[0023] [Figure 8] Shows two displays working in series.

[0024] [Figure 9] Shows two displays working in parallel.

[0025] [Figure 10] FIG. 1 is a block diagram illustrating an exemplary composite display.

[0026] [Figure 11] FIG. 1 is a block diagram illustrating a system in which display parameter optimization is performed to determine the parameters and lighting to be used to reproduce an input target image using a display.

[0027] [Figure 12] 11 is a flow chart illustrating the combining of images from a first and second projector to produce an output image.

[0028] [Figure 13]4 is a flow chart illustrating a method for determining which images are likely to be shown by each of a plurality of projectors to result in a target image.

[0029] [Figure 14] FIG. 1 is a block diagram showing a projection system having separate primary and secondary light sources ("boost light sources") and two imaging elements that can direct or attenuate light onto a screen.

[0030] [Figure 15] 4 is a flow chart illustrating how to control a light source in a projection system having a primary and secondary (boost) light source.

[0031] [Figure 16] 1 shows exemplary image data with different image characteristics and corresponding intensity settings (control signals) for a supplemental (boost) light source. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0032] Throughout the following description, specific details are set forth to provide a more thorough understanding of the invention. However, the invention may be practiced without these details. In other instances, well-known elements have not been shown or described in detail to avoid unnecessarily obscuring the invention. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense.

[0033] One motivation for combining two or more low dynamic range projectors (projector tiling), or even two low peak brightness high contrast (dynamic range) projectors, is to boost the overall on-screen luminance (brightness) of the resulting image. Low dynamic range projectors are common and a commodity technology, and therefore command a much lower purchase price than a high dynamic range projector of similar total output brightness.

[0034] FIG. 1 shows a schematic diagram of a projector system including multiple projectors.

[0035] In some embodiments, all of the multiple projectors provide light to the same viewing area (e.g., the boundaries of the projectors' fields of view may be the same). Each of the multiple projectors may transmit light to any portion of the viewing area. The viewer sees the combined output of the projectors. In some embodiments, each of the multiple projectors projects over the entire display area of ​​the viewing screen.

[0036] In a system that combines low and high dynamic range projectors (LDR and HDR), the optimal ratio of light contributed to the final image by each of the multiple projectors can vary significantly. This variation is due to: Ambient light level at the screen Image Peak Brightness Image average brightness Light output of both projectors · Efficiency (lumens / watt) of both projectors Minimum black level for LDR projectors The amount of black in the image · Proximity of black to bright features (veil luminance) The presence of non-uniformities (or other artifacts) in an LDR projector that can be corrected by an HDR projector The presence of speckle (or other artifacts) in HDR projectors that can be reduced by using an LDR projector Ability to reduce projector power consumption by showing dimmer content (for power consumption optimization) This is a result of image and environment characteristics such as:

[0037] Below are five example cases that show how images from HDR and LDR projectors can be combined according to an exemplary embodiment of the present invention. "Bright" and "dim" refer to the brightness levels of the images. Case 1: Bright low dynamic range image, enhanced black levels

[0038] The image (FIG. 2A) has a high black level. Darker details are closely surrounded by white features. In this exemplary case, the desired brightness of the image exceeds the capabilities of the LDR projector.

[0039] An LDR projector may be controlled to output as much light as possible (see FIG. 2B), while an HDR projector may be controlled to capture some of the brighter features to simply increase the overall brightness of the image as shown in FIG. 2C. Case 2: Dim, low dynamic range image, high black levels

[0040] This image (Figure 3A) does not have a very high dynamic range. The LDR projector is bright enough to produce an image at the desired level. In this case, the LDR projector may simply show the input image "as is" (Figure 3B) and the HDR projector may output nothing or be off (Figure 3C). Case 3: Bright high dynamic range image, high black levels

[0041] This image (FIG. 4A) shows some detail in the darker areas, so the image does not have a very low black level. The brighter parts of the image exceed the brightness capabilities of the LDR projector. The LDR projector may display the image as shown in FIG. 4B, and the HDR projector may display the image as shown in FIG. 4C. Case 4: Bright high dynamic range image, low black levels

[0042] This image (FIG. 5A) has a very low black level with a complete lack of detail in the dark areas. Due to the high expected brightness of the candle flame, the LDR projector may be turned off all together or dimmed by using an aperture (FIG. 5B), and the HDR projector may generate the entire image (FIG. 5C). Case 5: Dim, low dynamic range image, low black levels

[0043] Here, the peak brightness of the image is very low (see FIG. 6A) and at the same time the black level is also very low. The LDR projector requires an aperture on the lens (more details below) to reduce the black level sufficiently. In this case, the peak brightness through a partially closed aperture is sufficient to display the image, so an HDR projector would not be needed. FIG. 6B shows the image output by the LDR projector with the aperture partially closed. FIG. 6C shows the (black / null) output of the HDR projector. Aperture / Overall Lamp Power Control

[0044] Low dynamic range projectors often produce dark gray images when attempting to show black due to limitations in light modulator technology. As an example, consider images in which the brightest areas have a brightness lower than the peak brightness of the projector. Herein, better contrast can be achieved by dimming the light source. In another example, it may be determined that the amount of detail in the dark areas of the target image is of higher importance to the viewer. In such a case, bright content may be sacrificed by dimming the projector to regain deeper black levels. Most low dynamic range projectors are lamp-based and cannot be easily dimmed or switched on and off (to produce pure black) on a scene-by-scene basis due to warm-up challenges.

[0045] When a low dynamic range projector needs to be turned "off" or simply turned down, an aperture can be placed in the light path (e.g., on the lens). The aperture can then be made smaller to improve the black level of the projected image. Also, note that the aperture is not binary. The aperture can be opened to a size determined by the desired image black level. It is envisioned that the aperture can change size fast enough so as not to create noticeable lag when changing scenes. The aperture function can also be implemented by some other electrical or mechanical means, such as an LCD board (electrically dimmable) or a fast shutter that closes and opens quickly.

[0046] LDR projectors have solid-state light sources with light output that can be controlled, and an aperture may not be necessary. In such embodiments, the light source may be dimmed an amount such that its light output is equivalent to the light available through a constrained aperture.

[0047] High dynamic range projectors may optionally include globally dimmable solid-state light sources and / or irises. Artifact Reduction

[0048] It may be beneficial for image quality to never completely close the aperture and accept a slightly higher black level: if the HDR projector exhibits poor image quality due to field non-uniformity or other artifacts, having at least a baseline amount of light from the LDR projector can help perceptually reduce the artifacts.

[0049] If an LDR projector displays image artifacts such as vignetting or other non-uniformities, an HDR projector can be used to correct the non-uniformity of the light field. Projector Balancing Algorithm Display expression

[0050] To determine the settings for each component projector, the capabilities of each projector may be taken into account.

[0051] Previous approaches generally model imaging as a simple pipeline where each component takes an input, operates on it, and passes it on to the next stage. This approach works well for systems that consist of a relatively small number of controllable elements, e.g., light sources, modulators, or irises, combined with a relatively large number of passive optical elements, such as mirrors or lenses. However, it is less desirable for more complex systems. Such systems may combine multiple displays (projectors), or feed the output of one display into a subsequent display. In this case, parameters of later stages in the pipeline may be adjusted to compensate for artifacts or performance limitations of earlier stages.

[0052] It is useful to think of each display in an abstract sense as taking a set of display parameters P (e.g., pixel values), and source illumination S, which are then manipulated by the display to produce an output image O=F(P,S), where the function F models the behavior of the particular display hardware. This abstract view of a display is illustrated in FIG.

[0053] This modular approach allows multiple displays to be connected almost arbitrarily to form a network of abstract displays and passive optical elements to model more complex imaging systems. Multiple displays in a network can be connected either in series to form a single optical path, in parallel to combine multiple optical paths, or in a combination of series and parallel designs.

[0054] An example of a series connection of two displays is shown in Figure 8 for a system with two amplitude modulators connected in series. Such a configuration is used in some extended dynamic range (EDR) projectors that compensate for the limited contrast ratio of the individual amplitude modulators by cascading multiple modulators. The output contrast is then the product of the contrast ratios of the two modulators.

[0055] An example of a parallel arrangement can be found in projector super-resolution applications, where output images from multiple projectors are overlapped with slight deregistration to generate higher spatial frequency features present in the image from a single projector. This arrangement is shown in Figure 9.

[0056] In a parallel arrangement, the optical paths of two amplitude modulating projectors are combined (by a projection screen) to produce an output image.

[0057] Based on the arrangement, the output image can be mathematically determined by either adding or combining the images produced by the multiple component displays. Taking as an example two displays with functions F1 and F2 taking parameters P1 and P2 respectively, the parallel configuration results in the following expression for the output image:

number

number

[0058] It is also possible to form a composite display by placing arbitrarily many displays in a network and taking the component display parameters and the light source illumination as input to the composite display. An example of a parallel configuration is shown in FIG.

[0059] A composite display may then be represented as a particular type of abstract display, which may then be arranged into a network and / or grouped to form a higher level composite display. Provided that the component display imaging models, Fi, are known, a mathematical imaging model of the overall display system may be represented via serial and parallel combinations. Such an imaging model may be applied to optimize the operation of the display system. Display Parameter Optimization

[0060] One advantage of this representation is that once an overall imaging model of the display system is defined, optimal parameters for each individual display can be obtained through numerical optimization. Such optimization may incorporate multiple, possibly conflicting, goals to balance desirable properties such as artifact mitigation, maximization of component display lifetimes, total system efficiency, power consumption, and output image fidelity, among many other options.

[0061] Considering a display system as an abstract (possibly composite) display that takes parameters P and source illumination S, it may be possible for multiple parameters to be jointly optimized by generating an output image. Such a system is depicted in FIG. 11, where a display parameter optimization is performed to determine the parameters P and illumination S required to reproduce an input target image T for the abstract (possibly composite) display. The simulated (or measured) output of this display is then sent back through the system to several modules, namely, an image fidelity model, a system constraints model, and a quality heuristic model.

[0062] Although not explicitly labeled for clarity of the diagram, the model used by the system implicitly has access to the target image, the source illumination, and the current parameter selection. A camera positioned to capture an image indicative of the display's output may also be incorporated into the feedback loop. In some embodiments, optimization is performed using a cost function that includes the difference between the image captured by the camera and the desired output of the display system (the target image).

[0063] Each of the multiple models attempts to correct deviations of the output image or parameter selection from the desired properties. One common model is image fidelity. It is desirable for the image generated by the system to closely approximate the target image T, or a modified version of the target image, perhaps one in which perceptual factors are taken into account. The error between the output image and the target image is used by the model to calculate parameter adjustments. Optimization may proceed until either parameter convergence is achieved or the time budget is exhausted.

[0064] The system constraint model ensures that parameter selections result in results that are physically feasible (and desirable configurations). Such criteria may include requiring that the source illumination profile is within the available power, or that the modulator parameters vary between opaque and transmissive, i.e., produce no light. Desirable configurations may include selecting parameters that have spatial or temporal coherence, are within certain ranges (e.g., see the LCoS linearity discussion above), or minimize power usage and / or maximize component lifetimes.

[0065] Image quality heuristics may be used to compensate for motions that are not easily modeled or are costly to model for imaging models. Image quality heuristics may include moire, diffraction, temporal motion, and color fringing, among other artifacts. The heuristic models are intended to help compensate for these using empirical image quality criteria. Image quality heuristics are also provided to adjust multiple parameters that optimize for human perception properties, such as veil luminance, adaptation level, average picture level, changes in sensitivity to metamerism and saturation / luma error. Sensitivity to these properties can be exploited in content generation.

[0066] FIG. 12 shows an HDR+LDR projector system depicted in the abstract display framework described above.

[0067] The LDR and HDR projectors may themselves be composite displays. An exemplary embodiment with desirable properties for commercial applications would have a relatively high-powered LDR projector that can achieve full-screen white suitable for typical average picture levels, combined with a lower-powered HDR projector that can achieve much higher peak brightness, but does not have the power to do so across the entire screen. Such a system could be much more efficient and less costly than creating a single projector capable of increased full-screen white values ​​due to the distribution of luminance in a typical image. In such an embodiment, it is desirable to provide controls that allow global dimming of the LDR projector. Some exemplary methods of providing such global dimming use an aperture, a controllable shutter, and / or a variable output light source. The aperture is a very simple display that modulates the intensity of the LDR projector, which in principle could be replaced by an LDR projector that can be dynamically modulated by the light source S1.

[0068] The display parameter optimization searches for the LDR parameters P1, aperture / drive level parameters P2, and HDR parameters P3 that make the output image O best match the target image T. The system of FIG. 12 then replaces the abstract display in the previous figures, with parameters P={P1, P2, P3} and S={S1, S3}. The output image, as modeled by the imaging model, is then:

number

[0069] Improved display parameters may be obtained through optimization, which may involve minimizing a sum of cost functions that represent image fidelity, image quality, and system constraints, such as:

number

[0070] In this specification, an image fidelity model is a function C that weights the errors between the images produced by the system, F(P,S), to produce a scalar that indicates how good the current set of parameters is. Common examples of C are the mean squared error (MSE) or the mean absolute error (MAE).

[0071] The functions Qi represent image quality heuristics / models that also generate a scalar value indicating how good the current parameters are in terms of unmodeled artifacts, e.g., moiré, color fringing, or diffraction artifacts. The constants α and βi control the relative importance (which may conflict) given to various terms, providing a way to favor one objective of content generation over another.

[0072] The constraints K impose conditions on parameters, for example if a modulator in a projector must operate in a range between fully transparent and fully opaque. They are represented herein as set-valued constraints that are either satisfied (Kj(P,S)=0) or not satisfied. However, existing optimization techniques may relax these conditions to allow violation of minor constraints.

[0073] Although not explicitly enumerated, the constraint function K and the image quality model Q may also have a dependency on the output image O=F(P,S).

[0074] It is now possible to depict several different schemes for separating image content between LDR and HDR projectors. Several different examples are presented here: Smooth blending between HDR and LDR projectors

[0075] Although HDR projectors are necessary in high brightness areas, from an image quality perspective it may also be desirable to use HDR projectors in areas below the full screen white level of an LDR projector, which requires splitting the content between two projectors.

[0076] One straightforward way to approach this is to blur or diffuse the mask used by the HDR projector, for example by blurring an extended binary mask of pixels above the full-screen white of the LDR projector. A more sophisticated approach may compute an approximation of the veil luminance at each pixel to dynamically adjust the blending parameters.

[0077] There are many other options for how to split content between multiple component projectors. Examples of these options are described below: 1) A target luminance distribution where there is a favorable ratio between the total LDR projector contribution and the HDR projector contribution (e.g., 95% and 5%, respectively) for scenes of medium brightness with high black levels and highlights. 2) A target luminance distribution that favors the use of HDR projectors while minimizing the use of LDR projectors via dimmable light sources or external apertures. Such an objective could potentially reduce energy usage and cooling requirements, but also improve black levels for dark scenes with bright highlights. 3) Target a temporally consistent luminance distribution of one or both projectors to minimize temporal artifacts. 4) Reach the full widest dynamic range, highest peak luminance, or deepest black level of the combined display system to maximize perceived image quality.

[0078] In any of these approaches, the blending factor may be dynamically adjusted spatially within the scene to achieve the desired local behavior. For example, low-luminance content adjacent to high-luminance regions may be obscured by the veiling luminance of the highlights. In this case, neither the LDR nor the HDR projector needs to display content for those regions. Alternatively, some scenes may have large bright and large dim regions. The above adjustments may then be made to take into account the scattering behavior of the projector. Expanding the color gamut

[0079] If the primaries used in the HDR and LDR projectors are different, perhaps by design, it may be possible to extend the color gamut of the combined system. This can be achieved by mapping the target image into an appropriate color space and determining which mix of the two available primary color sets best represents the target color, e.g., choosing the widest possible set of primaries to improve metamerism. The process here is in principle similar to that used in extending the dynamic luminance range, as described throughout this document. Super Resolution

[0080] When HDR and LDR projectors are deregistered, it may be possible to increase the apparent resolution of the combined system to reduce aliasing near edges. This can be achieved by optimizing the high-resolution target image, which will automatically adjust the projector contributions between HDR and LDR to best approximate high spatial frequency features. Scattering compensation and feedback of surrounding conditions

[0081] Scattering from the viewing environment can result in dark image regions with elevated levels. Incorporating a heuristic variance model for either the target or the output image allows this to be taken into account to compensate for this effect. In this case, the imaging model F can be expressed as follows:

number

[0082] where R is a function modeling scattering from the viewing environment and F' is an imaging model of the system in a non-scattering viewing environment. Display parameters optimized using this imaging model automatically attempt to compensate for the resulting scattering.

[0083] A similar approach can use actual measurements of scattered light instead of the function R to dynamically compensate for light scattered from the viewing environment.

[0084] The method illustrated in FIG. 13 details one approach to determining which images are likely to be shown by which projectors and how they are calculated.

[0085] The decision box depicted in Figure 13 may incorporate a small amount of temporal hysteresis so that LDR and HDR projectors will not bounce around the threshold from image to image.

[0086] The "Tonemap Image" operation analyzes the luminance levels in the incoming image (if available) and maps them to the capabilities of the combined LDR and HDR projector. This operation also takes into account the ambient light level and the maximum overall luminance that a viewer would find comfortable when mapping the darker areas of the image.

[0087] The "black level adjustment" operation will increase the black level of the mapped image in cases where a viewer would not be able to perceive a lower black level. An example of this is black text in a white field where the veil luminance would not allow a viewer to distinguish a very low black level from a slightly increased one. To achieve this, a forward model of the projector may be used (to predict halos from brightness).

[0088] If after the above operations the image still has a low black level, the aperture size (the amount of light attenuated by the aperture or by dimming the light source) may be calculated to compensate for the increased natural black level of the LDR projector. By reducing the aperture, the peak brightness available from the LDR projector will also become darker. The reduced peak brightness may also be calculated.

[0089] If the LDR projector, with its reduced aperture size, would not likely provide enough light for the image, then an HDR projector may be used to generate the entire image. Note that, as explained in the aperture section above, it may be desirable to never completely block all light from the LDR projector.

[0090] If the black level is not low and the image contains highlights that cannot be shown using only the LDR projector due to insufficient brightness capabilities, separate images for the LDR and HDR projectors may be calculated. In this case, since the two images are being combined on the screen, care should be taken to "blend" them together so that border artifacts are not generated when adjacent pixels are fed from different projectors. The following approaches may be taken either individually or in combination: Threshold banding (always sum pixels) Use different gamma curves for each projector Spatial variation (slight blurring of one projector) Temporal dithering

[0091] An example of threshold banding would be in a small pixel area surrounding a bright feature, where both projectors would contribute light and sum together to produce multiple pixels. The size of this area could be calculated from the veil luminance effect, when there is a fairly smooth transition between the highlight and the adjacent feature (a bright spot in the gradient), or it could simply be a fixed number of pixels. Using brightness boosters for multi-stage projection.

[0092] FIG. 14 shows a schematic diagram of a projection system having two imagers in which an auxiliary boost light source is used when required to reproduce a particular high brightness and / or low contrast image.

[0093] High dynamic range projectors use two or more imaging stages to reduce the black level when generating an image. Each one of these imaging stages has a loss associated with it, so when generating very bright images, there is much more light loss in a multi-stage projector compared to a single-stage projector. Light can be added, if necessary, before the final imaging stage to promote the efficiency of the system when a low black level is not required.

[0094] The imaging elements used in the light path of a projection system are by their nature non-ideal. When forming an image, they allow light to leak through dark areas and absorb some light in bright areas, at the expense of overall contrast. To address this, projector manufacturers have created systems with multiple imaging elements to reduce the amount of light leaking through the system in dark areas. This in turn required many brighter light sources to compensate for the transmission loss through two (or more) imaging elements in bright areas. These projectors exhibit significantly lower operating efficiency when showing bright images compared to single stage projectors.

[0095] The projection system according to the exemplary embodiment in claim 14 will analyze the nature of the image being projected, and in the case of low contrast, high brightness images, will add a calculated uniform amount of light before the final imaging stage. The added light will then have to travel through perhaps only a single imaging stage and therefore incur much lower transmission losses. Thus, the operating efficiency of the system will be substantially increased when producing bright images. When producing images that require much less light and higher contrast, little or no light will be added before the last imaging element to preserve the low black levels expected in a multi-stage system.

[0096] The boost light provided to the second imaging stage does not necessarily have to be uniform or uniform. In some embodiments, the boost light is non-uniform. An exemplary application of this is when the first imaging stage provides a light output that includes undesired light patches or other artifacts. For example, if the first stage is a light guiding stage, the first stage may provide static artifacts that are not guidable (e.g., an overall roll-off of intensity toward the edges, or visible patches and stripes that are not corrected for some reason from multiple different laser diodes). In such cases, the boost light may be structured in such a way that the sum of the boost light and the artifacts is a uniform or nearly uniform illumination. This may be done by providing a non-uniform pattern of boost light that is inversely proportional to the pattern of artifacts from the first stage.

[0097] FIG. 14 shows a "primary light source" and a "boost light source." The light output of both light sources may be controlled in an independent manner. The "primary light source" is expected to illuminate the first imager in a fair or otherwise prescribed manner. The "boost light source" is expected to illuminate the last imager.

[0098] The purpose of the first imaging element is to block or direct light away from the darker parts of the image so that the last imaging element probably does not have to block as much light from the darker parts of the image being projected, resulting in a high contrast image if desired. The first imaging element may, for example, modulate the phase and / or intensity of the light from the primary light source.

[0099] The "last imager" may be paired so that the boost light source has its own independent light path to the screen. This may be desirable in very high power systems where a single final stage imager may not be able to handle the thermal stresses or intensity associated with both light paths converging on its surface.

[0100] In a color projector, the methods may be implemented separately for each primary color in the system, or may operate in a color field sequential manner in one or more exemplary implementations.

[0101] 15 is a flow chart illustrating an intensity control method for the light source in such a projection system. Such a method may be implemented in a display controller. In an alternative embodiment, the method is implemented in an image processing system that provides output image data accompanied by control signals for the light source.

[0102] An algorithm is implemented to adjust the relative intensity settings of the two light sources. When displaying a low contrast image, or when veiling luminance in the observer's eye or other optical scattering in the system or environment masks surrounding dark areas, the boost light will be active so that boosting the intensity of those dark areas does not result in noticeable degradation of the displayed image.

[0103] For example, image statistics such as a histogram of the luminance distribution within the image, or other methods, may be used to determine the overall contrast requirements of the image. The boost light source may be used whenever possible, as it is a more efficient light path than that from the primary light source, and may always be used to provide brightness down to the darkest level present in the image.

[0104] The primary light source may be dimmed to compensate for the light being added to the image by the boost light source.

[0105] FIG. 16 shows an example image with different characteristics such as peak brightness, average brightness and black level for the supplemental (boost) light source as well as a reasonable intensity level.

[0106] Cases A and H show images that are uniformly white at full intensity. In cases B, C, D, I, and J, the boost light may be driven higher than the lowest level due to the veil luminance effect. Cases P and Q are also affected by the veil luminance, allowing some light to come from the boost light. In cases K, L, M, N, and O, the boost light is driven to the lowest brightness level present in the image. For example, the boost light may be provided at a level determined by multiplying the lowest brightness level of the image by a factor. That factor may be based on the contrast function of the second modulator. For example, if the lowest brightness level in a particular image is Lmin=1 cd / m2 and the contrast of the second modulator C2=2000:1, the boost light may be provided at a brightness sufficient to achieve 2000 cd / m2 with the fully open modulator C2, while allowing the light level to be reduced to 1 cd / m2 by setting the second modulator to its lowest light transmission state.

[0107] In some embodiments, if a dark patch exceeds a threshold size such that it would not be masked by the veiling luminance effect, the boost light would likely be turned off completely and the non-black areas of the screen would be illuminated, perhaps with two imaging elements in series, greatly reducing the amount of light leaking into the dark areas. In the exemplary cases E, F, G, R, S, T, and U, there is enough dark content that the boost light is powered off to preserve the black level.

[0108] The boost light and the main light source do not necessarily have to be separate from each other. In some embodiments, an optical system is provided that directs some or all of the light from the main light source directly to the final imager, bypassing the first imager. For example, a variable beam splitter may be applied to divert some light from the main light source to the final imager. Some embodiments have both a separate boost light source and provisions for diverting light from the main light source to the final imager.

[0109] In some embodiments, one or more optical elements are provided to combine light from the boost light source with the light being modulated by the first imaging element and direct the combined light to the last imaging element, the one or more optical elements in some embodiments comprising a prism.

[0110] In some embodiments, the boost light source comprises multiple light sources, such as multiple light emitting diodes (LEDs). In one exemplary embodiment, the boost light source is disposed around the periphery of the first imaging element. For example, the boost light source may comprise a ring of LEDs. Suitable reflectors, diffusers, spaces and / or other optical elements may be provided to ensure that the light from the boost light source is evenly distributed to the last imaging element.

[0111] 2A to 6A show exemplary images in five cases with different characteristics as described above. The following describes, by way of example, how the supplemental (boost) light source may be controlled for each of these five cases. In an exemplary embodiment, the projector system used in the following examples may include a highly efficient projector with a directable light source (primary light source and first imager), a second imager, and a booster stage that illuminates only the second imager. The second imager may comprise, for example, a reflective or transmissive spatial light modulator, such as an LCD panel, LCOS, DMD, reflective LCD, etc. Case 1: Bright low dynamic range image, enhanced black levels

[0112] The boost stage is used to illuminate most of the image. The first, stimulating and high contrast stages are used to add minimal highlights to the image. Stimulating is rarely needed. Case 2: Dim, low dynamic range image, high black levels

[0113] The boost stage is used to illuminate the entire image, and the guiding stage is not used. Case 3: Bright high dynamic range image, high black level

[0114] The boost stage is fully on. The induction stage, which provides maximum induction, is also fully on. Case 4: Bright high dynamic range image, low black levels

[0115] The boost stage is off. The image is generated using only the stimulation stage. Case 5: Dim low dynamic range image, low black levels

[0116] The boost stage is on, but at a reduced intensity to preserve some of the black levels in the image, and the guiding stage is off since no highlights are needed.

[0117] The techniques described herein may be applied, without limitation, to displays of the type described in U.S. Patent Application No. 61 / 893,270, filed October 20, 2013, which is hereby incorporated by reference herein for all purposes. Using a combination of projectors to present stereoscopic content

[0118] A combined projector or light source system such as those described herein is useful for multiple applications requiring efficient or low-cost or high-brightness reproduction of three-dimensional (stereoscopic) content.

[0119] A stereoscopic image pair comprises an image intended for viewing with the right eye and an image intended for viewing with the left eye. Image parallax creates the depth effect. Neither parallax would render the image perceived to be in the plane of the projection screen. Parallax between the left and right eye images would render objects perceived to be further away from the projection screen plane, either closer to the viewer (audience) or, if inverted, further away (perceived to be behind the screen plane).

[0120] One characteristic of movies and other stereoscopic image content is that a comfortable viewing experience is likely to be achieved if the disparity between the left and right eye views is not too large (e.g., depicted objects are not perceived as too close to the viewer). The difference between the left and right eye views in a stereoscopic image pair is therefore usually kept small. Even in image pairs with depicted content that is perceived as very close (or very far) to the viewer, many image areas in the left and right eye views will typically be the same. This is because in almost all cases only a few objects are likely to be rendered as close or far to the viewer.

[0121] Many, but not all, practical stereoscopic projection systems require filtering of the light reflected from the projection screen before it enters each of the observer's eyes. The filtering results in multiple different images being provided to the observer's left and right eyes. Filtering is often provided using eyeglasses that provide multiple different filters for the left and right eyes. Common techniques use color filters (notch filters for some or all of the primary colors for the left and right eyes), circular or linear polarizing filters, temporal shutters or temporal polarizing switches.

[0122] The projection system is configured to generate different images for the left and right eyes with different corresponding (to filters at the right and left eyes) light characteristics, for example, different narrow band primaries for the left and right eye views, or clockwise and counterclockwise circularly polarized light, or light with orthogonal linear polarization states, or temporal light fields that match the polarization of a temporal shutter or polarization switch at the eye.

[0123] All of these filtering techniques have in common that a large amount of light is lost between the projector's light source and the observer's eyes, compared to a similar non-stereoscopic projection system. Stereoscopic projection systems are also more complex and therefore more costly than non-stereoscopic projection systems. Another challenge is that it is not always possible or easy to upgrade an existing non-stereoscopic projector to operate as a stereoprojector.

[0124] In the systems described herein, it is possible to use one projector in a non-stereoscopic mode with a light source that is compatible with both the left-eye and right-eye filters (e.g., a broadband light source in the case of a color notch filter-based system, or a random polarization system in the case of either a circular or linear polarization filter system, or a permanently on light source in the case of any temporal shutter filtering system). The non-stereoscopic projector will generate those portions of the image that are common to both the left-eye and right-eye views.

[0125] A second projector (one or more projectors) may then be used to display the portions of the images that differ between the left and right eye views. The second projector projects light with the characteristics (wavelength, or polarization, or temporal image field) required for the left and right eye filters.

[0126] There are several advantages to using such a system: most of the components are already included in the design, so the additional cost of enabling stereoscopic projection is minimal compared to the systems described herein.

[0127] The power requirements for the second projector can be lower because the image regions with parallax between left and right are typically not large relative to all pixels of the image. Light steering may be used to steer light to display areas that correspond to depicted objects that are perceived to be out of the plane of the display screen.

[0128] Producing good separation (=contrast) between the left and right eye is not easy and is costly. A less-than-perfect separation would result in some light intended for the right eye getting into the left eye. This effect is known as ghosting, and reduces image quality and causes headaches. Because the second projector power requirements are lower than the main projector, and the cost of making such a second projector is lower, more care can be taken to ensure that the left and right eye views are truly separated.

[0129] A second, low powered projector can be cost effectively added to upgrade an existing non-stereoscopic projection system, enabling it to display stereoscopic images. Power output relationship between LDR / HDR projectors

[0130] In a projector system as described herein, it should be possible to combine an LDR projector with, for example, five times the power of an HDR projector, which would allow for a more economical setup, since HDR projectors are much more expensive than LDR projectors. Non-Limiting Listed Exemplary Embodiments The following are non-limiting enumerated exemplary embodiments. 1. A method for displaying an image defined by image data, the method comprising: generating a first modulated light by modulating light from a first light source using a first imaging element; providing a boost light; combining the boost light and the first modulated light; further modulating the combined light using a second imaging element; A method comprising: 2. The method according to aspect 1, wherein combining the boost light and the first modulated light comprises illuminating a surface of the second imaging element with both the boost light and the first modulated light. 3. The method according to embodiment 1 or 2, wherein combining the boost light and the first modulated light comprises directing the boost light and the first modulated light to a prism. 4. The method according to aspect 2, wherein the boost light evenly illuminates a surface of the second imaging element. 5. A method according to aspect 2, wherein the boost light is positioned to provide structured illumination to a surface of the second imaging element with a desired luminance profile. 6. A method according to embodiment 5, wherein the structured illumination has a higher brightness on some portions of a surface of the second imaging element than on other portions of the surface of the second imaging element, and the brightness of the brightest portions of the structured illumination is at least twice the brightness of the lowest brightness portions of the structured illumination. 7. The method according to any one of aspects 1-6, wherein operating the boost light source comprises controlling an output of light by the boost light source. 8. The method according to embodiment 7, wherein controlling the output of light by the boost light source is based in part on image contrast. 9. The method according to embodiment 8, comprising determining contrast of the image by processing an image histogram for the image. 10. The method according to any one of aspects 1 to 9, comprising dimming the first light source in combination with manipulating the boost light source. 11. A method according to any one of aspects 1 to 9, comprising the steps of processing the image data to identify any dark patches exceeding a threshold size, and turning off the boost light source again in response to identifying a dark patch exceeding the threshold size. 12. The method according to any one of aspects 1 to 11, wherein generating the boost light comprises operating a boost light source separate from the first light source. 13. The method according to any one of aspects 1 to 11, wherein generating the boost light comprises directing light from a first light source to a second imaging element. 14. The method according to embodiment 13, wherein directing light from the first light source to the second imaging element comprises controlling a variable beam splitter. 15. A method according to embodiment 13, wherein the step of directing light from a first light source to a second imaging element includes transmitting the light through a switch having an input port arranged to receive light from the first light source and two or more output ports, one of the output ports arranged to transmit the light to the second imaging element. 16. The method according to aspect 13 or 15, comprising adjusting the amount of boost light provided to the second imaging element by time division multiplexing. 17. A method according to any one of aspects 1 to 16, comprising: processing image data to determine a lowest brightness level present in the image; and operating a boost light source at a level corresponding to the lowest brightness level in the image. 18. A method according to any one of aspects 1 to 16, comprising the steps of processing image data to simulate a veil luminance, determining the lowest perceptible luminance level present in the image, and operating a boost light source at a level corresponding to the lowest perceptible luminance level. 19. The method according to any one of aspects 1 to 17, wherein the second imaging element comprises a spatial light modulator. 20. The method according to any one of aspects 1 to 17, wherein the second imaging element comprises an LCD panel, an LCOS, a reflective LCD panel, or a DMD. 21. A method for generating a signal to control a projector that displays an image according to image data, the projector comprising a first imaging element configured to provide modulated light to a second imaging element for further modulation by the second imaging element, and a boost light configured to transmit additional illumination light for modulation by the second imaging element, the method comprising simulating a veil luminance to determine a lowest perceptible luminance level in the image, and generating a signal to set the boost light to a level corresponding to the lowest perceptible luminance level. 22. The method according to embodiment 21, comprising performing a step of simulating a veil luminance in response to image data satisfying a condition. 23. The method according to embodiment 22, comprising processing the image data to determine a contrast of the image, the condition including determining a contrast below a threshold. 24. The method according to aspect 22 or 23, the method comprising detecting any dark features in the image, and the condition comprising determining that all of the plurality of dark features are smaller than a threshold size. 25. The method according to embodiment 24, comprising generating a signal to set the boost light source off if any of the plurality of dark features is greater than a threshold size. 26. A method according to any one of aspects 21 to 23, comprising processing image data to detect dark features in the image data, the method comprising generating a signal to set a boost light source off if any of a plurality of dark features is greater than a threshold size. 27. A method according to any one of aspects 21 to 26, comprising processing image data to determine an amount of dark image, and generating a signal to set the boost light source off if the image is predominantly dark. 28. A method according to any one of aspects 21 to 27, comprising generating a signal to set the boost light to a level corresponding to the lowest detectable brightness level in combination with generating a signal to reduce the level of the primary light source illuminating the first imaging element. 29. A method according to any one of aspects 21 to 27, performed by a controller in a projector. 30. The method according to any one of aspects 21 to 27, wherein the method is performed by an image processing system configured to provide output image data accompanied by a control signal for the boost light. 31. The method according to any one of aspects 21 to 30, wherein the boost light uniformly illuminates the second modulator. 32. The method according to any one of aspects 21 to 30, wherein the boost light non-uniformly illuminates the second modulator. 33. An optical projector, 1. A light projector comprising: a first imaging element configured to provide modulated light to a second imaging element for further modulation by the second imaging element; and a boost light configured to transmit illumination light to the second imaging element for modulation by the second imaging element. 34. The light projector according to embodiment 33, wherein the first imaging element is configured to modulate one or both of the phase and amplitude of the light from the primary light source. 35. The light projector according to aspect 33 or 34, wherein the boost light is separate from the primary light source. 36. The light projector according to embodiment 35, wherein the boost light has multiple light sources. 37. The light projector according to embodiment 35, wherein the multiple sources of boost light include multiple light emitting diodes (LEDs). 38. The light projector according to embodiment 35, wherein the multiple sources of boost light include multiple laser diodes. 39. The light projector according to any one of aspects 36 to 38, wherein the multiple sources of boost light are arranged around the periphery of the first imaging element. 40. The light projector according to any one of aspects 36 to 39, wherein the multiple sources of boost light are arranged in a ring. 41. The light projector according to any one of aspects 36 to 40, wherein the multiple sources of boost light are individually controllable to provide a desired pattern of boost light at the second imaging element. 42. The light projector according to embodiment 33 or 34, wherein the boost light has an optical system configured to direct light from the primary light source directly to the second imaging element. 43. The light projector according to embodiment 42, wherein the optical system comprises a variable beam splitter. 44. The light projector according to embodiment 33 or 43, comprising a controller configured to process image data and output control signals for the first and second imaging elements and the boost light. 45. A light projector according to embodiment 44, wherein the controller is configured to simulate a veil brightness to determine the lowest perceptible brightness level in the image and set the boost light to a level corresponding to the lowest perceptible brightness level. 46. ​​The light projector according to embodiment 45, wherein the controller is configured to execute a step of simulating a veil brightness in response to image data satisfying a condition. 47. The light projector according to embodiment 46, wherein the controller is configured to process the image data and determine a contrast of the image, and the condition includes determining that the contrast is below a threshold value. 48. The light projector according to aspect 46 or 47, wherein the controller is configured to detect any dark features in the image, and the condition includes determining that all of the dark features are smaller than a threshold size. 49. The light projector according to embodiment 48, wherein the controller is configured to set the boost light source off if any of the dark features are greater than a threshold size. 50. A light projector according to any one of aspects 47 to 49, wherein the controller is configured to process image data, detect dark features in the image data, and set the boost light source off if any of the dark features are larger than a threshold size. 51. A light projector according to any one of aspects 47 to 50, wherein the controller is configured to process image data, determine an amount of dark image, and set the boost light source off if the image is predominantly dark. 52. A light projector according to any one of aspects 47 to 51, wherein the controller is configured to set the boost light to a level corresponding to the lowest detectable brightness level in combination with reducing the level of illumination of the first imaging element. 53. The light projector according to any one of aspects 33 to 52, wherein the second imaging element comprises a spatial light modulator. 54. The light projector according to any one of aspects 33 to 52, wherein the second imaging element comprises an LCD panel, an LCOD, a reflective LCD panel, or a DMD. 55. A method of light projection comprising controlling a plurality of imaging stages arranged in series to generate modulated light, and selectively adding light before a last one of the imaging stages if a low black level is not required. 56. A light projection method according to embodiment 55, comprising processing image data to determine a contrast of an image represented by the image data, the method comprising adding light if the contrast is below a threshold. 57. The light projection method according to embodiment 56, comprising determining contrast by processing an image histogram. 58. The light projection method according to any one of aspects 55 to 57, comprising a step of uniformly distributing the light added in the last one of the imaging steps. 59. An optical projection method according to any one of aspects 55 to 58, wherein the step of controlling the multiple imaging stages includes controlling the imaging stages to modulate one or more of the phase and amplitude of the light incident on the imaging stages. 60. A light projection method according to any one of aspects 55 to 59, comprising varying an amount of light added based on data defining an image to be projected. 61. The light projection method according to any one of aspects 55 to 57, comprising a step of non-uniformly distributing the light applied in the last one of the multiple imaging steps. 62. A light projection method according to embodiment 61, comprising a step of structuring the added light such that the added light summed with multiple artifacts from earlier multiple imaging stages results in uniform illumination at the end of the multiple imaging stages. 63. An optical projector, a first imaging stage arranged to modulate light from a primary light source; a second imaging stage arranged to further modulate the light modulated by the first imaging element; a boost light arranged to add light after the first imaging stage and before the second imaging stage such that the added light is modulated by the second imaging stage; and a controller operative to process image data to manipulate the boost light when a low black level is not required. 64. A light projector according to aspect 63, wherein the controller is configured to process the image data to determine a contrast of an image represented by the image data, and to operate the boost light to add light if the contrast is below a threshold. 65. The light projector according to embodiment 64, wherein the controller is configured to determine the contrast by processing an image histogram. 66. The light projector according to any one of aspects 63 to 65, wherein the boost light is positioned to evenly illuminate the second imaging stage. 67. The light projector according to any one of aspects 63 to 65, wherein the first imaging stage is controllable to modulate one or more of the phase and amplitude of light incident on the first imaging stage. 68. The light projector according to any one of aspects 63 to 67, wherein the controller is configured to vary the amount of light added by the boost light based on image data. 69. A method for projecting a light pattern defined by image data, the method comprising: generating a first modulated light by modulating light from a first light source using a first imaging element; providing a boost light; further modulating the first modulated light to modulate a boost light; and combining the modulated boost light and the further modulated first modulated light. 70. The method according to embodiment 69, wherein combining the modulated boost light and the further modulated first modulated light comprises projecting the modulated boost light and the further modulated first modulated light onto a surface. 71. The method according to embodiment 69 or 70, wherein the modulated boost light has a higher black level than the further modulated first modulated light. 72. The method according to any one of aspects 69 to 71, wherein the modulated boost light has a higher peak brightness than the further modulated first modulated light. 73. The method according to any one of aspects 69 to 72, wherein the modulated boost light has a lower dynamic range than the further modulated first modulated light. 74. The method according to any one of aspects 69 to 73, wherein both the further modulating of the first modulated light and the modulating of the boost light are performed at a second imaging element. 75. The method according to any one of aspects 69 to 74, wherein both the further modulating the first modulated light and the modulating the boost light apply the same modulation. 76. The method according to embodiment 75, comprising evenly illuminating a surface of the second imaging element with boost light. 77. The method according to any one of aspects 69 to 76, wherein providing the boost light comprises controlling an output of light by a boost light source. 78. The method according to embodiment 77, wherein the step of controlling the output of light by the boost light source is based in part on the contrast of the image data. 79. The method according to embodiment 78, comprising determining contrast of the image data by processing an image histogram for the image data. 80. The method according to any one of aspects 69-79, comprising dimming the first modulated light in combination with providing the boost light. 81. A method according to any one of aspects 69 to 80, comprising the steps of processing the image data to identify any dark patches that exceed a threshold size, and turning off the boost light in response to identifying a dark patch that exceeds the threshold size. 82. The method according to embodiment 75, comprising the step of non-uniformly illuminating a surface of the second imaging element with boost light. 83. The method according to any one of correspondences 69 to 82, wherein the step of providing boost light comprises operating a separate boost light source from the first light source. 84. The method according to any one of aspects 69 to 82, wherein providing the boost light comprises directing light from a first light source to a second light modulator. 85. The method according to embodiment 84, wherein directing the light from the first light source to the second light modulator comprises controlling a variable beam splitter. 86. The method according to any one of aspects 69 to 85, comprising: processing image data to determine the lowest brightness level present; and providing boost light at a level corresponding to the lowest brightness level. 87. A method according to any one of aspects 69 to 85, comprising the steps of processing image data to simulate a veil luminance, determining a lowest perceptible luminance level present in the image, and providing boost light at a level corresponding to the lowest perceptible luminance level. 88. A projector system comprising a plurality of projectors, the plurality of projectors including at least a first projector and a second projector arranged such that light projected by the first and second projectors is combined into a projected image for display, the first and second projectors having a plurality of different image characteristics selected from dynamic range, black level and peak brightness. 89. A projector system according to aspect 88, comprising a control system connected to receive image data defining image content to be projected by the projector system and to control the projector system to project the image content; The control system is configured to process the image data to generate modified image data for projection by at least one of the first and second projectors. 90. A projector system according to aspect 89, wherein the control system is configured to process image data to determine a dynamic range, a black level and an average brightness level, and to generate modified image data based on the dynamic range, the black level and the maximum brightness level. 91. A projector system according to embodiment 90, wherein the first projector has a higher dynamic range than the second projector, a higher peak brightness than the second projector, and a lower black level than the second projector. 92. In a projector system according to aspect 91, when the image data has a brightness in a high brightness area higher than the maximum brightness of the second projector, the control system controls the brightness threshold to cause the first projector to project light at least in the higher brightness area. 93. A projector system according to embodiment 92, wherein when the black level is above the black level threshold, the control system is configured to control the second projector to project as much light of the image as is within the capabilities of the second projector. 94. A projector system according to any one of aspects 91 to 93, wherein the control system is configured to generate modified image data for the first projector by a method comprising generating a binary mask of a plurality of pixels having a luminance above a full-screen white value of the second projector. 95. A projector system according to aspect 94, wherein the control system is configured to expand and blur the binary mask. 96. A projector system according to any one of aspects 91 to 95, wherein the control system is configured to generate modified image data for the second projector by a method comprising clipping the luminance of a plurality of pixels in the image data having luminance values ​​above a full-screen white value of the second projector. 97. A projector system according to any one of aspects 89 to 96, wherein the control system is configured to provide image data to the second projector unaltered if the dynamic range, black level and average brightness level are within the capabilities of the second projector. 98. A projector system according to any one of aspects 89 to 97, wherein the second projector has a controllable aperture, and the control system is configured to control the aperture to reduce the black level of the second projector in at least some cases where the black level of the image data is less than the black level of the second projector. 99. A projector system according to any one of aspects 89 to 98, wherein the control system has an imaging model for the projector system, and the control system is configured to obtain values ​​of control parameters for the first and second projectors by performing optimization. 100. The projector system according to aspect 99, wherein performing the optimization includes minimizing a sum of cost functions. 101. A projector system according to aspect 100, wherein the cost function includes a plurality of cost functions related to image fidelity, image quality and system constraints. 102. The projector system according to aspect 101, wherein the cost function related to image fidelity includes a mean squared error value or a mean absolute error value. 103. A projector system according to aspect 101 or 102, wherein the cost function related to image quality includes one or more heuristics that indicate how favorable the current set of control parameters is with respect to multiple artifacts not modeled by the imaging model. 104. The projector system according to aspect 103, wherein the heuristics include one or more of heuristics for moire, color fringes, and diffraction artifacts. 105. The projector system according to any one of aspects 101 to 104, wherein the plurality of constraints limit values ​​of the control parameters to physically realizable parameters. 106. The projector system according to any one of aspects 99 to 105, wherein the control system is configured to attempt to achieve a desired ratio of the total light output of the first and second projectors. 107. A projector system according to any one of aspects 99 to 105, wherein the control system is biased to control one of the first and second projectors such that it provides as much light as possible to the projected image. 108. The projector system according to any one of aspects 99 to 107, wherein the imaging model includes a heuristic distribution model. 109. A projector system according to any one of aspects 89 to 108, wherein the first and second projectors have a plurality of different primary colors, and the controller is configured to balance the light output by the first and second projectors to achieve a plurality of colors in the projected image that are outside the color gamut of at least one of the first and second projectors. 110. A projector system according to any one of aspects 89 to 109, wherein the controller is configured to balance the light output by the first and second projectors to achieve optimized reproduction of high spatial frequency features in the image content of the projected image. 111. The projector system according to any one of aspects 89 to 110, wherein the control parameters include a plurality of pixel values ​​for the first and second projectors. 112. The projector system according to any one of aspects 89 to 111, wherein the control parameters include a plurality of light source values ​​for the first and second projectors. 113. The projector system according to any one of aspects 89 to 112, wherein the control system is configured to take into account ambient light in the area of ​​the projected image. 114. A method or apparatus according to any one of the above aspects applied to project light in a vehicle headlight. 115. A method and apparatus according to any one of the above aspects, comprising combining light projected from a two-dimensional projector with light comprising a stereoscopic image pair projected by one or more other projectors, the left-eye and right-eye images of the stereoscopic image pair being distinguishable from one another in one or both of time and distinguishable light characteristics, and the light projected by the two-dimensional projector comprising light corresponding to both the left-eye and right-eye images. 116. An apparatus having any novel and non-obvious feature, combination of features, or sub-combination of features as described herein. 117. A method having any novel and non-obvious step, operation, combination of steps and / or operations, or sub-combination of steps and / or operations as described herein. Interpretation of Terms

[0131] Throughout this description and claims, unless circumstances clearly require otherwise, "Comprises," "comprises," and the like are to be construed in an inclusive sense, as opposed to an exclusive or exhaustive sense, i.e., "including but not limited to." "Connected", "coupled" or any variation thereof means any connection or coupling, either direct or indirect, between two or more elements, and the coupling or connection between the elements may be physical, logical, or a combination thereof. The words "herein," "above," "below," and words of similar import, when used to describe this specification, shall refer to this specification as a whole and not to any particular portions of this specification. "Or" refers to a list of two or more items and covers all of the following multiple interpretations of the word: any of the items in the list, all of the items in the list, and any combination of the items in the list. The singular forms "a", "one" and "the" include any appropriate plural forms.

[0132] Directional terms such as "vertical," "lateral," "horizontal," "upward," "downward," "front," "rear," "inside," "outside," "vertical," "side," "left," "right," "front," "rear," "top," "bottom," "lower," "upper," "lower," and the like, when used in this description and any accompanying claims (if any), depend on the particular orientation of the device being described and shown. The subject matter described herein may assume a variety of alternative orientations. Thus, these directional terms are not precisely defined and should not be narrowly interpreted.

[0133] The embodiments of the present invention may be implemented using specifically designed hardware, configurable hardware, programmable data processors configured by providing software (which may optionally include "firmware") executable on a data processor, application specific computers or data processors specifically programmed, configured, or constructed to perform one or more steps in the methods detailed herein, and / or combinations of two or more of these. Examples of specifically designed hardware include logic circuits, application specific integrated circuits ("ASICs"), large scale integrated circuits ("LSIs"), very large scale integrated circuits ("VLSIs"), etc. Examples of configurable hardware include one or more programmable logic devices, such as programmable array logic ("PALs"), programmable logic arrays ("PLAs"), and field programmable gate arrays ("FPGAs"). Examples of programmable data processors include microprocessors, digital signal processors ("DSPs"), embedded processors, graphics processors, mathematical co-processors, general purpose computers, server computers, cloud computers, mainframe computers, computer workstations, etc. For example, one or more data processors in control circuitry for a device may implement the methods described herein by executing software instructions in program memory accessible to those processors.

[0134] Although processes or blocks are presented in a given order, alternatives may perform routines having steps in a different order or utilize systems having blocks. Also, some processes or blocks may be removed, moved, added, subdivided, combined, and / or modified to provide alternative or subcombinations. Each of these processes or blocks may be implemented in a variety of different ways. Also, while multiple processes or blocks are at times shown as being performed in series, these processes or blocks may instead be performed in parallel or may be performed at different times.

[0135] The present invention may also be provided in the form of a program product. The program product may comprise any non-transitory medium carrying a set of computer-readable instructions that, when executed by a data processor, cause the data processor to perform the method of the present invention. A program product according to the present invention may be in any of a wide variety of forms. The program product may include non-transitory media such as, for example, magnetic data storage media including floppy disks, hard disk drives, optical data storage media including CD-ROMs, DVDs, electronic data storage media including ROMs, flash RAMs, EPROMs, physically embedded or pre-programmed chips (e.g., EEPROM semiconductor chips), nanotechnology memories, and the like. The computer-readable signals in the program product may optionally be compressed or encrypted.

[0136] In some embodiments, the present invention may be implemented in software. For clarity, "software" includes any instructions executed by a processor, and may include (but is not limited to) firmware, resident software, microcode, etc. Both the processing hardware and software may be, in whole or in part, centralized or distributed (or a combination thereof) as is known to those skilled in the art. For example, the software and other modules may be accessible via local memory, over a network, via a browser or other application in a distributed computing context, or via other means suitable for the purposes described above.

[0137] Where a component (e.g., a software module, a processor, an assembly, a display, an aperture, a device, a circuit, etc.) is referred to above, unless otherwise indicated, the reference to that component (including the reference to the "means") should be interpreted as including any component that performs the function of the described component (i.e., is functionally equivalent) as an equivalent of that component, including multiple components that are not structurally equivalent to the disclosed structures that perform that function in the exemplary embodiments in which the invention is illustrated.

[0138] Specific examples of systems, methods, and devices are described herein for illustrative purposes. These are merely examples. The techniques provided herein may be applied to multiple systems other than the exemplary systems described above. Many changes, modifications, additions, omissions, and permutations are possible within the practice of the invention. The invention includes variations of the described embodiments that will be apparent to those skilled in the art. These include variations obtained by replacing features, elements, and / or operations with equivalent features, elements, and / or operations, mixing and matching features, elements, and / or operations from different embodiments, combining features, elements, and / or operations from the embodiments as described herein with features, elements, and / or operations of other technologies, and / or omitting or combining features, elements, and / or operations from the described embodiments.

[0139] Accordingly, it is intended that the following appended claims, and any claims thereafter introduced, be interpreted as including all such modifications, permutations, additions, omissions, and subcombinations that may reasonably be construed. The scope of the claims should not be limited by the preferred embodiments described in the examples, but should be accorded the broadest interpretation consistent with the description as a whole.

Claims

1. 1. A projector system comprising a plurality of projectors, the plurality of projectors including at least a first projector and a second projector arranged such that light projected by the first projector and a second projector are combined into a projected image for display, the first projector and the second projector having a plurality of different image characteristics selected from at least one of a dynamic range, a black level, and a peak brightness; a control system connected to receive image data defining image content to be projected by the projector system and to control the projector system to project the image content; the control system is configured to process the image data to generate modified image data for projection, the modified image data comprising a first modified image for projection by the first projector and a second modified image for projection by the second projector; the control system has an imaging model for the projector system, the control system being configured to obtain values ​​of the control parameters for the first projector and the second projector by performing an optimization of a plurality of control parameters using the imaging model; the plurality of control parameters include a plurality of display parameters for the first projector and the second projector; the optimizing comprises iteratively determining an error between an output image and a target image specified by the image data such that a sum of a number of cost functions is minimized, and calculating adjustments to the number of control parameters based on the error; the plurality of cost functions including a plurality of cost functions related to at least one of image fidelity, image quality, and a plurality of system constraints; Projector system.

2. 2. The projector system of claim 1, wherein the control system is configured to process the image data to determine a dynamic range, a black level, and a brightness level, and to generate the modified image data based on the dynamic range, the black level, and the brightness level.

3. The projector system of claim 2 , wherein the first projector has a higher dynamic range than the second projector and a higher peak brightness than the second projector.

4. 4. The projector system of claim 3, wherein if the modified image data has a plurality of pixels in a plurality of higher brightness areas that include a plurality of brightness values ​​higher than a maximum brightness of the second projector, the control system causes the first projector to project light in at least the plurality of higher brightness areas.

5. 5. The projector system of claim 4, wherein if the black level is above a black level threshold, the control system is configured to generate the second modified image for projection by the second projector to output as much light as possible from the second projector.

6. 6. The projector system of claim 3, wherein the control system is configured to generate the first modified image data for projection by the first projector by dilating and blurring a binary mask of pixels having luminance values ​​above a full-screen white value of the second projector.

7. 7. A projector system as claimed in any one of claims 3 to 6, wherein the control system is configured to generate the second modified image data for projection by the second projector by a method comprising clipping the luminance of pixels in the image data having luminance values ​​above a full-screen white value of the second projector.

8. 8. The projector system of claim 1 , wherein the control system is configured to provide the image data to the second projector unmodified if the dynamic range, black level and average brightness level are within the capabilities of the second projector.

9. 9. The projector system of claim 1 , wherein the second projector has a controllable aperture, and the control system is configured to control the aperture to reduce the black level of the second projector in at least some cases where the black level of the image data is less than the black level of the second projector.

10. 10. The projector system of claim 1, wherein a cost function related to image fidelity comprises a mean squared error or a mean absolute error value that represents how closely the projected image approximates an input target image.

11. 11. The projector system of claim 1, wherein the image quality related cost function has one or more heuristics that indicate how favorable a current set of control parameters is with respect to artifacts not modeled by the imaging model.

12. The projector system of claim 11 , wherein the heuristics include heuristics for one or more of moiré, color fringes, and diffraction artifacts.

13. A projector system as described in any one of claims 1 to 12, wherein the plurality of system constraints limit the selection of a plurality of values ​​of the plurality of control parameters so that the modified image data that the projector system is to be instructed to display when the plurality of control parameters are set is physically realizable by the projector system.

14. 14. The projector system of claim 1 , wherein the control system is configured to generate the modified image data for the first projector and the second projector based at least in part on a desired division of a total light output of the first projector and the second projector.

15. 14. The projector system of claim 1, wherein the control system controls one of the first projector and the second projector such that the one of the first projector and the second projector provides as much light as possible to the projected image.

16. The projector system of claim 1 , wherein the imaging model includes a heuristic distribution model.

17. 17. The projector system of claim 1 , wherein the first projector and the second projector have different primary colors, and the controller is configured to balance light output by the first projector and the second projector to achieve colors in the projected image that are outside the color gamut of at least one of the first projector and the second projector.

18. 18. The projector system of claim 1 , wherein the controller is configured to balance the light output by the first projector and the second projector to achieve optimized reproduction of high spatial frequency features in the image content of the projected image.

19. 19. The projector system of claim 1, wherein the control parameters include desired pixel values ​​of the first modified image data and the second modified image data.

20. 20. The projector system of claim 1, wherein the control parameters include desired pixel brightness values ​​for the first projector and the second projector.

21. 21. The projector system of claim 1, wherein the control system is configured to take into account ambient light in the area of ​​the projected image when determining the optimal ratio of light to be ultimately provided by each of the first and second projectors.

22. 2. The projector system of claim 1, wherein the control system comprises an image fidelity model configured to determine an error between an output image of the projector system and a target image identified by the image data, and to adjust multiple display parameters for the first projector and the second projector using the determined error.

23. 23. The projector system of claim 22, wherein the control system includes a system constraint model configured to limit a selection of values ​​of the control parameters such that the modified image data that the projector system is to be instructed to display when the control parameters are set is physically realizable by the projector system.

24. the control system comprises a quality heuristic model configured to compensate for one or more optical phenomena; 24. The projector system of claim 23, wherein the one or more optical phenomena include one or more of moire, color fringes, and diffraction artifacts.

25. 25. The projector system of claim 24, wherein the control system is configured to perform the optimization based on weighted outputs of the image fidelity model, the system constraint model, and the quality heuristic model.

Images