Light turning method for projector and light turning projector

Abstract

Example embodiments of the techniques provide efficient light steering systems and methods. An example method of steering light for a projector system includes illuminating an active surface of a light steering module. The active surface of the light steering module may include a plurality of individually controllable pixels. The method may also include determining a target steering phase pattern for each of a plurality of different regions of the active surface of the light steering module. The method may also include controlling the light steering module according to the respective target steering phase pattern for each region to improve light steering efficiency.

Description

Technical Field

[0001] The present disclosure relates to light steering projector systems and methods. Some embodiments provide efficient light steering projector systems and methods. Background Art

[0002] A projector system modulates a light beam to produce the desired image. In many cases, it is necessary to create a light field with a specified brightness profile. The applications of light projection systems are very wide, ranging from architectural lighting to the display of realistic images. The projected light pattern may be dynamic (for example, video) or static (for static images or static applications, such as a typical car headlight beam made of an arbitrarily shaped optical surface projected through a lens onto the road surface). Light may be projected onto a variety of screens and other surfaces, which may be flat or curved. Such surfaces may be fully reflective (for example, canvas projector screens used in movie theaters, walls, or buildings) or partially reflective (for example, the windshield of a car). Screens may be low gain or high gain, Lambertian or highly directional, and high contrast or low contrast. Light may be projected onto solid objects or onto a certain volume of medium (for example, fog).

[0003] Light efficiency and dynamic range are both primary concerns in commercial projector design. Even though most images require only a few localized, extremely bright highlights above the average picture level to appear realistic, high contrast and peak brightness are crucial for achieving superior image quality (brightness, color). On the other hand, projector systems should be highly efficient to minimize power consumption and simplify thermal management. This latter concern makes achieving extremely high peak brightness by increasing the power of the projector light source impractical.

[0004] Amplitude spatial light modulators (SLMs), commonly used to create the hue and color of an image by selectively blocking light through pixels, tend to have low optical efficiency because the blocked light is absorbed.

[0005] High dynamic range (HDR) image projection can be achieved by providing two or more light modulators. Many light modulators (e.g., LCD panels) generate the desired light field by subtraction (i.e., by absorbing unwanted light). Efforts have been made to create the desired light field by redistributing light.

[0006] There is a general desire to improve projector systems. Summary of the Invention

[0007] The present invention has multiple aspects. These aspects include, but are not limited to:

[0008] Systems and methods for projecting light patterns using light steering;

[0009] • Systems and methods for optimizing target light steering patterns.

[0010] One or more aspects described herein may be applied to, for example (but not limited to):

[0011] ●Cinema projector;

[0012] Virtual reality (VR) devices, such as VR glasses;

[0013] Virtual reality (AR) devices, such as AR glasses;

[0014] Head-up display

[0015] Car headlights;

[0016] ●Etc.

[0017] One aspect of the present invention provides a method for light steering for a projector. The method may include illuminating an active surface of a light steering module. The active surface of the light steering module may include a plurality of individually controllable pixels. The method may also include dividing the active surface into a plurality of different regions. Each region may include a corresponding subset of the plurality of individually controllable pixels. The method may also include determining a target steering phase pattern for each of the plurality of different regions of the active surface of the light steering module. The method may also include independently controlling each different region of the active surface based on the unique target steering phase pattern for each different region to improve light steering efficiency.

[0018] A "targeted steering phase pattern" refers to a specific phase pattern or phase grating applied to the entire spatial phase modulator or within a defined region thereof. When properly illuminated, this pattern is expected to produce a light field on a specified target plane (e.g., the surface of the spatial amplitude modulator). The goal is to align this generated light field as closely as possible with the intended target light field.

[0019] Determining a target steering phase pattern for each region may be based at least in part on a steering efficiency function for the region.

[0020] Determining a target steering phase pattern for each region may be based at least in part on a desired target light field.

[0021] Determining the target steering phase pattern for each region may include performing an optimization.

[0022] Performing the optimization may include constraining the optimization so that the sum of the contributions of all regions provides a desired target light field.

[0023] Performing the optimization may include determining a target steering phase pattern for each region in such a manner that each region preferentially contributes more light toward locations within the desired target light field that result in a smaller steering angle required to steer light away from the region.

[0024] Performing the optimization may include optimizing an objective function that, for each region, causes a target steering phase pattern to incur a relatively high cost when a relatively large steering angle is required to steer light from the region, and causes a relatively low cost when a relatively small steering angle is required to steer light from the region.

[0025] Performing the optimization may include optimizing the constraints so that each region contributes at least a certain fraction of the light at each location (eg, discrete point) in the desired target light field.

[0026] The advantage of setting optimization constraints (for example, requiring each region to contribute at least a certain fraction of light to every point in the target image) is that it ensures specific characteristics or qualities of the generated image. In summary, optimization constraints help guide the optimization process to produce images that meet specific standards or specifications, while also providing flexibility and efficiency in the optimization process.

[0027] This additional constraint requires that each region direct some light to each location within the target region. Although the method always prioritizes minimizing the steering angle, this additional constraint also requires that some light be directed to each target point.

[0028] The optimization may include iterative optimization.

[0029] Performing the optimization may include estimating the actual light field output from the light steering module when a particular target steering phase pattern is applied to various regions of the light steering module.

[0030] The actual light field can be estimated based on:

[0031]

[0032] Where x and y represent the positions in the light field, B(x, y) is the estimated actual light field, b is the index value across all N regions, and S b (x, y) represents the steering efficiency of region b for steering light toward point (x, y) in the actual light field, and P b (x,y) is the predicted contribution of region b to the actual light field without correction for steering efficiency.

[0033] The method may further comprise estimating S using the sum of basis functions b (x,y).

[0034] When compared to the size of the point spread associated with the projector, S b (x,y) can include smooth, slowly varying functions.

[0035] Performing the optimization may include compensating for one or more defocusing effects associated with the projector using one or more point spread functions (PSFs).

[0036] Compensating for one or more defocus effects may include effectively convolving the defocus with one or more PSFs.

[0037] Compensating for one or more defocus effects may include performing a deconvolution procedure based on the one or more PSFs.

[0038] Compensating for one or more defocusing effects may include at least partially accounting for physical limitations of the projector.

[0039] Performing the optimization may include estimating a deconvolved actual light field output by the light steering module when a particular target steering phase pattern is applied to various regions of the light steering module.

[0040] The actual output light field R(x,y) of the deconvolution can be determined based on:

[0041]

[0042] Among them, x and y represent the positions in the actual output light field of the deconvolution, b is the index value throughout all N regions, S b (x, y) represents the steering efficiency of region b for steering light to point (x, y) in the convolution actual output light field, and T b (x,y) represents the deconvolution steering target for region b (i.e., the light field corresponding to the contribution).

[0043] T b The determination of (x,y) can be based on:

[0044]

[0045] Where a is the index value across N regions, and K1, K2, ..., K N is a constant to be determined through optimization.

[0046] Performing the optimization may include constraining each region to have an equal amount (eg, power) of available incident light.

[0047] The method may also include providing each region with an equal amount (eg, power) of available incident light.

[0048] The method may further include adjusting the supply of incident light for each zone based on the availability of incident light.

[0049] The method may also include verifying whether the one or more target steering phase patterns fall within one or more system capabilities of the projector.

[0050] The method may further include adjusting at least one target steering phase pattern based on one or more system capabilities of the projector.

[0051] Another aspect of the present invention provides a light steering projector. The light steering projector may include a light steering module having an active surface. The active surface of the light steering module may include a plurality of individually controllable pixels. The light steering projector may also include at least one spatial light modulator configured to spatially modulate light from the light steering module. The light steering projector may also include a projector system configured to project the spatially modulated light. The light steering projector may also include a controller. The controller may be configured to divide the active surface into a plurality of different regions, each region including a corresponding subset of a plurality of individually controllable pixels. The controller may also be configured to determine a target steering phase pattern for each of the plurality of different regions of the active surface of the light steering module. Each region may include a corresponding subset of a plurality of individually controllable pixels. The controller may also be configured to control the light steering module according to the corresponding target steering phase pattern of each region to improve light steering efficiency. The controller may also be configured to independently control each different region of the active surface according to the unique target steering phase pattern of each different region to improve light steering efficiency.

[0052] The light redirecting projector can be configured to perform a method having any feature or combination of features described herein.

[0053] Another aspect of the present invention provides a data processing apparatus comprising means for performing a method having any feature or combination of features described herein.

[0054] Another aspect of the present invention provides a computer program product comprising instructions which, when the program is executed by a computer, cause the computer to perform a method having any feature or combination of features described herein.

[0055] Another aspect of the present invention provides a computer-readable medium having stored thereon the computer program product described herein.

[0056] Additional aspects and example embodiments are illustrated in the accompanying drawings and / or set forth in the following description.

[0057] It is emphasized that the invention relates to all combinations of the above-mentioned features, even if these are recited in different claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The accompanying drawings illustrate non-limiting example embodiments of the invention.

[0059] Figure 1 is a schematic diagram of an example optical path.

[0060] Figure 2 is a schematic diagram of an example active surface of a phase modulator of a light steering module.

[0061] Figure 3 is a block diagram illustrating a method according to an example embodiment of the present invention.

[0062] Figure 3A is a block diagram illustrating a method according to an example embodiment of the present invention.

[0063] Figure 4 is a schematic diagram of an example architecture of a light projection system.

[0064] Figure 5 is a graph showing the steering efficiency of the phase modulator plotted along the horizontal center line of the screen.

[0065] Figure 6 is a graph showing the steering efficiency of two real phase modulators and four virtual phase modulators (for a single primary color), where the bold line indicates the maximum value in each case. DETAILED DESCRIPTION

[0066] In order to facilitate a more thorough understanding of the present invention, specific details are set forth throughout the following description. However, the present invention can be practiced without these details. In other cases, well-known elements have not been shown or described in detail to avoid unnecessary obscurity of the invention. Accordingly, the description and drawings are to be regarded in an illustrative rather than a restrictive sense.

[0067] Figure 1 1 is a schematic diagram of an example optical path 10 of a light projection system that can project an image onto a screen or other surface. In some embodiments, the light projection system includes a high dynamic range (HDR) projector. In some embodiments, the HDR projector includes a projector with a contrast ratio greater than about 1 / 2000. In some embodiments, the contrast ratio of the HDR projector is about 1 / 5000. In some embodiments, the contrast ratio of the HDR projector is about 1 / 60000. In some embodiments, the brightness range of the HDR projector is about 5 milliters to about 300 nits.

[0068] The example optical path 10 includes at least one light steering module 12 that can be illuminated by incident light 13. The light steering module 12 redirects the incident light 13 to generate a desired light pattern on an image plane. Optionally, one or more downstream lenses (e.g., lenses 14 and 15) or other optical elements (e.g., mirrors, etc.) are configured to capture the redirected light and focus it onto at least one spatial light modulator (SLM) 16 (e.g., at least one amplitude light modulator, such as a digital micromirror device (DMD), a liquid crystal on silicon (LCoS) spatial light modulator, etc.). The SLM 16 typically includes an amplitude spatial light modulator. A projection lens 17 (or projector system) can collect the spatially modulated light and project it onto a screen (not shown).

[0069] The light steering module 12 is controllable to redirect incident light 13 incident on the light steering module 12. In some embodiments, the light steering module 12 includes one or more phase modulators. The one or more phase modulators can each include a plurality of individually controllable pixels, each of which can be operated to change the phase of the incident light. The pixels of the phase modulator can be arranged in a two-dimensional array.

[0070] In some embodiments, the light redirecting module 12 includes one or more liquid crystal on silicon (LCoS) phase modulators. In some embodiments, the light redirecting module 12 includes one or more micro-electromechanical systems (MEMS) phase modulators.

[0071] By controlling the pixels of the phase modulator to apply a phase shift pattern to the incident light, the incident light can be steered as needed. The phase pattern can be generated by processing image data that directly or indirectly specifies the brightness or relative light intensity as a function of position in the image to be projected. Each phase modulator (or each pixel of the phase modulator) can be controlled to steer the light according to the phase pattern, thereby steering the incident light to create a light field in which the light has a desired intensity distribution as a function of position. In some embodiments, the light is steered (for example, by a light steering module 12) as described in WO2015054797, entitled "Light Field Projector and Method".

[0072] The incident light 13 includes primary color light (eg, red light, green light, blue light), may be coherent light, or may be light emitted from one or more laser light sources.

[0073] In some embodiments, the projector system includes multiple optical paths 10. Each optical path 10 can correspond to a primary color (e.g., at least one optical path 10 and light redirection module 12 corresponds to red light, at least one optical path 10 and light redirection module 12 corresponds to green light, and at least one optical path 10 and light redirection module 12 corresponds to blue light). In some embodiments, the incident light 13 cycles through the primary colors (e.g., red, then green, then blue, then back to red, then green, then blue, etc.). In some embodiments, the optical paths 10 can include a single light redirection module 12.

[0074] Phase modulators typically exhibit a decrease in light steering efficiency as a function of the steering angle, i.e., the difference between the steering angle of the steered light and the steering angle of the non-steered light (e.g., when the light steering module 12 is set according to a uniform phase pattern). Specifically, the light steering efficiency typically decreases as the steering angle increases. By dividing the active surface of the phase modulator into multiple areas and independently controlling each area to display its own phase pattern to steer the incident light, the overall efficiency of the incident light 13 transmitted to the screen can be improved. This can be achieved by selecting a portion of the desired output light to be steered by each area in such a way that the steering angle tends to become smaller, thereby improving the overall light steering efficiency. For example, in some cases, dividing the active surface of the phase modulator can reduce the amount of light that is steered by larger steering angles and / or reduce the maximum steering angle used.

[0075] Steering efficiency can be measured experimentally to obtain a steering efficiency function, especially for conventional, unseparated steering. Figure 5 As shown, these functions can be obtained through a camera-based calibration process. Notably, analysis shows that due to the pre-tilt nature of the phase modulator, steering efficiency is not only optimal when pointing "straight ahead." The steering efficiency asymmetry caused by the opposing pre-tilts of the phase modulator provides an opportunity to improve steering efficiency. By exploiting this asymmetry, the present method introduces splitter steering, expanding the library of unique steering efficiency functions from two to many.

[0076] exist Figure 6 In [1], splitter plate steering is presented as a method to expand the number of unique steering efficiency functions from two to four, which can be blended to optimize the overall "average" steering efficiency. Figure 6This concept is illustrated, showing four steering efficiency functions derived from four virtual phase modulators used in spacer steering. Notably, these functions exhibit maxima at different locations across the screen, a result of the inherent pre-tilt property of the phase modulators. Remarkably, pre-tilt helps generate four maxima, doubling the capability compared to traditional steering methods. Furthermore, these maxima are evenly spaced across the screen, enhancing the versatility of steering options. The bold lines in the figure represent the maximum values ​​obtained from the four steering efficiency functions, significantly exceeding the maximum values ​​achievable using only two steering efficiency functions for a real phase modulator. When assuming a frame requires precisely positioned highlights that coincide with these four maxima, the best results can be achieved by steering one virtual panel for each highlight.

[0077] The active surface of the phase modulator can be divided into two or more regions. Generally, the active surface 22 of the phase modulator 20 of the light redirecting module 12 can be divided into a plurality of N regions, where N is any suitable number and N≥2. Figure 2 Regions 24-1, 24-2, 24-3, ..., 24-N are shown, collectively or generally referred to as regions 24. In some embodiments, each region 24 is the same size (i.e., each region has the same surface area and / or number of pixels). However, this is not required in all cases. In some embodiments, a first region 24 is larger than a second region 24 (i.e., the first region has a larger surface area than the second region). In addition, two regions 24 can have the same or different shapes.

[0078] In some embodiments, the number of regions or the characteristics of one or more regions (eg, boundaries, size, shape, etc.) changes dynamically in real time.

[0079] Figure 3 is a block diagram illustrating an example method 30 for optimizing the steering efficiency of the light steering module 12. In some embodiments, the method 30 is performed for each primary color of light.

[0080] In block 32, the active surface of the light steering module (or one or more phase modulators) 12 is divided (e.g., logically divided) into a plurality of regions (e.g., two regions) 24. As described elsewhere herein, pixels within each of the regions 24 may be individually controllable to display a desired phase pattern to accordingly steer incident light.

[0081] In some embodiments, the active surface of the light redirecting module 12 is divided into 16 or fewer regions (e.g., 2 regions) 24. In some embodiments, the active surface of the light redirecting module 12 is divided into 2 to 10 regions 24. In some embodiments, the number of regions 24 into which the active surface of the light redirecting module 12 is divided can be based on at least a tradeoff or comparison between the increased computational demands of having more regions 24 and the increased light redirection efficiency achieved by having more regions 24. In some embodiments, the number of regions 24 into which the active surface of the light redirecting module 12 is divided is based on an expectation of achieving a logical correspondence between the layout of the active surface and the light redirection function.

[0082] For example, a phase pattern comprising square pixels displayed by the light redirection module 12 generates a square light field. The square light field may be repeated periodically (e.g., due to higher order diffraction). In some embodiments, the phase pattern to be displayed is selected so that a portion of the generated light field is black, while the repeated portion of the light field remains square. In some embodiments, it is computationally reasonable to match the square end result of the light field with a square light source. The square light source can be provided, for example, by dividing the light redirection module 12 into square areas. In some embodiments, the size of the light redirection module 12 can be approximately ~2000x4000 pixels, in which case it is desirable to divide the light redirection module 12 into 2 square areas of ~2000x2000 pixels or 8 square areas of ~1000x1000 pixels.

[0083] In block 33, optimization parameters are determined. The optimization performed in method 30 determines the contribution to the target light field that will be delivered by steering light through each of the regions 24 in block 32. The optimization of method 30 seeks to select a contribution from each of the regions 24 such that:

[0084] The sum of the contributions of all regions 24 provides the target light field;

[0085] • For each of the regions 24 , the contribution is selected in the following manner: Each region 24 preferentially contributes more light to those locations within the target light pattern that require a smaller steering angle when steering light from the region 24 .

[0086] Since the steering angle required to redirect light to a point in the target light pattern may be different for different regions 24, it is possible to improve the overall efficiency of light delivery to the target light pattern by optimizing the contribution made by each region in the region 24, thereby reducing the light steering angle and improving the light steering efficiency overall.

[0087] In some embodiments, optimization constraints may be set in block 33. Optimization constraints may, for example, require that each region contribute at least a certain fraction of light at each location in the target image. In some embodiments, the optimization constraints of block 33 may be set as hard constraints (e.g., conditions that must be met for the optimization of method 30 to occur). In some embodiments, the optimization constraints of block 33 may be set as soft constraints (e.g., constraints that take the form of appropriate terms in the objective function). If the new target image (e.g., for a new frame) is the same as the previous target image of the previous frame, the optimization solution for the previous target image may be used for the new target image, and no new optimization needs to be performed.

[0088] In block 34, optimization is performed. The optimization may be an iterative optimization, in which the process of refining the contribution distribution among the various regions 24 is repeated multiple times (e.g., six times or until other termination criteria are met). The overall steering efficiency of any contribution distribution among the various regions 24 may be determined and used as an indicator of whether additional iterations should be performed (e.g., as a termination criterion for the iterative optimization in block 34), whether sufficient light steering efficiency has been achieved, etc. Block 35 may determine whether the optimization in block 34 has sufficiently converged to an optimal contribution distribution. In some embodiments, the optimization in block 34 has "sufficiently converged" (or "maximized" or "minimized") if the output of this optimization does not vary by more than a configurable threshold amount (e.g., less than 0.5%, 1%, 2%, 5%, etc.). For example, the optimization in block 34 may have sufficiently converged if the output average light steering efficiency weighted against the target light steering efficiency and / or the actual contribution of each region 24 does not change by more than a configurable threshold amount in each successive iteration. In some embodiments, the optimization has "sufficiently converged" (or has been "maximized" or "minimized") if, in each successive iteration, one or more inputs to the optimization of block 34 do not change by more than a configurable threshold amount. In some embodiments, the block 35 convergence criteria may include a configurable threshold number of iterations of the block 34 optimization. If the block 34 optimization has not sufficiently converged, the method 30 returns to block 34 to perform another optimization cycle. Otherwise, the method 30 proceeds to block 36.

[0089] By considering the optimized contribution of each region 24 as the target light pattern for that region, a phase pattern 36A to be displayed by the corresponding region 24 of the light steering module 12 can be generated in block 36. In some embodiments, the phase pattern 36A generated for each region is stored, for example, in a data repository (e.g., data repository 55 described elsewhere herein). In some embodiments, the phase patterns 36A for each region can be combined to generate, for example, control input data for the light steering module 12, and such data can be used to control the light steering module 12.

[0090] Optional block 37 verifies whether the optimized result (e.g., phase pattern 36A) falls within the capabilities of the particular projection system being used. For example, block 37 can verify whether the light projection system is capable of providing the optimized amount of light (e.g., according to phase pattern 36A). If the optimized result is not within the capabilities of the projection system, method 30 can proceed to block 38, where the optimized result (and / or the generated phase pattern 36A) is adjusted based on the capabilities of the projection system. Otherwise, method 30 proceeds to block 39 and terminates. In some embodiments, in addition to or as an alternative to block 37, the capabilities of the particular projection system being used can be incorporated as a constraint into the optimization process of blocks 34 / 35.

[0091] Figure 3A is a block diagram illustrating an example method 30A for optimizing the steering efficiency of a light steering module.

[0092] Method 30A is identical to method 30, except that method 30A includes block 40. Block 40 determines whether it is necessary or desirable to perform the optimization described herein for a particular target light field. For example, block 40 may determine whether to perform the optimization described above based on the maximum light intensity corresponding to highlights in the image to be displayed and / or the total amount of light required to provide highlights in the image to be displayed with an intensity exceeding a given threshold. If block 40 determines that optimization is not desirable or desirable, method 30A may terminate at block 41 and generate and apply a phase pattern for the entire phase modulator 12. Otherwise, method 30A proceeds to block 32.

[0093] In some embodiments, the light steering efficiency is determined based on a detailed physics-based model of the light projection system (e.g., performed as part of the query at block 40 and / or otherwise performed during the execution of method 30 or 30A). For example, the model may include parameters such as the finite pixel size of the light steering module (one or more phase modulators) 12, parameters characterizing one or more optical components of the system, etc. In some embodiments, the light steering efficiency is determined experimentally. For example, the light steering efficiency can be determined experimentally using a camera-based or photodiode-based calibration procedure.

[0094] In one example, when a particular phase pattern 36A is applied to each region 24 of the phase modulator, an estimate of the actual light field is generated that takes into account the steering efficiency of the phase modulator of the light steering module 12 and is calculated as follows:

[0095]

[0096] Where x and y represent the position in the output light field, B(x,y) is the estimated output light field, b is the index value across all N regions, and S b(x, y) represents the steering efficiency of region b for steering light toward point (x, y) in the output light field, and P b (x,y) is the predicted contribution of region b to the output light field without correction for steering efficiency. b (x,y) may be determined from a predicted image at infinity (eg, by Fourier transform) of the phase pattern applied to region b, followed by convolution with a point spread function (PSF).

[0097] The example of formula (1) does not take into account any non-steerable light (e.g., light that cannot be steered by the phase modulator of the light steering module 12) or any base light (e.g., non-steerable light added to the steered light). For example, the non-steerable light and / or base light may have been subtracted before. Compared with the size of the PSF, S b (x,y) is usually a smooth, slowly varying function. In some cases, S b (x,y) is approximated by the sum of basis functions (e.g., sine and cosine functions, polynomials, etc.).

[0098] Real optical systems and / or light sources may have a limited ability to resolve points and / or focus light onto a point source. At least one (but typically multiple) effects that limit the ability of an optical system or light source to resolve points and / or focus light onto a point source can be mathematically described (or encapsulated) by one or more point spread functions (PSFs). One or more PSFs can be convolved with one or more ideal system images to estimate one or more real system images.

[0099] In a light steering system, one or more PSFs may be large. For example, it may be desirable to computationally pre-compensate for the defocusing effects of one or more PSFs. For example, pre-compensation may cancel out any effective convolution (e.g., physical effects rather than computational effects) with one or more PSFs. To perform pre-compensation, R(x,y) and T may be used. b (x,y), which are B(x,y) and P respectively. b The deconvolution version of (x,y). Since R(x,y) and T b (x,y) is deconvolved (pre-compensated for PSF effects), so based on R(x,y) and T b (x,y) Drive The systems described herein (eg, light steering module 12) can advantageously produce a light field that is substantially close to (or identical to) B(x,y).

[0100] In some embodiments, the deconvolution takes into account physical limitations. For example, the deconvolution can take into account the fact that the light field may only have positive values.

[0101] The deconvolved output light field R(x,y) (excluding any basis light and / or unsteered light) can be expressed as follows:

[0102]

[0103] Among them, T b (x,y) represents the deconvolution steering target of region b (i.e., the light field corresponding to the contribution).

[0104] Given R(x,y) and the steering efficiency function S of each area in the region (b=1,…,N) b (x,y), a steering target (or “contribution”) T can be chosen for each region b. b (x, y), so that in accordance with "each (x, y) point throughout b's S b (x,y)T b Under the requirement that the sum of the N regions (x, y) is R(x, y)” (this requirement can be called “Constraint 1” and can be expressed in words as: the steering light from all N regions together produces the desired output light field), the average value of the effective steering efficiency is maximized relative to the desired output light field R(x, y).

[0105] In this example, it is assumed that the intensity of the light incident on the phase modulator of the light steering modulator 12 is uniform. That is, each region b may receive 1 / N of the available incident light. Typically, the incident light is close to uniform. If deviations from the uniformity of the incident light are significant enough to be of concern, such deviations can be absorbed in the measured steering efficiency function, or the intensity of the incident light can be explicitly considered in the calculation as a function of the position on the phase modulator. In some cases, region (or regions) b may have no (i.e., zero) incident light.

[0106] As discussed elsewhere, the incident light 13 may include light of different primary colors. According to equation (2), a set of optimized steering targets may be selected for each primary color.

[0107] At this point, the amount of light available is preferably not fixed during the determination process. Rather, a determination is made as to how much light is needed, and during downstream processing, the determined amount can be compared with the actual amount of light available. Based on the comparison, appropriate actions can be taken (e.g., clipping some pixels, over-steering light, steering some light off-screen, etc.). Such actions can be performed, for example, as part of optional block 38 described elsewhere herein.

[0108] As a non-limiting example, if it is determined that there is more light available than required, then:

[0109] • The output light field B(x,y) can be scaled up (eg, on a screen).

[0110] • Some of the excess light can be dumped. Dumping some of the excess light can advantageously reduce black levels (e.g., make dark areas appear darker), thereby increasing the effective contrast ratio of the system. In some embodiments, some of the light is dumped using apertures that absorb light (and associated heat load).

[0111] ●And so on.

[0112] In some embodiments, for region b, assume that T b (x,y) has the following form:

[0113]

[0114] Where a is the index value of N regions; K1, K2, ..., K N is a constant to be determined by optimization (eg, the optimization of method 30, 30A), and β is a power exponent, which is ≥ 1. In such embodiments, the optimization includes setting K1, K2, ..., K N The value that produces the best overall steering efficiency is selected. Equation (3) can have the following characteristics:

[0115] ●T b (x,y) and S b (x, y) are positively correlated (i.e., each region is preferably controlled to steer light toward a location in the output light field that gives that region greater steering efficiency). The strength of this attraction is set by the parameter β: larger values ​​of β result in more perfect solutions, but at the expense of making them harder to find.

[0116] • Constraint 1 is automatically satisfied when the contribution of each region b is defined by formula (3).

[0117] ·It can be achieved by calculating a set of constants K1, K2, ..., K N The value of is chosen appropriately to adjust the contribution of each region b (e.g., T b (x,y)) so that each contribution involves the same amount of light ("Constraint 2").

[0118] The total power of light incident on area b is I b It can be expressed, for example, as follows:

[0119] I b =∫T b (x,y)dxdy (4).

[0120] Constraint 2 can be expressed, for example, as follows:

[0121]

[0122] Formulas (2) to (5) (eg, four equations in four unknowns) can be solved, for example, iteratively.

[0123] For example, I b Can be considered as:

[0124] I b =K b G b (6)

[0125] in,

[0126]

[0127] thus,

[0128]

[0129] The iterative solution can be expressed, for example, as follows:

[0130]

[0131] where i is the iteration index; and is used Given by formula (7). The computational cost may, for example, consist primarily of the computational cost of evaluating the integral of formula (7) (e.g., one integral per region per iteration). Initially, for example, the constants K1, K2, ..., K N It can be set as follows:

[0132] In some embodiments, a convergence parameter f can be introduced. The convergence parameter f can, for example, allow a trade-off or optimization between stability (e.g., achieved by selecting a smaller f) and convergence speed (e.g., achieved by selecting a larger f). The iterative solution with the convergence parameter f can, for example, be expressed as follows:

[0133]

[0134] In some cases, β = 8. In some cases, the convergence parameter f = 1. In some cases, 6 iterations are performed. In some cases, β ranges from 1 to 10. In some cases, the convergence parameter f is less than or equal to 2. In some cases, the convergence parameter ● ranges from 0.5 to 1. In some cases, the number of iterations performed ranges from 1 to 10.

[0135] In some embodiments, formula (3) is replaced by the following:

[0136]

[0137] Figure 4An example architecture of a light projection system 50 is shown. A controller 52 controls the light steering module 12 and the spatial light modulator 16. A data repository 54 can store image data 55 to be displayed by the light projection system 50. In some embodiments, the image data 55 is dynamically processed (e.g., by the controller 52) to generate a phase pattern (e.g., phase pattern 36A) for the light steering module 12. In some embodiments, the image data 55 is at least partially pre-processed (i.e., before displaying a moving picture, etc.) to generate a phase pattern (e.g., phase pattern 36A) for the light steering module 12. In some embodiments, the generated phase pattern 36A is stored in the data repository 54.

[0138] In some embodiments, the controller 52 combines the phase patterns corresponding to the regions 24 (e.g., phase pattern 36A) into one or more phase patterns corresponding to the entire active surface of the light redirecting module 12 (or one or more phase modulators). In some embodiments, the phase pattern for a region 24 includes multiple combined phase patterns. In some embodiments, each of the multiple phase patterns for the region 24 is identical to another phase pattern in the multiple phase patterns.

[0139] It is not necessary for all areas (e.g., each area 24 and / or each area b described herein) to be located on the same phase modulator or the same light steering module. Some projectors may include two or more phase modulators or light steering modules for each color channel. It may be desirable to include two or more phase modulators to reduce the power of the light redirected by each phase modulator, especially when high output power is desired. The above method can be applied to improve light steering efficiency when multiple areas are distributed across two or more phase modulators or light steering modules.

[0140] When this document refers to a component (e.g., a software module, a processor, a component, a device, a circuit, etc.), unless otherwise specified, reference to the component (including reference to a "mechanism") should be interpreted as including any component that performs the function of the component as an equivalent of the component (i.e., functionally equivalent), and also including components that are not structurally equivalent to the structure disclosed in the example embodiments shown in the present invention for performing the function.

[0141] Embodiments of the present invention may be implemented using specially designed hardware, configurable hardware, a programmable data processor configured by providing software (which may optionally include "firmware") that can be executed on the data processor, a special-purpose computer or data processor that is specially programmed, configured, or constructed to perform one or more steps of the methods explained in detail herein, and / or a combination of two or more thereof. Examples of specially designed hardware are: logic circuits, application specific integrated circuits ("ASICs"), large-scale integrated circuits ("LSIs"), very large-scale integrated circuits ("VLSIs"), etc. Examples of configurable hardware are: one or more programmable logic devices, such as programmable array logic ("PALs"), programmable logic arrays ("PLAs"), and field programmable gate arrays ("FPGAs"). Examples of programmable data processors are: microprocessors, digital signal processors ("DSPs"), embedded processors, graphics processors, mathematical coprocessors, general-purpose computers, server computers, cloud computers, mainframe computers, computer workstations, etc. For example, one or more data processors in the control circuitry of a device may implement the methods described herein by executing software instructions in a program memory accessible by the processor.

[0142] Processing can be centralized or distributed. In the case of distributed processing, information including software and / or data can be stored centrally or distributedly. Such information can be exchanged between different functional units via a communication network (e.g., a local area network (LAN), a wide area network (WAN), or the Internet), wired or wireless data links, electromagnetic signals, or other data communication channels.

[0143] The present invention can also be provided at least in part in the form of a program product. The program product can include any non-transient medium that carries a set of computer-readable instructions that, when executed by a data processor, cause the data processor to perform the method of the present invention. The program product according to the present invention can be in any of a variety of forms. The program product can include, for example, non-transient media such as magnetic data storage media (including hard disk drives), optical data storage media (including CDROMs, DVDs), electronic data storage media (including ROMs, flash RAMs, EPROMs), hard-wired or pre-programmed chips (such as EEPROM semiconductor chips), nanotechnology memories, etc. The computer-readable signals on the program product can be optionally compressed or encrypted.

[0144] In some embodiments, the present invention may be implemented at least in part in software. The software may, for example, run on a commercially available graphics processor unit (GPU). For greater clarity, "software" includes any instructions executed on a processor and may include (but is not limited to) firmware, resident software, microcode, code for configuring configurable logic circuits, applications, application software, and the like. As known to those skilled in the art, both processing hardware and software may be centralized or distributed (or a combination thereof) in whole or in part. For example, the software and other modules may be accessed through local memory, a network, a browser, or other applications in a distributed computing environment, or other device segments suitable for the above purposes.

[0145] The Software and other modules may be located on servers, workstations, personal computers, tablets and other devices suitable for the purposes described herein.

[0146] Definition of terms

[0147] Throughout the specification and claims, unless the context clearly requires otherwise:

[0148] ● “including”, “having”, etc. should be understood in an inclusive sense rather than an exclusive or exhaustive sense; that is, in the sense of “including but not limited to”;

[0149] ● "Connected", "coupled" or any variation thereof refers to any direct or indirect connection or coupling between two or more elements; the coupling or connection between elements can be physical, logical, or a combination thereof;

[0150] "Herein," "above," "hereunder," and words of similar meaning when used to describe this specification shall refer to this specification as a whole and not to any particular part of this specification;

[0151] ● “or” when referring to a list of two or more items includes all of the following interpretations of the word: any of the items in the list, all of the items in the list, and any combination of the items in the list;

[0152] ● The singular forms "a", "an", and "the" also include the meaning of any appropriate plural forms. These terms ("a", "an", and "the") refer to one or more unless otherwise specified;

[0153] ● "and / or" is used to indicate that one or both of the stated situations may occur, for example, A and / or B includes (A and B) and (A or B) both;

[0154] ●When applied to numerical values, “approximately” means the value ±10%;

[0155] ● When a feature is described as "optional" or "optionally" present, or as present "in some embodiments", the disclosure is intended to encompass embodiments in which the feature is present and other embodiments in which the feature is not necessarily present, as well as other embodiments in which the feature is excluded. Further, when describing any combination of features in this application, this statement is intended to serve as an antecedent basis for the use of exclusive terminology (e.g., "solely," "only," etc., in relation to the combination of features) and the use of "negative limitations" to exclude the presence of other features; and

[0156] ● “First” and “Second” are used for descriptive purposes and should not be understood as indicating or implying relative importance.

[0157] or indicate the quantity of the technical features referred to.

[0158] As used in this specification and any appended claims (where present), directional terms such as "vertical," "lateral," "horizontal," "upward," "downward," "forward," "backward," "inward," "outward," "left," "right," "front," "backward," "top," "bottom," "below," "above," "below," and the like, will depend on the particular orientation of the device being described and illustrated. The subject matter described herein can assume a variety of alternative orientations. Accordingly, these directional terms are not strictly defined and should not be interpreted narrowly.

[0159] Where a numerical range is stated, the range includes all subranges of that range. Unless the context clearly dictates otherwise, or any portion of the range is expressly excluded, the statement of a range is intended to support the values ​​at the endpoints of the range, at any intervening value up to the tenth of the lower limit of the range, and at any subrange or collection of subranges of that range. Where the range includes one or both endpoints, ranges excluding one or both of those endpoints are also encompassed by the present invention.

[0160] Certain numerical values ​​described herein are preceded by "about." In such cases, "about" literally refers to the exact numerical value, the exact numerical value ±5%, and all other numerical values ​​close to or approximately equal to the numerical value. Unless otherwise indicated, a particular numerical value is included in the "about" specific enumerated numerical value, wherein the particular numerical value is substantially equivalent to the specific enumerated numerical value in the presence of the specific enumerated numerical value. For example, a statement that something has a numerical value of "about 10" should be understood as a collection of the following statements:

[0161] ● In some embodiments, the value is 10;

[0162] In some embodiments, the value ranges from 9.5 to 10.5;

[0163] And if, based on the context, one of ordinary skill in the art would understand that the values ​​within a certain range are substantially equivalent to 10 (because the values ​​within the range would be understood to provide substantially the same results as the value 10), then "about 10" also includes:

[0164] In some embodiments, the range of values ​​is C to D, where C and D are the lower and upper endpoints of the range, respectively, and the range includes all values ​​substantially equivalent to the value 10.

[0165] For purposes of illustration, specific examples of systems, methods, and apparatus are described herein. These are merely examples. The techniques provided herein may be applied to systems other than the example systems described above. In the practice of the present invention, numerous changes, modifications, additions, omissions, and substitutions may be made. The present invention encompasses variations of the described embodiments that will be apparent to those skilled in the art, including variations obtained by replacing features, elements, and / or actions with equivalent features, elements, and / or actions; mixing and matching features, elements, and / or actions from different embodiments; combining features, elements, and / or actions from the embodiments described herein with features, elements, and / or actions from other technologies; and / or omitting combinations of features, elements, and / or actions from the described embodiments.

[0166] It will be apparent to those skilled in the art after reading this disclosure that the individual embodiments described and illustrated herein each have discrete components and features that can be readily separated or combined with the features of any other described embodiment without departing from the scope of the invention.

[0167] Any aspect described above in relation to the apparatus may also apply to the method, and vice versa.

[0168] Any method described can be implemented in the order of events described or in any other order that is logically possible. For example, although the processes or blocks exist in a given order, alternative examples can perform routines with steps in a different order or use systems with blocks in a different order, and some processes or blocks can be deleted, moved, added, subdivided, combined and / or modified to provide alternative combinations or sub-combinations. Each of these processes or blocks can be implemented in a variety of different ways. Moreover, although processes or blocks are sometimes shown as being performed in series, these processes or blocks can be performed in parallel, simultaneously, or at different times.

[0169] Various features are described herein as being present in “some embodiments”. Such features are not mandatory and may not be present in all embodiments. Embodiments of the present invention may include zero, any one, or any combination of two or more of such features. Even if such features are shown in different figures and / or described in different sections or paragraphs, the present disclosure contemplates all possible combinations of such features. This is limited to the extent that certain features of such features are incompatible with other features of such features, because it is impossible for a person of ordinary skill in the art to construct a practical embodiment that combines such incompatible features. Therefore, the description that “some embodiments” have feature A and “some embodiments” have feature B should be interpreted as explicitly indicating that the inventor also contemplates embodiments that combine features A and B (unless otherwise stated in the description or features A and B are not compatible at all). This is true even if feature A and feature B are illustrated in different figures and / or mentioned in different paragraphs, sections or sentences.

[0170] It is therefore intended that the following appended claims and claims introduced thereafter be interpreted to include all such modifications, permutations, additions, omissions, and sub-combinations as can reasonably be inferred. The scope of the claims should not be limited to the preferred embodiments listed in the examples, but should be given the broadest interpretation consistent with the entire description of the specification.

Claims

1. A method for redirecting light for a projector, comprising: illuminating an active surface of a light redirecting module, the active surface of the light redirecting module comprising a plurality of individually controllable pixels; dividing the active surface into a plurality of distinct regions, each region comprising a respective subset of the plurality of individually controllable pixels; A target steering phase pattern is determined for each of the multiple different areas of the active surface of the light redirecting module, and each different area of ​​the active surface is independently controlled according to its unique target steering phase pattern to improve light redirection efficiency.

2. The method according to claim 1, wherein Determining the target steering phase pattern for each region is based at least in part on a steering efficiency function for that region.

3. The method according to claim 2, wherein: Determining the target steering phase pattern for each region is based at least in part on a desired target light field.

4. The method according to any one of claims 1 to 3, wherein Determining the target steering phase pattern for each region includes performing an optimization.

5. The method according to claim 4, wherein Performing the optimization includes constraining the optimization so that the sum of contributions from all regions contributes to the desired target light field.

6. The method according to claim 4 or 5, wherein: Performing the optimization includes determining the target steering phase pattern for each region in such a manner that each region preferentially contributes more light to locations within the desired target light field that result in a smaller steering angle required to steer light from the region.

7. The method according to claim 4 or 5, wherein: Performing the optimization includes optimizing an objective function, wherein for each region, the objective function causes a target steering phase pattern to incur a relatively high cost when a relatively large steering angle is required to redirect light from the region, and the objective function causes a relatively low cost when a relatively small steering angle is required to redirect the light from the region.

8. The method according to any one of claims 4 to 6, wherein Performing the optimization includes constraining the optimization so that each region contributes at least a certain proportion of light at each location in the desired target light field, the location being, for example, a discrete point.

9. The method according to any one of claims 4 to 8, wherein: The optimization includes iterative optimization.

10. The method according to any one of claims 4 to 9, wherein Performing the optimization includes estimating the actual light field output from the light steering module when a particular target steering phase pattern is applied to various regions of the light steering module.

11. The method according to claim 10, wherein: The actual light field estimation is based on: Where x and y represent the positions in the light field, B(x, y) is the estimated actual light field, b is the index value across all N regions, and S b (x, y) represents the steering efficiency of region b for steering light to point (x, y) in the actual light field, and P b (x,y) is the predicted contribution of region b to the actual light field without correction for steering efficiency.

12. The method of claim 11 , comprising estimating S using a sum of basis functions b (x,y).

13. The method according to claim 11 or 12, wherein: When compared to the size of the point spread associated with the projector, S b (x,y) includes smooth slowly varying functions.

14. The method according to any one of claims 4 to 9, wherein: Performing the optimization includes compensating for one or more defocusing effects associated with the projector using one or more point spread functions (PSFs).

15. The method according to claim 14, wherein Compensating for the one or more defocus effects includes canceling an effective convolution with the one or more PSFs.

16. The method according to any one of claims 14 to 15, wherein Compensating for the one or more defocus effects includes performing a deconvolution procedure based on the one or more PSFs.

17. The method according to any one of claims 14 to 16, wherein Compensating for the one or more defocusing effects includes at least partially accounting for physical limitations of the projector.

18. The method according to any one of claims 14 to 17, wherein Performing the optimization includes estimating a deconvolved actual light field output by the light steering module when a specific target steering phase pattern is applied to various regions of the light steering module.

19. The method according to claim 18, wherein The actual output light field R(x,y) of the deconvolution is determined based on: Where x and y represent the positions in the actual output light field of the deconvolution, b is the index value across all N regions, S b (x, y) represents the steering efficiency of region b for steering light to point (x, y) in the deconvolved actual output light field, and T b (x,y) represents the deconvolution steering target for region b.

20. The method according to claim 19, wherein Determine the T b (x,y) is based on: Where a is the index value of N regions; K1, K2, ..., K N is a constant to be determined by the optimization; and wherein β is a power parameter ≥ 1.

21. The method according to any one of claims 1 to 20, wherein Performing the optimization involves constraining each region to have an equal amount of available incident light, such as power.

22. A method according to any one of claims 1 to 21, comprising providing each region with an equal amount of available incident light, such as power.

23. A method according to any one of claims 1 to 22, comprising adjusting the incident light provision for each zone based on the availability of incident light.

24. The method of any one of claims 1 to 23, comprising verifying whether one or more of the target steering phase patterns fall within one or more system capabilities of the projector.

25. The method of claim 24, comprising adjusting at least one of the target steering phase patterns based on the one or more system capabilities of the projector.

26. A light redirecting projector, comprising: a light redirecting module having an active surface, wherein the active surface of the light redirecting module includes a plurality of individually controllable pixels; at least one spatial light modulator configured to spatially modulate light from the light steering module; a projector system configured to project the spatially modulated light; as well as A controller, wherein the controller is configured as: dividing the active surface into a plurality of distinct regions, each region comprising a respective subset of the plurality of individually controllable pixels; determining a target steering phase pattern for each of a plurality of different regions of an active surface of the light steering module; as well as Each different region of the active surface is independently controlled according to a unique target steering phase pattern for each different region to improve light steering efficiency.

27. The light redirecting projector of claim 26, configured to perform the method of any one of claims 2 to 25.

28. A data processing apparatus comprising means for executing the method according to any one of claims 1 to 25.

29. A computer program product comprising instructions which, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 25.

30. A computer readable medium having stored thereon the computer program product of claim 29.

Images