Projection system and method having dynamic target geometry

Abstract

A projection system and method includes a light source configured to emit light in response to image data; a phase light modulator configured to receive the light from the light source and apply a spatially varying phase modulation to the light, thereby steering the light and generating a projection light; and a controller configured to dynamically determine a target geometry of a projection surface onto which the projection light is projected based on at least one of a user input or a sensor signal, determine a phase configuration for a frame of the image data based on the target geometry, and provide a phase control signal to the phase light modulator, the phase control signal being configured to cause the phase light modulator to generate the projection light in accordance with the phase configuration for the frame.

Description

Technical Field

[0001] Cross - References to Related Applications This application claims priority to U.S. Provisional Application No. 63 / 144,027, filed on February 1, 2021, and European Patent Application No. 21154553.8, each of which is hereby incorporated by reference in its entirety.

[0002] 1. Field of the Disclosure This application generally relates to projection systems and methods for driving a projection system.

Background Art

[0003] 2. Description of Related Art Digital projection systems typically utilize a light source and an optical system to project an image onto a surface or screen. The optical system includes components such as mirrors, lenses, waveguides, optical fibers, beam splitters, diffusers, spatial light modulators (SLMs), etc. Some projection systems are based on an SLM that performs spatial amplitude modulation. In such systems, the light source provides a light field that embodies the brightest level that can be reproduced on the image, and the light is attenuated (e.g., discarded) to generate the desired scene level. In such a configuration, light that is not projected to form any part of the image is attenuated or discarded. The projected light passes through a lens on one side of the projection system. The projection system aligns with a predetermined screen area by adjusting elements of the projection system (e.g., the position of the system itself, the configuration of various lens optical systems, etc.) to ensure that the screen is perpendicular to the optical axis of the projector and the projected image is in focus.

[0004] Thus, comparative digital projection systems require a static screen geometry. If the screen geometry changes (e.g., if the position or orientation of the projection system changes intentionally or unintentionally), the image can change. [Overview of the Initiative] [Problems that the invention aims to solve]

[0005] Various aspects of this disclosure relate to circuits, systems, and methods for projection displays that use phase modulation to provide image projection onto one or more surfaces having dynamic geometry. [Means for solving the problem]

[0006] In one exemplary aspect of the present disclosure, a projection system is provided comprising: a light source configured to emit light in response to image data; a phase light modulator configured to receive light from the light source and apply spatially variable phase modulation to the light, thereby steering the light and generating projected light; and a controller configured to dynamically determine a target geometry of a projection surface onto which the projected light is projected, based on at least one of a user input or a sensor signal, determine a phase configuration for a frame of image data based on the target geometry, and provide a phase control signal to the phase light modulator, the phase control signal configured to cause the phase light modulator to generate the projected light according to the phase configuration for the frame.

[0007] In another exemplary aspect of the present disclosure, a projection method is provided, comprising the steps of: emitting light from a light source in response to image data; receiving light by a phase light modulator; applying spatially variable phase modulation to the light by the phase light modulator, thereby steering the light to generate projected light; dynamically determining a target geometry of a projection surface onto which the projected light is projected, based on at least one of user inputs or sensor signals; determining a phase configuration for a frame of image data based on the target geometry; and providing a phase control signal to the phase light modulator, the phase control signal being configured to cause the phase light modulator to generate projected light according to a phase configuration for a frame. [Brief explanation of the drawing]

[0008] These and other more detailed and specific features of various embodiments are more fully disclosed in the following description with reference to the attached drawings.

[0009] [Figure 1] This disclosure shows a block diagram illustrating an exemplary projection system from various aspects.

[0010] [Figure 2] This disclosure illustrates an exemplary phase modulator in various aspects.

[0011] [Figure 3] This disclosure illustrates another exemplary phase modulator from various aspects.

[0012] [Figure 4] This figure shows exemplary settings for a projection system from various aspects of the present disclosure.

[0013] [Figure 5] This disclosure presents another exemplary setting for a projection system from various aspects.

[0014] [Figure 6] This disclosure illustrates an exemplary process flow from various aspects.

[0015] [Figure 7] This document illustrates another exemplary process flow from various aspects of this disclosure. [Modes for carrying out the invention]

[0016] The present disclosure and aspects thereof can be embodied in various forms including hardware or circuitry controlled by a computer-implemented method, a computer program product, a computer system and network, a user interface, as well as an application programming interface, and hardware implementation methods, signal processing circuits, memory arrays, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), etc. The foregoing summary is only intended to give a general idea of the various aspects of the present disclosure and is in no way intended to limit the scope of the present disclosure.

[0017] In the following description, numerous details, such as circuit configurations, timings, operations, etc., are described to provide an understanding of one or more aspects of the present disclosure. It will be readily apparent to those skilled in the art that these specific details are merely exemplary and are not intended to limit the scope of the present application.

[0018] Furthermore, although the present disclosure mainly focuses on examples where various circuits are used in a digital projection system, it will be understood that this is merely an example of an implementation. It will be further understood that the disclosed systems and methods can be used in any device that needs to project light, such as in movie theaters, consumer and other commercial projection systems, head-up displays, virtual reality displays, etc.

[0019] Projector System

[0020] In a projection system based on SLM, the device must be accurately aligned with a given screen area. The device position and lens optics must be specifically adjusted to ensure that the image is accurately aligned on the screen (e.g., the screen must be perpendicular to the optical axis of the device) and in focus. If the projector is moved intentionally or accidentally, the alignment may be disrupted. In an environment such as for home or office use, this can happen frequently. Furthermore, such projection systems typically require the screen to be flat, and any change or deviation from the flatness of the screen can result in distortion of the projected image. Even in a cinema environment where the device is large and typically placed in a dedicated room, technicians spend a significant amount of time aligning the device with the screen and / or ensuring the flatness of the screen. In an environment where multiple projection devices are used on a single screen (e.g., 3D movie projection), this time increases accordingly.

[0021] The use of a phase light modulator (PLM) can mitigate these problems. For example, while a comparative projection system operates by blocking a common light source at each pixel, a PLM-based projection system operates by steering light from areas where it is desirable to be darker to areas where it is intended to be brighter. Furthermore, based on the architecture of the PLM itself, such a projection system can have a significantly faster response time (i.e., the time required to change the projected image) compared to a comparative projection system.

[0022] Figure 1 shows a block diagram of an exemplary projection system 100 in various aspects of the present disclosure. Specifically, Figure 1 shows a projection system 100 comprising: a light source 101 configured to emit a first light 102; an illumination optical system 103 configured to receive the first light 102, redirect or otherwise modify the first light to generate a second light 104; a PLM 105 configured to apply spatially varying phase modulation to the second light 104 to steer the second light 104 and generate a third light 106; a first projection optical system 107 configured to receive the third light 106, redirect or otherwise modify the third light to generate a fourth light 108; a filter 109 configured to filter the fourth light 108 to generate a fifth light 110; and a second projection optical system 111 configured to receive the fifth light 110 and project it onto a screen 113 as a sixth light 112.

[0023] The projection system 100 further includes a controller 114 configured to control various components of the projection system 100, such as a light source 101 and / or a PLM 105. In some implementations, the controller 114 may additionally or alternatively control other components of the projection system 100, including but not limited to the illumination optics 103, the first projection optics 107, and / or the second projection optics 111. The controller 114 may be one or more processors, such as the central processing unit (CPU) of the projection system 100. The illumination optics 103, the first projection optics 107, and the second projection optics 111 may each include one or more optical components, such as mirrors, lenses, waveguides, optical fibers, beam splitters, and diffusers. Furthermore, although Figure 1 shows a single modulator, the projection system 100 may include other modulators in addition to the PLM 105. For example, the first projection optics 107 may include an amplitude-based SLM that can be controlled by the controller 114. Except for the screen 113, the components shown in Figure 1 may be integrated into the housing to provide a projection device. Such a projection device may include additional components such as memory, input / output ports, communication circuits, and a power supply.

[0024] The light source 101 may be, for example, a laser light source. Generally, the light source 101 is any light emitter that emits coherent light. In some aspects of this disclosure, the light source 101 may have a plurality of individual light emitters, each corresponding to a different wavelength or wavelength band. The light source 101 emits light in response to an image signal provided by the controller 114. The image signal includes image data corresponding to a plurality of frames displayed sequentially. The image signal may originate from an external source in a streaming or cloud-based manner, from the internal memory of the projection system 100 such as a hard disk, from a removable medium operationally connected to the projection system 100, or a combination thereof.

[0025] Filter 109 may be provided to mitigate the effects caused by internal components of the projection system 100. In some systems, the PLM 105 (described in more detail below) includes a cover glass which can cause reflections, device switching can temporarily cause undesirable steering angles, and various components can cause scattering. To counteract this and reduce the floor level of the projection system 100, filter 109 may be a Fourier ("DC") filter component configured to block a portion of the fourth light 108. Thus, filter 109 can increase contrast by reducing the floor level from light near 0 angle, corresponding to elements such as cover glass reflections and stroke transition states. This DC block region may be actively used by an algorithm to prevent certain types of light from reaching the screen. In some aspects of this disclosure, filter 109 prevents undesirable light from reaching the screen by steering the light to an optical dump located outside the active image region in response to control from controller 114.

[0026] The second projection optical system 111 may include optical components configured to expand the field of view (FOV) of the projection device 101. Such optical components may include, but are not limited to, fisheye lenses or other wide-angle optical components.

[0027] Phase optical modulator

[0028] As shown in Figure 1, the controller 114 also controls the PLM 105 that receives light from the light source 101. The PLM 105 imparts a spatially varying phase modulation to the light and redirects the modulated light to the second projection optical system 111. The PLM 105 may be a reflective type, in which the PLM 105 reflects incident light with a spatially varying phase, or it may be a transmissive type, in which the PLM 105 imparts a spatially varying phase to the light passing through the PLM 105. In some aspects of this disclosure, the PLM 105 may have a liquid crystal on silicon (LCOS) architecture or be a micro-electromechanical system (MEMS) such as a digital micromirror device (DMD).

[0029] Figure 2 shows an example of a PLM 105 implemented as a reflective LCOS PLM 200, shown in a partial cross-sectional view. As shown in Figure 2, the PLM 200 includes a silicon backplane 210, a first electrode layer 220, a second electrode layer 230, a liquid crystal layer 240, a cover glass 250, and a spacer 260. The silicon backplane 210 includes the electronic circuits associated with the PLM 200, such as complementary metal oxide semiconductor (CMOS) transistors. The first electrode layer 220 includes an array of reflective elements 221 arranged in a transparent matrix 222. The reflective elements 221 may be formed of any optically reflective material such as aluminum or silver. The transparent matrix 222 may be formed of any optically transparent material such as a transparent oxide. The second electrode layer 230 may be formed of any optically transparent conductive material such as a thin film of indium tin oxide (ITO). The second electrode layer 230 may be provided as a common electrode corresponding to a plurality of the reflective elements 221 of the first electrode layer 220. In such a configuration, each of the plurality of reflective elements 221 is coupled to the second electrode layer 230 via its respective electric field, thereby dividing the PLM 200 into an array of modulation elements (e.g., pixels). Thus, each of the plurality of reflective elements 221 (or a subset thereof) can be addressed via electronic circuits located within the silicon backplane 210, thereby allowing the state of the corresponding reflective element 221 to be modified.

[0030] The liquid crystal layer 240 is positioned between the first electrode layer 220 and the second electrode layer 230 and contains multiple liquid crystals 241. The liquid crystals 241 are particles that exist in an intermediate phase between solid and liquid; in other words, the liquid crystals 241 exhibit some degree of directional order but no positional order. The direction in which the liquid crystals 241 tend to point is called the "director." The liquid crystal layer 240 modifies the incident light entering from the cover glass 250 based on the birefringence Δn of the liquid crystals 241, which can be expressed as the difference between the refractive index parallel to the director and the refractive index perpendicular to the director. Thus, the maximum optical path difference can be expressed as the birefringence multiplied by the thickness of the liquid crystal layer 240. This thickness is set by a spacer 260 that seals the PLM 200 and ensures a set distance between the cover glass 250 and the silicon backplane 210. The liquid crystal 241 generally aligns along the electric field line between the first electrode layer 220 and the second electrode layer 230. As shown in Figure 2, the liquid crystal 241 near the center of the PLM 200 is oriented in this way, but the liquid crystal 241 near the periphery of the PLM 200 is substantially unoriented because there is no electric field line. The orientation of the liquid crystal 241 can be determined pixel by pixel by addressing each of the multiple reflective elements 221 via a phase drive signal.

[0031] Figure 3 shows another example of the PLM 105 implemented as the DMD PLM 300, shown in a partial cross-sectional view. As shown in Figure 3, the PLM 300 includes a backplane 310 and several controllable reflective elements as modulation elements, each of which includes a yoke 321, a mirror plate 322, and a pair of electrodes 330. Although only two electrodes 330 are visible in the cross-sectional view of Figure 3, each reflective element can actually include additional electrodes. Not specifically shown in Figure 3, the PLM 300 may further include spacer layers, support layers, hinge components for controlling the height or orientation of the mirror plate 322, etc. The backplane 310 includes the electronic circuits accompanying the PLM 300, such as CMOS transistors and memory arrays.

[0032] The yoke 321 may be formed from or contain an electrically conductive material to allow a bias voltage to be applied to the mirror plate 322. The mirror plate 322 may be formed from any highly reflective material such as aluminum or silver. The electrodes 330 are configured to receive a first voltage and a second voltage, respectively, and may be individually addressable. Depending on the values ​​of the voltages on the electrodes 330 and the voltage on the mirror plate 322 (e.g., the bias voltage), a potential difference exists between the mirror plate 322 and the electrodes 330, which generates an electrostatic force acting on the mirror plate 322. The yoke 321 is configured to allow vertical movement of the mirror plate 322 in response to the electrostatic force. The equilibrium position of the mirror plate 322, which occurs when the electrostatic force and the spring-like force of the yoke 322 are equal, determines the optical path length of the light reflected from the upper surface of the mirror plate 322. Therefore, each of the multiple controllable reflective elements is controlled to provide several (three in the illustration) discrete heights, and thus several discrete phase configurations or phase states. As shown in the illustration, each phase state has a flat profile. In some aspects of this disclosure, electrodes 330 may be supplied with voltages different from each other to give a slope to the mirror plate 322. Such slopes may be used in conjunction with the optical damping of the type described above.

[0033] The PLM 300 can achieve high switching speeds, such as switching from one phase state in the order of tens of microseconds. To provide the entire cycle of phase control, the total optical path difference between the state in which the mirror plate 322 is at its highest point and the state in which the mirror plate 322 is at its lowest point should be approximately equal to the wavelength λ of the incident light. Therefore, the height range between the highest and lowest points should be approximately equal to λ / 2.

[0034] Regardless of which specific architecture is used for the PLM 105, it is controlled by the controller 114 to take a specific phase configuration for each pixel. Thus, the PLM 105 utilizes an array of modulation elements, such as a 960×540 array. The number of modulation elements in the array can correspond to the resolution of the PLM 105. Due to the steering properties that can be implemented, light can be steered to any location on the reconstructed image plane and is not bound to the same pixel grid as the PLM 105. Because the PLM 105 is capable of fast response times, it can generate high-resolution moving images on the reconstructed image plane. The operation of the PLM 105 may be affected by the data bandwidth of the projection system 100, the stroke quantization of the PLM 105, and / or the response time of the PLM 105. The maximum resolution can be determined by the point-spread function (PSF) of the light source 101 and based on the parameters of the various optical components in the projection system 100.

[0035] Dynamic screen geometry

[0036] The PLM 105 may be the PLM 200 in Figure 2 or the PLM 300 in Figure 3, and can create a projected image at any distance from the projection system 100. Furthermore, the PLM 105 can form the projected image in any shape, such that the selected projection surface (e.g., screen 114, the wall of the room containing the projection system 100) does not need to be perpendicular to the optical axis. In implementations where the projection surface is a flat plane, the projected image may be a 2D image. However, in some implementations, the projection surface may not be a flat plane. In such implementations, the projected image is effectively a 3D image, which may correspond to a 2D image mapped to a non-flat surface. Furthermore, since the PLM 105 is a digitally controlled device, the effective screen geometry can be changed on the fly. In some implementations, changing the screen geometry may be done in response to real-time user input.

[0037] Figure 4 shows an example of a setting in which the projection system may be implemented. In particular, Figure 4 shows a projection device 401 which may be the projection system 100 shown in Figure 1, but without the screen 113. The projection device 401 is placed in a room where a first wall 411 and a second wall 412 are shown. For ease of explanation, various elements in Figure 4 that are not directly involved in projection (e.g., furniture) are not described separately here. While stationary, the projection device 401 can dynamically project images onto one or more of several different locations, shown as a first projection surface 421 and a second projection surface 422 on the first wall 411, and a third projection surface 423 on the second wall 412. The first projection surface 421, the second projection surface 422, and the third projection surface 423 may be screens, or simply parts of the first wall 411 and the second wall 412, respectively. Although only three projection planes are shown in Figure 4, in reality, the target projection plane may be any plane within the FOV of the projector device 401. Since the projection device 401 has a fast response time (for example, due to the PLM 105), the projection planes do not have to be adjacent to each other and may be located on different walls.

[0038] Image position, size, and / or other parameters may be adjusted via an interface, as shown in Figure 5. Figure 5 shows a user-controlled input device 501 that projects a calibration code 511 to a screen position selected by the user. The input device 501 may be a remote control or smartphone attached to the projection device 401. In some implementations, the input device 501 is configured to emit in the infrared (IR) spectrum, so that the calibration code 511 is invisible to the human eye. To detect the calibration code 511, the projection device 401 may be equipped with a camera, such as an IR sensor. The projection device 401 can locate and monitor the calibration code 511 in real time. In other implementations, the input device 501 does not need to emit the calibration code 511, and instead, the projection device 401 may be controlled through the use of an app installed on the input device 501. For example, the user may pre-specify multiple candidate screen positions (e.g., corresponding to a first projection plane 421, a second projection plane 422, and a third projection plane 423) and then select from among the candidate screen positions. In further implementations, the projection device 401 may include distance or depth sensors that detect surface features of the projection plane, such as a time-of-flight (TOF) sensor. Surface features may include flat surface features (e.g., position and geometry) and / or non-flat surface features (e.g., geometry and relief).

[0039] The projection device 401 may include both a camera and a distance sensor, and one or both sensors may be used to perform additional operations such as color correction (for example, to correspond to paint color). Additionally or alternatively, the input device 501 may include a camera and / or a distance sensor and communicate with the projection device 401 (for example, by transmitting data representing a map of a room or part of a room).

[0040] Figure 6 shows an exemplary process flow for a projection method using dynamic screen geometry according to various aspects of the present disclosure. The process flow may be performed in or by a controller of the projection system, such as the controller 114 of the projection system 100 (corresponding to the projection device 401) shown in Figure 1. To perform the process flow, the projection system 100 may have instructions stored in a non-temporary computer-readable medium (e.g., a hard disk, a removable storage medium, random access memory (RAM), etc.) such that when the instructions are executed by the controller 114, the projection system 100 performs the operation shown in Figure 6.

[0041] In operation 601, the controller 114 determines the position geometry and / or relief (commonly referred to as “target geometry” or “target configuration”) of the screen or projection surface. This determination can take user input, camera input, TOF sensor input, or one or more selected from similar data as input. The determination in operation 601 may be performed in multiple stages. For example, the controller 114 may first determine a rough position and / or geometry for the screen, and then refine the position, geometry, and / or relief. Figure 6 shows that operation 601 is performed at the beginning of the process flow, but in some implementations, operation 601 may be repeated continuously throughout the entire image projection. Thus, the target geometry and / or relief of the projection surface is determined dynamically, at least during or near the actual time of image projection.

[0042] In operation 602, the controller 114 determines an appropriate phase configuration for frames of image data. Frames may be in the form of image signals provided from an external, internal, or removable source. The image signal comprises a series of frames at a rate dependent on the frame rate of a particular application. Each frame contains image data for generating an image on a screen at a specific resolution. This disclosure is not particularly limited in terms of the frame rates and / or resolutions that may be implemented. For example, the frame rate may be 24Hz or 48Hz for cinema applications, 30Hz, 60Hz, or 120Hz for home applications, etc. The resolution may also be 2K (2048×1080), 4K (4096×2160), 1080p (1920×1080), consumer 4K (3840×2160), etc. The decision in step 602 can take as input one or more selected from the output of step 601, the pixel data contained in the frames of image data, or similar data. In addition to the phase configuration, operation 602 may also include determining an appropriate brightness level for the light source 101.

[0043] The decision in operation 602 may include calculating a beam steering drive solution for the configuration of PLM 105 that approximates the target light field when reconstructed. In some examples, the calculation is performed on the phasor field M(x,y,0)=A in the modulation plane. M (x,y,0)∠φ M (x,y,0) (called the "modulation field") and the phasor field R(x',y',z')=A in the reconstruction plane R (x',y',z')∠φ R A mapping (e.g., a bidirectional mapping) can be established between (x', y', z') (called the “reconstructed field”). The mapping may be expressed as any numerical wave propagation method, including but not limited to the Fresnel method or the Rayleigh-Sommerfeld method.

[0044] In operation 603, the controller 114 controls the components of the projection system 100 to project light according to the phase configuration determined in operation 603. Operation 603 may include controlling the light source 101 to emit light and / or controlling the PLM 105 to modulate light according to the phase configuration. This control can be performed by using an emission control signal provided to the light source 101 and / or a phase control signal provided to the PLM 105. As a result of operation 603, a frame of image data is projected onto a projection surface having the appropriate position and geometry. In implementations where operation 601 is performed continuously throughout the entire image display, operation 603 may also include emitting visible light and multiplexed IR illumination. The IR illumination can then be detected by the projection system 100 and used to refine the determination of the screen geometry. The light source may be configured to emit visible light and multiplexed infrared light.

[0045] In operation 604, the controller 114 decides whether to change the screen geometry. If no change occurs, the process flow returns to operation 602 to determine the appropriate phase configuration for the next frame in the image data. If a change occurs, the process flow returns to operation 601 to determine the appropriate screen geometry before the next frame is processed and displayed. The controller 114 may decide to make a change in response to user input, signals from cameras and / or TOF sensors associated with the projection system 100, etc. In some implementations, operation 604 may occur once per frame, in which case operations 602-604 are repeated per frame. In other implementations, operation 604 may occur only once every few frames (for example, once every second). In this case, operations 602 and 603 are repeated for each frame, and operation 604 is performed at appropriate intervals.

[0046] The projection system 100 may be configured to display multiple images on multiple projection surfaces so that they are visible to an observer simultaneously. Figure 7 shows an exemplary process flow of such an operation that may complement or replace operations 601, 602, and / or 603 shown in Figure 6. To perform the process flow, the projection system 100 may have instructions stored in a non-temporary computer-readable medium (e.g., a hard disk, removable storage medium, RAM, etc.) such that, when the instructions are executed by the controller 114, the projection system 100 performs the operations of Figure 7.

[0047] In operation 701, the controller 114 determines the number of different screens (or other projection planes or target areas of projection planes) represented by a positive integer N > 1. Operation 701 may also include determining the appropriate geometry for each screen. In operation 702, the index variable n, which varies from 1 to N, is initialized or reinitialized to 1.

[0048] In operation 703, the controller receives a frame of image data, and in operation 704, the controller 114 determines an appropriate phase configuration for the frame of image data corresponding to the nth projection plane. As described above, the frame may be in the form of an image signal provided from an external, internal, or removable source. The image signal comprises a series of frames at a rate dependent on the frame rate for the particular application. Each frame contains image data for generating an image on a screen at a specific resolution. This disclosure is not particularly limited in terms of the frame rates and / or resolutions that may be implemented. The frame rate may be the base frame rate divided by N. For example, the base frame rate may be 24Hz or 48Hz for cinema applications, and 30Hz, 60Hz, or 120Hz for home applications, etc. The base resolution may also be 2K (2048×1080), 4K (4096×2160), 1080p (1920×1080), consumer 4K (3840×2160), etc. Therefore, each frame period (i.e., the reciprocal of the frame rate corresponding to the duration of the frame) is divided into N sub-periods. The decision in operation 704 can take as input one or more selected from the output of operation 701, the pixel data contained in the frames of image data, or similar data. In addition to phase configuration, operation 704 may include determining an appropriate luminance level for the light source 101.

[0049] In operation 705, the controller 114 controls the components of the projection system 100 to project light according to the phase configuration determined in operation 704. Operation 705 may include controlling the light source 101 to emit light and / or controlling the PLM 105 to modulate light according to the phase configuration. This control can be performed by using an emission control signal provided to the light source 101 and / or a phase control signal provided to the PLM 105. As a result of operation 705, a frame of image data is projected onto the nth projection plane having the appropriate position and geometry. Operation 703 may also include emitting visible light and multiplexed IR illumination, which may be detected by the projection system 100 and used to refine the determination of the screen geometry.

[0050] In operation 706, the controller 114 determines whether operations 703-704 have been completed for all N projection planes by comparing index n with N. If n is less than N, n is incremented in operation 707 and the process flow returns to operation 703. Otherwise, the process flow returns to operation 702 and n is reinitialized to 1 for the next frame in the image data.

[0051] Figure 7 illustrates a time-division projection method for multiple displays, where each frame is displayed sequentially at full brightness or resolution, but at a frame rate lower than the base frame rate. In other implementations, the projection system 100 can display frames in a spatially multiplexed manner. In such implementations, each frame may be displayed simultaneously on all N projection planes at full frame rate, but at a brightness or resolution lower than the base brightness or frame rate.

[0052] effect

[0053] The beam steering characteristics and fast response times of the projection systems and methods described herein make it possible to realize projection systems that can dynamically move and / or reshape images and / or screen geometry. Compared with other projection systems, the systems described herein are suitable for a wide range of projection applications in a wide range of environments. For example, the projection systems and methods disclosed herein may be suitable for home environments, office environments, live venues, automotive environments, industrial environments, commercial and / or retail environments, or group (e.g., "crowdsourced") environments.

[0054] Furthermore, while the above description primarily concerns dynamically moving and / or reshaping an image to adapt to screen geometry, the disclosure is not limited thereto. In some implementations, the projection systems and methods described above may dynamically move and / or reshape the image for laser safety reasons. For example, the projection system may determine if a person has moved into the beam path and move the beam away from the person (e.g., from the person's eyes).

[0055] Furthermore, the above systems, methods, and effects may also be achieved through the use of multiple cooperating projection devices, each configured according to the above aspects of this disclosure. In such examples, the multiple projection devices may operate to cover a larger spatial portion than can be achieved by a single projection device alone (e.g., by tiling each projection image with or without overlap across a large projection surface), to increase the brightness of the projection image (e.g., by having two or more of the multiple projection devices project the same image onto the same portion of the projection surface), and / or to achieve stereoscopic projection (e.g., by having two of the multiple projectors project different eye views of the same image onto the same portion of the projection surface). In one particular example, the individual projection devices may be implemented as automotive headlights, together with automotive headlights, or using automotive headlights, thereby enabling one or more automobiles to cooperate (e.g., in a drive-in theater) to project images onto a large screen from different positions.

[0056] The systems, methods, and devices described herein may take any one or more of the following configurations:

[0057] (1) A projection system comprising: a light source configured to emit light in response to image data; a phase light modulator configured to receive the light from the light source and apply a spatially variable phase modulation to the light, thereby steering the light and generating projected light; and a controller configured to dynamically determine a target geometry of a projection surface onto which the projected light is projected based on at least one of a user input or a sensor signal, determine a phase configuration for a frame of the image data based on the target geometry, and provide a phase control signal to the phase light modulator, wherein the phase control signal is configured to cause the phase light modulator to generate the projected light according to the phase configuration for the frame.

[0058] (2) The projection system according to (1), wherein the phase optical modulator includes a plurality of modulation elements arranged in an array and a circuit configured to modify the state of each of the plurality of modulation elements in response to the phase control signal.

[0059] (3) The projection system according to (1) or (2), wherein the phase light modulator is a digital micromirror device.

[0060] (4) The projection system according to (1) or (2), wherein the phase light modulator is a liquid crystal on semiconductor device.

[0061] (5) The projection system according to any one of (1) to (4), further comprising an infrared sensor configured to detect a calibration code projected onto the projection surface and to generate the sensor signal based on the calibration code.

[0062] (6) The projection system according to any one of (1) to (5), further comprising a depth sensor configured to detect surface features of the projection surface and generate the sensor signal based on the surface features.

[0063] (7) The projection system according to any one of (1) to (6), wherein the controller is configured to: determine the number of target regions on the projection plane; divide the frame period of the image data into a number of sub-periods corresponding to the number of target regions; and for each sub-period, perform in a time-division manner an operation to dynamically determine the target geometry, an operation to determine the phase configuration, and an operation to provide the phase control signal.

[0064] (8) The projection system according to (7), wherein the plurality of target regions are not continuous with each other.

[0065] (9) The projection system according to any one of (1) to (8), further comprising a filter that blocks a portion of the projected light.

[0066] (10) The projection system according to any one of (1) to (9), wherein the light source is a coherent light source.

[0067] (11) A projection method comprising: emitting light from a light source in response to image data; receiving the light by a phase light modulator; applying a spatially variable phase modulation to the light by the phase light modulator to steer the light and generate projected light; dynamically determining a target geometry of a projection surface onto which the projected light is projected, based on at least one of user input or sensor signals; determining a phase configuration for a frame of the image data based on the target geometry; and providing a phase control signal to the phase light modulator, the phase control signal being configured to cause the phase light modulator to generate the projected light according to the phase configuration for the frame.

[0068] (12) The method according to (11), wherein the phase optical modulator includes a plurality of modulation elements arranged in an array, and the method includes: modifying the state of each of the plurality of modulation elements in response to the phase control signal.

[0069] (13) The method according to (11) or (12), wherein the phase optical modulator is a digital micromirror device.

[0070] (14) The method according to (11) or (12), wherein the phase optical modulator is a liquid crystal on semiconductor device.

[0071] (15) The method according to any one of (11) to (14), further comprising the steps of: detecting a calibration code projected onto the projection surface using an infrared sensor; and generating a sensor signal based on the calibration code.

[0072] (16) The method according to any one of (11) to (15), further comprising the steps of: detecting surface features of the projection surface with a depth sensor; and generating a sensor signal based on the surface features.

[0073] (17) The method of any one of (11) to (16), further comprising: determining the number of target regions of the projection plane; dividing the frame period of the image data into a number of sub-periods corresponding to the number of target regions; and for each sub-period, performing in a time-division manner an operation to dynamically determine the target geometry, an operation to determine the phase configuration, and an operation to provide the phase control signal.

[0074] (18) The method according to (17), wherein the target regions are not contiguous with each other.

[0075] (19) The method according to any one of (11) to (18), further comprising the step of blocking a portion of the projected light with a filter.

[0076] (20) A non-temporary computer-readable medium that stores instructions, when executed by the processor of the projection device, causing the projection device to perform an operation including the method described in any one of (11) to (19).

[0077] With respect to the processes, systems, methods, heuristics, etc., described herein, the steps of such processes, etc., are described as being performed in a certain ordered sequence, but it should be understood that such processes may be carried out using the described steps performed in an order other than that described herein. Furthermore, it should be understood that certain steps may be performed simultaneously, other steps may be added, or certain steps described herein may be omitted. In other words, the descriptions of processes herein are provided for the purpose of illustrating certain embodiments and should not be construed in any way as limiting the scope of the claims.

[0078] Therefore, it should be understood that the above description is intended to be illustrative and not restrictive. Many embodiments and uses other than those provided will become apparent from reading the above description. The scope should not be determined by reference to the above description, but rather by reference to the appended claims, along with the full scope of equivalents to which such claims qualify. Future developments are expected and intended in the art described herein, and the disclosed systems and methods are expected to be incorporated into such future embodiments. In short, it should be understood that this application is modifiable and can be modified.

[0079] All terms used in the claims are intended to be given their broadest reasonable interpretation and their ordinary meaning as understood by a person familiar with the art described herein, unless otherwise expressly indicated herein. In particular, the use of singular articles such as “a,” “it,” and “the said” should be read to describe one or more of the elements indicated, unless the claim expressly limits it to the contrary.

[0080] This abstract of the disclosure is provided to allow readers to quickly confirm the nature of the technical disclosure. The abstract is submitted with the understanding that it is not to be used to interpret or limit the scope or meaning of the claims. In addition, it is found that in the preceding detailed description, various features are grouped together in various embodiments for the purpose of improving the flow of the disclosure. This method of disclosure should not be interpreted as reflecting an intention that the claimed embodiments incorporate more features than are expressly described in each claim. Rather, as reflected in the following claims, the subject matter of the invention is less than all the features of a single disclosed embodiment. Thus, the following claims are incorporated hereby incorporated into the detailed description, and each claim stands alone as separately claimed subject matter.

[0081] Various aspects of the present invention can be understood from the enumerated example embodiments (EEEs) listed below. EEE1. A projection system: A light source configured to emit light in response to image data; A phase light modulator configured to receive light from the light source, apply spatially varying phase modulation to the light, thereby steering the light and generating projected light; Based on at least one of user input or sensor signals, the target geometry of the projection surface onto which the projection light is projected is dynamically determined. Based on the target geometry, the phase configuration for the frame of the image data is determined. Provides a phase control signal to the aforementioned phase optical modulator. A controller configured such that the phase control signal is configured to cause the phase light modulator to generate the projected light according to the phase configuration for the frame, the controller comprises A projection system. EEE2. The projection system according to claim 1, wherein the phase optical modulator includes a plurality of modulation elements arranged in an array and a circuit configured to modify the state of each of the plurality of elements in response to a phase control signal. EEE3. The projection system according to claim 2, wherein the phase light modulator is a digital micromirror device. EEE4. The projection system according to claim 2, wherein the phase light modulator is a liquid crystal on semiconductor device. EEE5. The projection system according to any one of claims 1 to 4, further comprising an infrared sensor configured to detect a calibration code projected onto the projection surface and to generate the sensor signal based on the calibration code. EEE6. The projection system according to any one of claims 1 to 5, further comprising a depth sensor configured to detect surface features of the projection surface and generate the sensor signal based on the surface features. EEE7. The projection system according to any one of claims 1 to 6, wherein the controller is configured to: determine the number of target regions on the projection plane; divide the frame duration of the image data into a number of sub-durations corresponding to the number of target regions; and for each sub-duration, perform in a time-division manner an operation to dynamically determine the target geometry, an operation to determine the phase configuration, and an operation to provide the phase control signal. EEE8. The projection system according to claim 7, wherein the plurality of target regions are not continuous with each other. EEE9. The projection system according to any one of claims 1 to 8, further comprising a filter for blocking a portion of the projected light. EEE10. The projection system according to any one of claims 1 to 9, wherein the light source is a coherent light source. EEE11. A method for driving a projection system: The steps include: emitting light from a light source in response to image data; The steps include receiving the light with a phase light modulator; The steps include: applying spatially varying phase modulation to the light using the phase light modulator, thereby steering the light and generating projected light; A step of dynamically determining the target geometry of the projection surface onto which the projection light is projected, based on at least one of user input or sensor signals; The steps include determining the phase configuration for the frame of the image data based on the target geometry; The step of providing a phase control signal to the phase optical modulator, wherein the phase control signal is configured to cause the phase optical modulator to generate the projected light according to the phase configuration for the frame, method. EEE12. The phase optical modulator includes a plurality of modulation elements arranged in an array, and the method is: This includes modifying the state of each of the plurality of modulation elements in response to the phase control signal. The method according to claim 11. EEE13. The method according to claim 12, wherein the phase optical modulator is a digital micromirror device. EEE14. The method according to claim 12, wherein the phase optical modulator is a liquid crystal on semiconductor device. EEE15. The step of detecting the calibration code projected onto the projection surface using an infrared sensor; The further step includes generating the sensor signal based on the calibration code, The method according to any one of claims 11 to 14. EEE16. A step of detecting the surface features of the projection surface using a depth sensor; The step further includes generating the sensor signal based on the surface features, The method according to any one of claims 11 to 15. EEE17. Determining the number of target regions on the projection surface; The frame duration of the image data is divided into a number of sub-durations corresponding to the number of target regions; For each sub-period, the operation further includes performing, in a time-division manner, an operation to dynamically determine the target geometry, an operation to determine the phase configuration, and an operation to provide the phase control signal. The method according to any one of claims 11 to 16. EEE18. The method according to claim 17, wherein the target regions are not contiguous with each other. EEE19. The method according to any one of claims 11 to 18, further comprising the step of blocking a portion of the projected light with a filter. EEE20. A non-temporary computer-readable medium that stores instructions, when executed by the processor of a projection device, causing the projection device to perform an operation including the method according to any one of claims 11 to 19.

Claims

1. It is a projection system: A light source configured to emit light in response to image data; A phase light modulator configured to receive light from the light source, apply spatially varying phase modulation to the light, thereby steering the light and generating projected light; Based on at least one of user input or sensor signals, the target geometry and relief of the projection surface onto which the projection light is projected are dynamically determined. Based on the target geometry and the relief, the phase configuration for the frame of the image data is determined. Provides a phase control signal to the aforementioned phase optical modulator. A controller configured such that the phase control signal is configured to cause the phase light modulator to generate the projected light according to the phase configuration for the frame, so that the projected light provides a projected image that approximates the image represented by the image data. A projection system.

2. The projection system according to claim 1, wherein the phase optical modulator includes a plurality of modulation elements arranged in an array, and a circuit configured to modify the state of each of the plurality of modulation elements in response to the phase control signal.

3. The projection system according to claim 2, wherein the phase light modulator is a digital micromirror device or a liquid crystal on semiconductor device.

4. The projection system according to any one of claims 1 to 3, further comprising an infrared sensor configured to detect a calibration code projected onto the projection surface and to generate the sensor signal based on the calibration code.

5. The projection system according to any one of claims 1 to 4, further comprising a user-controlled input device configured to project a calibration code onto a user-selected projection plane.

6. The projection system according to claim 5, wherein the user-controlled input device is configured to emit light in the infrared spectrum.

7. The projection system according to any one of claims 1 to 4, wherein the light source is configured to emit visible light and multiplexed infrared light.

8. The projection system according to any one of claims 1 to 7, further comprising a depth sensor configured to detect surface features of the projection surface and generate the sensor signal based on the surface features.

9. The projection system according to claim 8, wherein the surface features include non-flat surface features.

10. The aforementioned controller is: The number of target regions on the projection surface is determined; The frame duration of the image data is divided into a number of sub-durations corresponding to the number of target regions; The projection system according to any one of claims 1 to 9, configured to perform, in a time-division manner, operations for dynamically determining the target geometry and the relief, operations for determining the phase configuration, and operations for providing the phase control signal, for each sub-period.

11. The projection system according to claim 10, wherein the target regions are not continuous with each other.

12. The projection system according to any one of claims 1 to 11, further comprising a filter for blocking a portion of the projected light.

13. The projection system according to any one of claims 1 to 12, wherein the light source is a coherent light source.

14. A method for driving a projection system: The steps include: emitting light from a light source in response to image data; The steps include: receiving the light with a phase light modulator; The steps include: applying spatially varying phase modulation to the light using the phase light modulator, thereby steering the light and generating projected light; The steps include dynamically determining the target geometry and relief of the projection surface onto which the projection light is projected, based on at least one of user input or sensor signals; The steps include determining the phase configuration for the frame of the image data based on the target geometry and the relief; A step of providing a phase control signal to the phase light modulator, wherein the phase control signal is configured to cause the phase light modulator to generate the projected light according to the phase configuration for the frame, such that the projected light provides a projected image that approximates the image represented by the image data, method.

15. The phase optical modulator includes a plurality of modulation elements arranged in an array, and the method is: This includes modifying the state of each of the plurality of modulation elements in response to the phase control signal. The method according to claim 14.

16. The method according to claim 15, wherein the phase light modulator is a digital micromirror device or a liquid crystal on semiconductor device.

17. The steps include: detecting the calibration code projected onto the projection surface using an infrared sensor; The further step includes generating the sensor signal based on the calibration code, The method according to any one of claims 14 to 16.

18. The further step includes projecting a calibration code onto a user-selected projection plane using a user-controlled input device. The method according to any one of claims 14 to 17.

19. The method according to claim 18, further comprising emitting light of the infrared spectrum by the user-controlled input device.

20. The method according to any one of claims 14 to 17, further comprising emitting infrared light and visible light with the light source, and multiplexing the infrared light with the visible light.

21. The steps include: detecting the surface characteristics of the projection surface using a depth sensor; The step further includes generating the sensor signal based on the surface features, The method according to any one of claims 14 to 20.

22. The method according to claim 21, wherein the surface features include non-flat surface features.

23. Determining the number of target regions on the projection surface; The frame period of the image data is divided into a number of sub-periods corresponding to the number of target regions; For each sub-period, the operation of dynamically determining the target geometry and the relief, determining the phase configuration, and providing the phase control signal are performed in a time-division manner, The method according to any one of claims 14 to 22.

24. The method according to claim 23, wherein the target regions are not continuous with each other.

25. The method according to any one of claims 14 to 24, further comprising the step of blocking a portion of the projected light with a filter.

26. A non-temporary computer-readable medium that stores instructions, when executed by the processor of the projection device, causing the projection device to perform an operation including the method according to any one of claims 14 to 25.

Images