Image projector for laser scanning of a spatial light modulator

Abstract

An image projector employs a laser scanning illumination device to illuminate a spatial light modulator (SLM), wherein an angular beam expander element, typically a diffuser or micro-lens array (MLA), adjacent to the SLM or in a conjugate plane to the SLM enhances the fill of the exit aperture while minimizing the impact on the scanning accuracy of the laser illumination on the SLM. Various schemes for synchronizing the rolling shutter update of the SLM during the scanning illumination are also disclosed, as well as systems employing a binary switchable SLM.

Description

Technical Field

[0001] The present invention relates to image projectors, and more particularly to image projectors having various configurations for illuminating spatial light modulators. Background Technology

[0002] It is known that images are projected by illuminating a spatial light modulator (SLM) such as a liquid crystal display (LCD), digital light processing (DLP) chip, or liquid crystal on silicon (LCOS) modulator and projecting an image for a user to view. Such projectors are commonly used in near-eye displays, where the projected image is typically collimated and introduced into a light guide. The image propagates along the light guide via internal reflection until it is typically coupled to the user's eye by a partially reflective surface or by a diffractive element. This can help to expand the effective optical aperture from which the image is projected toward the eye.

[0003] Existing publication WO 2019 / 111237 A1 discloses a projector in which illumination of an SLM is performed sequentially by switching or scanning illumination sources across an LCOS modulator. The aforementioned publication... Figure 9A and Figure 12B Reproduced here as Figures 1A and 1B, with original reference numerals cited in parentheses. The laser (10) transmits light focused by the lens (12). The converging beam is reflected by the scanning mirror (14) onto the polarization beam splitter (PBS1). The reflecting lens (R1) further focuses the beam via a second polarization beam splitter (PBS2) to be focused onto the LCOS (20). The reflected beam is reflected by a third polarization beam splitter (PBS3) onto the collimating reflecting lens (22). The collimated beam then exits through the aperture (24), which is also the entrance aperture into the waveguide. The foregoing reference also discloses that the activation of the LCOS can be synchronized with a linear scan of a line of illumination (Figure 1B). Summary of the Invention

[0004] This invention is an image projector.

[0005] According to the teachings of embodiments of the present invention, an image projector is provided for projecting an image via an exit aperture, the image being a representation of an input digital image. The image projector includes: (a) a spatial light modulator (SLM) providing a two-dimensional array of pixel elements defining an SLM resolution, each of the pixel elements being controllable to modulate properties of light transmitted or reflected by the pixel element; (b) a scanning illumination device deployed to scan an illumination beam across the two-dimensional array of the spatial light modulator in a two-dimensional scanning mode while synchronously modulating the intensity of the illumination beam; (c) a controller electronically connected to the spatial light modulator and the scanning illumination device; and (d) projection optics including at least one optical element configured to project illumination from the spatial light modulator to generate an output image guided to the exit aperture. The controller is configured to: (i) process an input digital image to generate a first modified image, the first modified image substantially corresponding to a reduced-resolution version of the input digital image, the resolution of which is less than that of the SLM in at least one dimension; (ii) process the input digital image to generate a second modified image, the second modified image including pixel modulation data for pixel elements of the SLM corresponding to high spatial frequency variations in the input digital image; (iii) excite the SLM according to the pixel modulation data of the second modified image; and (iv) drive a scanning illumination device to modulate the intensity of an illumination beam while scanning across a two-dimensional array to illuminate the SLM according to the first modified image, wherein the SLM thereby provides high spatial frequency modulation of the illumination to project an output image having pixel intensities corresponding to the digital image.

[0006] According to another feature of an embodiment of the invention, the two-dimensional scanning mode has a fast direction aligned with the rows of the SLM and a slow direction aligned with the columns of the SLM, wherein the SLM is excited as the pixel rows advance in the slow direction in a rolling update ahead of the illumination beam.

[0007] According to another feature of an embodiment of the invention, the controller is further configured to apply a reverse excitation to the pixel elements of the SLM as the pixel row following the illumination beam is updated in a rolling fashion.

[0008] According to another feature of the embodiments of the present invention, the SLM is a ferroelectric liquid crystal display in which each pixel element can switch between a bright state and a dark state.

[0009] According to another feature of an embodiment of the invention, the controller is configured to apply at least one criterion to determine a sub-region of the SLM required to generate high-resolution output in a corresponding subfield of the output image, and the controller is configured to excite the sub-region of the SLM while de-exciting a plurality of pixel elements outside the sub-region.

[0010] According to another feature of an embodiment of the invention, the pixel element is in a lit state when not excited, and wherein the controller is configured to modulate the intensity of the illumination beam when driving the scanning illumination device to scan both the interior and exterior of a sub-region of the SLM.

[0011] According to another feature of an embodiment of the present invention, at least one criterion includes: determining a region of the input digital image that contains high spatial frequency content.

[0012] According to another feature of an embodiment of the invention, at least one criterion includes: determining a subfield of the output image corresponding to the current gaze direction of the viewer's eyes.

[0013] According to another feature of an embodiment of the invention, the controller is configured to drive the scanning illumination device to illuminate only a sub-region of the SLM.

[0014] Another feature of an embodiment of the invention is that an angle beam extender is provided, which is deployed adjacent to the SLM.

[0015] Another feature of an embodiment of the invention also provides a micro-lens array (MLA) which is deployed in a non-focused manner close to the SLM.

[0016] According to another feature of an embodiment of the invention, the MLA for each pixel element of the SLM includes a lens, and each lens is aligned with the corresponding pixel element.

[0017] According to another feature of an embodiment of the invention, MLA is attached to the surface of SLM.

[0018] Another feature of an embodiment of the invention is that the MLA is integrated with a transparent substrate covering the SLM.

[0019] According to another feature of an embodiment of the invention, the MLA and SLM are facing each other, and wherein the substrate provides a flat surface opposite to the MLA.

[0020] According to another feature of an embodiment of the invention, the MLA is formed of a material having a first refractive index, and wherein a transparent adhesive fills the space between the MLA and the SLM, the transparent adhesive having a second refractive index different from the first refractive index.

[0021] According to another feature of an embodiment of the invention, the substrate is part of a field lens that extends across the SLM.

[0022] According to another feature of an embodiment of the invention, the field lens is implemented as a doublet lens, which presents a flat surface opposite to the SLM.

[0023] According to another feature of an embodiment of the invention, the invention also provides: (a) an intermediate optical device with positive optical power, which is deployed in the optical path between the SLM and the collimating device, the intermediate optical device refocusing illumination from the SLM onto the image plane; and (b) an angular beam expander, which is deployed at the image plane, wherein projection optics are deployed to project illumination from the image plane to generate an output image guided to an exit aperture.

[0024] According to another feature of an embodiment of the invention, the following are also provided: (a) an angular beam expander deployed at the illumination image plane, wherein the scanning illumination device is configured to generate a modulated scanning illumination pattern at the illumination image plane; and (b) an intermediate optical device with positive optical power deployed in the optical path between the illumination image plane and the SLM, wherein the intermediate optical device refocuses illumination from the illumination image plane onto the SLM.

[0025] Another feature of an embodiment of the invention is that the angular beam expander is a microlens array.

[0026] According to the teachings of embodiments of the present invention, an image projector for projecting an image via an exit aperture, the image being a representation of an input digital image, the image projector comprising: (a) a spatial light modulator (SLM) providing a two-dimensional array of pixel elements arranged in rows and columns, each of the pixel elements being controllable to modulate the properties of light transmitted or reflected by the pixel element; (b) a scanning illumination device deployed to scan an illumination beam across the two-dimensional array of the spatial light modulator in a two-dimensional scanning mode, while synchronously modulating the intensity of the illumination beam, the two-dimensional scanning mode having a fast direction aligned with rows and a slow direction aligned with columns; and (c) a controller, which is connected to the spatial light modulator... The spatial light modulator and the scanning illumination device are electronically connected; and (d) the projection optics includes at least one optical element configured to project illumination from the spatial light modulator to generate an output image guided to an exit aperture, wherein the controller is configured to: (i) excite the SLM using a rolling update of pixel data for a pixel row advancing in the slow direction along the scanning mode; (ii) drive the scanning illumination device to illuminate the pixel row advancing in the slow direction after the rolling update of the pixel data; and (iii) apply a reverse excitation to the pixel elements of the SLM as the pixel row following the illumination beam is updated.

[0027] According to the teachings of embodiments of the present invention, an image projector for projecting an image via an exit aperture, the image being a representation of an input digital image, the image projector comprising: (a) a spatial light modulator (SLM) providing a two-dimensional array of pixel elements, each of the pixel elements being controllable to modulate the properties of light transmitted or reflected by the pixel element; (b) a scanning illumination device deployed to scan an illumination beam across the two-dimensional array of the spatial light modulator in a two-dimensional scanning mode, while synchronously modulating the intensity of the illumination beam; (c) a controller electronically connected to the spatial light modulator and the scanning illumination device; and (d) projection optics comprising components configured to project... Illumination from a spatial light modulator to generate at least one optical element of an output image guided to an exit aperture, wherein a controller is configured to: (i) apply at least one criterion to determine a sub-region of the SLM required to generate the output in a corresponding subfield of the output image, the controller being configured to: excite the sub-region of the SLM while de-exciting a plurality of pixel elements outside the sub-region; and (ii) drive a scanning illumination device to illuminate at least the sub-region of the SLM to project an output image having a pixel intensity corresponding to the digital image into at least the corresponding subfield, the pixel intensity being determined by the modulation state of the pixel elements and the intensity of the illumination from the scanning illumination device falling on the pixel elements.

[0028] According to another feature of an embodiment of the invention, the pixel element is in a lit state when not excited, and wherein the controller is configured to modulate the intensity of the illumination beam when driving the scanning illumination device to scan both the interior and exterior of a sub-region of the SLM.

[0029] According to another feature of an embodiment of the present invention, at least one criterion includes: determining a region of the input digital image that contains high spatial frequency content.

[0030] According to another feature of an embodiment of the invention, at least one criterion includes: determining a subfield of the output image corresponding to the current gaze direction of the viewer's eyes.

[0031] According to another feature of an embodiment of the invention, the controller is configured to drive the scanning illumination device to illuminate only a sub-region of the SLM.

[0032] According to the teachings of embodiments of the present invention, an image generation component for use with scanning directional illumination from a laser source is also provided, the image generation component comprising: (a) a spatial light modulator (SLM) providing a two-dimensional array of pixel elements, each of which can be controlled to modulate the properties of light transmitted or reflected by the pixel elements; and (b) an angular beam expander deployed adjacent to the SLM.

[0033] According to another feature of an embodiment of the present invention, the SLM is a reflective spatial light modulator.

[0034] Another feature of an embodiment of the invention is that the angular beam expander is a microlens array (MLA), which is deployed to approach the SLM non-focusedly.

[0035] According to another feature of an embodiment of the invention, the MLA for each pixel element of the SLM includes a lens, and each lens is aligned with the corresponding pixel element.

[0036] According to another feature of an embodiment of the invention, MLA is attached to the surface of SLM.

[0037] Another feature of an embodiment of the invention is that the MLA is integrated with a transparent substrate covering the SLM.

[0038] According to another feature of an embodiment of the invention, the MLA and SLM are facing each other, and wherein the substrate provides a flat surface opposite to the MLA.

[0039] According to another feature of an embodiment of the invention, the MLA is formed of a material having a first refractive index, and wherein a transparent adhesive fills the space between the MLA and the SLM, the transparent adhesive having a second refractive index different from the first refractive index.

[0040] According to another feature of an embodiment of the invention, the substrate is part of a field lens that extends across the SLM.

[0041] According to another feature of an embodiment of the invention, the field lens is implemented as a doublet lens, which presents a flat surface opposite to the SLM. Attached Figure Description

[0042] The invention has been described herein by way of example only with reference to the accompanying drawings, in which:

[0043] Figures 1A and 1B cited above correspond to the existing publication WO 2019 / 111237 A1, respectively. Figure 9A and Figure 12B ;

[0044] Figure 2A This is a schematic diagram of an image projector with a teaching structure and operation according to an embodiment of the present invention, which employs an angular beam expander in a conjugate focal plane prior to a spatial light modulator (SLM).

[0045] Figure 2B This is a schematic diagram of an image projector with a teaching structure and operation according to another embodiment of the present invention, which employs an angular beam expander in the conjugate focal plane after the SLM;

[0046] Figure 3 This is a schematic diagram of an image projector with teaching construction and operation according to another embodiment of the present invention, which employs an angular beam expander adjacent to the SLM;

[0047] Figure 4A yes Figure 3 A schematic side view of a compact implementation of the arrangement, which employs a polarizing beam splitter prism;

[0048] Figure 4B This illustrates the multiple light paths that pass forward from the SLM through the optical system. Figure 4A Similar partial views;

[0049] Figure 5A This is an enlarged schematic side view showing the effect of an angular beam expander implemented as a microlens array (MLA) on the ray path of incident light reflected from the surface of a reflective SLM.

[0050] Figure 5B It shows the total angular envelope of incident and reflected illumination, and... Figure 5A Similar views;

[0051] Figures 6A to 6G This is a schematic diagram of alternative implementations of MLA adjacent to reflective SLM, wherein the MLA is: on the outer side of the adjacent substrate, on the inner side of the adjacent substrate, directly applied to the surface of the SLM with one microlens per pixel element, directly applied to the surface of the SLM with each microlens overlapping with more than one pixel element, bonded to the SLM with a non-refractive index-matching adhesive, integrated with a field lens, and implemented with negative optical power.

[0052] Figure 7 This shows the implementation of SLM in more detail. Figure 6C Similar diagrams;

[0053] Figure 8 Is with Figure 7 A similar schematic diagram, but illustrating an application with a transmissive SLM;

[0054] Figure 9A This is a schematic diagram illustrating the roll-up update of the SLM pixel state according to an implementation of the present invention, followed by the progression of the two-dimensional illumination scanning mode;

[0055] Figure 9B This is a schematic diagram showing the contrast of an LCD pixel as a function of the time since self-excitation;

[0056] Figure 9C It is shown that it is used for Figure 9A A schematic diagram of the angular displacement of the illumination scanning mirror in the linear scanning direction in the implementation method;

[0057] Figure 10A and Figure 10B This is a schematic diagram illustrating two stages of the progress of the SLM pixel state roll-up update followed by the two-dimensional illumination scanning mode, according to the bidirectional scanning implementation method of the present invention.

[0058] Figure 10C It is shown that it is used for Figure 10A and Figure 10B A schematic diagram of the angular displacement of the illumination scanning mirror in the linear scanning direction in the implementation method;

[0059] Figures 11A to 11C This refers to the different implementation methods of the variants, and their differences from those of the variant implementation methods. Figures 10A to 10C A similar view, in which multi-pixel updates are performed during illumination scanning in the first direction, and reverse scanning is performed without updates;

[0060] Figures 12A to 12D Is with Figure 11A and Figure 11B A similar view shows four consecutive operational phases for an SLM updated in two separate halves;

[0061] Figure 13A This is a schematic diagram illustrating the temporal relationships of data loading, excitation, pixel contrast response, illumination, and reverse excitation of a given pixel element according to an aspect of the present invention;

[0062] Figure 13B It is used to implement Figure 13A A schematic diagram of the roll-up update, scan illumination, and reset of pixel elements across the SLM sequence;

[0063] Figure 13C This demonstrates alternative, simplified implementation methods. Figure 13B Similar views;

[0064] Figure 14A It is a schematic diagram of the desired pixel image output along the pixel row and the corresponding illumination pattern that can be achieved by the scanning illumination device;

[0065] Figure 14B This is a schematic diagram of a pixel activation mode for achieving a desired pixel image output according to an aspect of the present invention, wherein the SLM is selectively activated in some regions and not activated (or activated with a zero signal) in other regions.

[0066] Figure 14C It is by... Figure 14A Lighting patterns and Figure 14B An example of the corresponding image output generated by combining pixel excitation modes;

[0067] Figure 15AThis is a schematic diagram of a multi-beam laser scanning array with two beams arranged side-by-side, each representing an RGB color.

[0068] Figure 15B The overall distribution of optical intensity can be achieved by changing the balance between two lasers of a given color;

[0069] Figure 15C This demonstrates how this effect can be used to provide an improved approximation of linear scanning in illumination scanning modes;

[0070] Figure 16 This is a flowchart illustrating the contributions of two parallel image processing procedures for driving a scanning illumination device and generating an output image according to another aspect of the present invention;

[0071] Figure 17A and Figure 17B The image output intensity variations along the pixel row are shown, respectively, both those that can be achieved by a laser scanner alone and those enhanced by the operation of a binary-mode SLM.

[0072] Figure 17C and Figure 17D They are respectively with Figure 17A and Figure 17B A similar view shows the effect of widening the scan illumination pattern;

[0073] Figure 18 The process is illustrated, showing the input digital image, the corresponding laser illumination image, and the output when the laser illumination image is modulated by SLM;

[0074] Figure 19A This is a schematic diagram showing the spatial variation of different colors across an SLM matrix scan (with the matrix set to a different distribution for each scan);

[0075] Figure 19B Is with Figure 9A A similar schematic diagram illustrates an application of two-dimensional scanning with a roll-up excitation, where a laser simultaneously scans separate lines of different colors while synchronously updating pixel values ​​with an appropriate binary image of the corresponding color.

[0076] Figure 19C and Figure 19D It shows the relationship with Figure 18 Examples of similar color images illuminated by lasers are shown, respectively, with and without modulation of the binary SLM pixel matrix;

[0077] Figure 20A and Figure 20BThe diagram illustrates how continuous scanning with different SLM pixel excitation modes can achieve pixel resolution apparent grayscale image variations using a binary switchable pixel array, where... Figure 20A The pixel pattern of three consecutive scans is shown, and Figure 20B The overall perceived image intensity is shown; and

[0078] Figure 20C and Figure 20D Is it separate from Figure 20A and Figure 20B A similar view, where combinations of different pixel excitation modes and different illumination intensities are used to achieve the same effect only in two consecutive scans. Figure 20B Similar results. Detailed Implementation

[0079] This invention relates to an image projector, its components, and its operating mode.

[0080] The principle and operation of the image projector according to the present invention can be better understood by referring to the accompanying drawings and description.

[0081] By way of introduction, the present invention relates to various aspects of image projectors in which one or more lasers are used to provide scanning illumination for a spatial light modulator (SLM). The subject matter described herein can be subdivided into several aspects of the invention, each existing independently, but they are most preferably used in combination for advantages.

[0082] The first aspect of the invention relates to enhancing image uniformity at the exit aperture of an image projector by employing various angular beam expander elements, typically diffusers or microlens arrays (MLAs), adjacent to or in the conjugate plane of the SLM, and corresponding structural features of the image projector device.

[0083] Another aspect of the invention relates to an image projector with a novel approach that synergistically employs an SLM and scanning laser illumination to generate a high-quality projected image, and in some cases, the image projector is suitable for use with a binary switchable SLM having only two brightness levels per pixel.

[0084] Another aspect of the invention relates to various improved techniques for the synchronous operation of SLM and scanning illumination systems. These and other aspects of the invention will become clearer from the following description.

[0085] Various aspects of the present invention are presented in the context of an image projector having projection optics that collimate the projected image. This implementation is particularly suitable for image projectors that introduce a collimated image into a light-guide optical element (LOE, alternatively called a waveguide) having two principal parallel outer surfaces for transmitting the image to the user's front via total internal reflection, wherein the image is coupled out toward the user's eyes, a common practice in virtual reality and augmented reality display systems. Such systems, as preferred embodiments of the invention, typically employ an arrangement of partially reflective inner surfaces angled to the principal surfaces of the waveguide or diffractive optics to progressively couple the image illumination toward the user's eyes. Such arrangements are well known in the art and are commercially available from many sources, and therefore will not be described in further detail here. However, it should be noted that the present invention is not limited to waveguide-based displays, and can be implemented using alternative designs of projection optics, such as projection optics with finite focal lengths for a range of different applications, including but not limited to: non-waveguide-based virtual reality displays, and projectors for projecting images onto a surface for rear or front projection onto a screen.

[0086] Various aspects of the present invention can also be applied to devices employing a wide range of different types of spatial light modulators (SLMs), including transmissive SLMs such as liquid crystal displays (LCDs) and reflective SLMs such as digital light processing (DLP) chips or liquid crystal on silicon (LCOS) modulators. In LCD and LCOS implementations where modulation is achieved through polarization changes, the optical devices implicitly include various polarizers and / or other elements required to select image illumination from illumination to be excluded from the image. These components are standard features of such image generation devices and are understood to be present even if not explicitly described. Configurations herein are shown as single-chip implementations, which can be monochrome displays or can generate color output images using illumination of three different colors (sequentially or in parallel, as will be discussed in some implementations below). However, it should be noted that various aspects of the present invention can also be used to implement color displays based on a three-chip architecture, as will be apparent to those skilled in the art.

[0087] Turning now to the first aspect of the invention, illuminating the SLM with a scanning beam of laser illumination offers numerous advantages (such as those described below), but presents challenges in achieving uniform filling of the exit aperture. Specifically, scanning laser illumination is typically highly directional, while the properties of the SLM are generally nondispersive for transmissive SLMs and primarily specular for reflective SLMs. Therefore, each region of the SLM is typically illuminated by a beam with a relatively narrow range of incident angles, and the modulated beam propagating toward the projection optics will therefore also typically cover the correspondingly narrow angular range, potentially causing only partial filling of the exit aperture. On the other hand, if the laser illumination is widely propagated at the source or within the scanning optics, this would compromise the ability to precisely control the illumination pattern on the SLM.

[0088] To address this problem, the present invention provides various solutions for implementing angular beam expander elements (typically diffusers or microlens arrays (MLAs)) adjacent to or in the conjugate plane of the SLM, thereby achieving efficient filling of the exit aperture while minimizing the impact on the scanning accuracy of the laser illumination on the SLM.

[0089] Now go to Figure 2A This illustrates an image projector constructed and operated according to an embodiment of the invention for projecting an image via an exit aperture (exit aperture) 24, the image being a representation of an input digital image. Generally, the image projector includes a spatial light modulator (SLM) 20 that provides a two-dimensional array of pixel elements, each of which can be controlled to modulate the properties of light transmitted or reflected by the pixel element. For simplicity, the SLM 20 is shown here with a transmissive SLM geometry, but the optical equivalent system can employ a reflective SLM.

[0090] The scanning illumination device is deployed to scan an illumination beam of a two-dimensional array of trans-spatial light modulators in a two-dimensional scanning mode, while simultaneously modulating the intensity of the illumination beam. Structurally, the scanning illumination device preferably includes: one or more lasers 10, most preferably a group of at least three different colors of lasers, and in some preferred cases, an array of six lasers including two lasers for each of the three colors; beam-shaping optics 12; and a scanning mechanism 14, which typically includes: two fast scanning mirrors driven to rotate about two orthogonal axes, or a single biaxial mirror, all of which have suitable actuator and driver circuitry as known in the art. Only the major sub-components of the scanning illumination device are shown schematically here, and structural details will be clear to those skilled in the art.

[0091] Controller 15 is electronically connected to SLM 20 and scanning illumination device (laser 10 and scanning mechanism 14). Controller 15 typically includes one or more processors, data storage components, and input / output components sufficient to perform the various functions described herein for receiving input digital images (e.g., video image sequences) and synchronously stimulating the SLM and scanning illumination device to generate a projected image as a desired visual representation of the input digital images. Controller 15 may be subdivided into multiple different sub-controllers, which may be housed together or positioned separately, and in some cases, at least some of the processing of controller 15 may be performed remotely from the device, for example by preprocessing video signals provided to the device via network communication. All such implementations of controller 15 will be apparent to those skilled in the art. Details of the controller components are omitted herein to simplify the presentation of the invention, but will be clear to those skilled in the art from the following description of their functions. Controller 15 is omitted in most of the following figures for simplification, but it should be understood that controller 15 is present in every embodiment.

[0092] A projection optics 22 is also provided, which provides at least one optical element configured to project illumination from a spatial light modulator to generate an output image guided to an exit aperture 24. Here, the optical element is schematically shown as a single refractive lens, but it can be equivalently implemented using reflective optics, additional or compound refractive elements, or any combination thereof. In the implementation schematically shown here, the projection optics are collimating optics that output a collimated image, wherein each pixel of the image is transmitted by a collimated beam having a corresponding angular direction, the collimated beam being adapted to be input to a waveguide for an augmented reality display. As described above, other implementations, for example, having a converging projected image focused at a given distance, also fall within the scope of the invention.

[0093] The system features described so far are largely general to all embodiments of the invention described below, and should be considered applicable to all embodiments unless otherwise stated.

[0094] In the particularly preferred, non-limiting embodiment shown herein, the image projector further includes an angular beam expander 16 deployed at the illuminated image plane focused by the scanning laser illumination. Thus, the scanning illumination device generates a modulated scanning illumination pattern at the illuminated image plane, falling onto the angular beam expander 16. Light from the illuminated image plane is refocused by an intermediate optics 18 with positive optical power, deployed in the optical path between the illuminated image plane and the SLM 20, such that light falling from the illuminated image plane is refocused at the SLM 20. The angular beam expander may be a diffuser, but is most preferably a microlens array (MLA).

[0095] The function of MLA or diffuser is determined by Figure 2A The difference between the dashed and solid arrows is illustrated. The dashed arrows depict laser beam propagation without an MLA, where illumination at the exit aperture 24 narrows the relatively narrow beam, causing non-uniform image projection into and from the waveguide. The solid arrows show increased divergence from the MLA, which in turn causes wider and fuller illumination at the exit aperture 24, thus providing a more uniform image output across the aperture.

[0096] The angular beam diffuser can be any optical element that disperses the beam with minimal lateral displacement, such as a diffuser or microlens. Preferably, the light dispersion is limited to a predetermined angular range, such that minimal light loss occurs outside the aperture 24.

[0097] In this configuration, to optimize image quality, light efficiency, and image uniformity across all scanning angles, it is preferable to satisfy one or both of the following conditions. In some cases, additional optical elements may be introduced to help satisfy these conditions.

[0098] 1. The plane containing the MLA or other angular beam expander 16 should be imaged onto the plane of the SLM 20. This ensures that the beam expansion achieved by the MLA results in a substantially undistorted illumination scan pattern generated by the scanning illumination device.

[0099] 2. The scanning mechanism 14 is preferably located on the plane of the image on the exit aperture 24, thereby ensuring that the source illumination from the scanning device is effectively guided to the exit aperture.

[0100] exist Figure 2A The two conditions are schematically represented by the corresponding double-ended arrows that identify the conjugate focal plane.

[0101] In an alternative implementation, the positions of the angle beam expander 16 and the SLM 20 can be swapped to modify the configuration. Figure 2AThe optical setup is as follows: In this case, the scanning illumination device illuminates the SLM 20 directly with a relatively narrow illumination beam, and the modulated light from the SLM is refocused by the intermediate optics 18 onto the image plane where the corner beam expander 16 is deployed. The projection optics 22 are deployed to project illumination from the image plane (i.e., falling on the corner beam expander 16) to generate an output image guided to the exit aperture 24.

[0102] exist Figure 2B The figure shows an implementation of the alternative embodiment, and also shows an implementation using a reflective SLM 20, wherein the reflective optics are used for the intermediate optical device 18 and the refractive optics are used for the projection optics 22.

[0103] Light from laser 10 is collimated by lens 12 and scanned onto image generator 20 by one or more fast scanning mirrors 14. Modulated light reflected from 20 is directed to focusing reflective lens 18, which refocuses the beam onto MLA 16. The angularly extended beam for each point in the image is then focused by projection optics 22 and directed to exit aperture 24. The optical paths from the scanning device to SLM 20 and from reflective lens 18 to MLA 16 are reflected in polarization beam splitter (PBS) 21, while the modulated image illumination from SLM 20 passes through PBS 21. (It should be understood here and throughout this document that wherever the PBS configuration is described, waveplates are provided to provide the described reflection or transmission sequence at the PBS, as is standard practice in the art. These elements are implicit and will not be described further herein.) This configuration provides a particularly compact configuration for LCOS optics.

[0104] For projection optics, the use of a reflective intermediate optics followed by a refractive optics is considered particularly advantageous because these two types of optics tend to produce opposite field curvatures, which therefore tend to overlap, resulting in a better-focused image across the entire field. Optionally, the MLA16 can be advantageously bent to match the field curvature at the intermediate image plane. The refractive optics, schematically shown herein as a single lens, are preferably implemented as a dual-lens group or other multi-element lens system configured to reduce chromatic aberration and / or other aberrations, as known in the art.

[0105] Compared to the embodiment of Figure 1A, in this embodiment, the image generator (LCOS 20 in this case) is illuminated by less divergent beam, thus making its image modulation more efficient. Furthermore, in this configuration, the PBS 21 is small and located far from the exit pupil 24, making the system more ergonomic. It should be noted that a hybrid optics device with partial reflection and partial refraction can also be used in the implementation of the optical configuration of Figure 1A, where a reflective MLA 16 replaces the LCOS 20, and the LCOS 20 is placed in the position shown for the MLA 16, either implemented as a transmissive SLM, or another PBS prism (not shown) is used before the refractive optics of the projection optics to allow the use of a reflective SLM.

[0106] Now go to Figure 3 As a response Figure 2A and Figure 2B A more compact alternative to the option of positioning the angle beam expander in a plane conjugate to the SLM plane is an alternative implementation that uses an angle beam expander deployed adjacent to the SLM itself. Figure 3 An example of this concept is illustrated in the diagram, and... Figure 4A and Figure 4B An example of this concept is adaptively illustrated by an exemplary implementation using reflective optics and a PBS prism.

[0107] Reference Figure 3 The diagram illustrates an angular beam expander, such as an MLA 16, placed adjacent to the SLM 20. The MLA 16 can be placed in front of, behind, or to the sides of the transmissive SLM 20. In the case of a reflective SLM, the adjacent MLA will refract the light twice, before and after reflection. Clearly, in this configuration, the optics are significantly simplified, eliminating the need for intermediate optics 18 and associated alignment and mechanical structures.

[0108] As previously mentioned, it is preferable to achieve image formation of the plane of the scanning mirror 14 onto the exit aperture 24. This can be facilitated optionally by means of field lens 30A and / or field lens 30B.

[0109] Figure 4A This shows a reflective SLM (LCOS 20). Figure 3 The optical implementation of the configuration is as follows: Light from laser 10 is reflected and focused by curved reflector 12R (equivalent to lens 12). Optionally, lens 34 can be added to improve imaging from scanner 14 onto exit aperture 24. The optical path is defined in a highly compact and efficient manner by PBS 21. The doublet lens, including 288 and 286, acts as a... Figure 3The field lenses 30A and 30B act as shown, but the light passes through them twice, before and after being reflected by the LCOS 20. The MLA 16 is positioned adjacent to the LCOS 20 and also acts on the light twice, before and after being reflected by the LCOS 20.

[0110] Figure 4B Only the dispersive beams from the LCOS reflected at different locations across the LCOS, generated by different positions of the scanning mirror 14, are shown. Each location on the LCOS generates a set of divergent beams extended by the MLA 16, which are collimated by the projection optics 22 to produce a set of parallel beams for each image point, such that the beams for each image point fill the exit aperture 24 and converge toward that aperture, thereby indicating good imaging from the plane of the scanner 14 to the plane of the exit aperture 24.

[0111] In a particularly preferred implementation shown here, the projection optics 22 is implemented as a birefringent lens comprising a reflective element 291 and one or more refractive elements 293 having a combination of refractive and reflective optical power configured to generate a flat field (focal plane) on the LCOS 20.

[0112] As described above, the angle beam expander in any of the above configurations can be implemented as a diffuser. When using a diffuser, a structured diffuser is most preferred, which is designed to diffuse the incident light into a predetermined angular distribution to avoid significant waste of light scattered at higher angles that will not contribute to image projection and will generate noise. Optionally, for example, a non-circular distribution, such as a rectangle, can be selected. A range of structured diffusers with different forms of angular distribution can be obtained from, for example, engineered diffusers available from RPC Photonics (New York, USA). TM The diffusers in the series were purchased from various sources.

[0113] In many cases, microlens arrays are considered to offer particular advantages when used in conjunction with SLMs. Figures 5A to 8 The various aspects and options regarding the combination of MLA adjacent to SLM are shown.

[0114] Figure 5AAn example of combining a matrix of microlenses 264 with an LCOS 260 having active reflective pixel elements 262 is shown. The microlenses 264 are advantageously located as close as possible to the active pixels. The microlenses can be convex or concave. The distance between the microlenses 264 and the pixels 262 should be substantially different from (preferably less than) the focal length of the microlenses, referred to herein as “non-focused proximity” to the pixel elements. This ensures that the reflected beam will have a wider divergence than the incident beam. Various ray paths are shown as 266 and 268. Therefore, the divergence 270 of the incident beam is less than the divergence 272 of the reflected beam. Figure 5B The image schematically shows the incident beam 274 of illumination, while the reflected broad beam 276 has a wider divergence.

[0115] To maintain good image resolution, the distance between the microlens 264 and the active pixel 262 should be within the depth of focus of the projection optics 22. Figure 6A The deployment in the pixel matrix (e.g.) is shown Figure 5A Microlenses on the outer side of the substrate in the middle, and Figure 6B A preferred alternative is shown, in which the microlens is located on the substrate surface facing the pixel element, and thus closer to the pixel element. This configuration also has the advantage of providing a flat, outward-facing substrate surface, which facilitates the juxtaposition of the assembly with other optical elements such as PBS prisms.

[0116] Figure 6C This illustrates the implementation of microlenses directly on the surface of an SLM chip, thereby achieving negligible image degradation. In some cases, SLM chips can have higher resolution than required for a given application; in such cases, microlens arrays can be implemented using microlenses larger than the matrix pixels, such as... Figure 6D As shown. On the other hand, maximum resolution is preferably achieved by employing an MLA with one lens per pixel element for the SLM, each lens being aligned with the corresponding pixel element.

[0117] Beam extension can also be achieved by using MLA (Graduated Indices Lens) to vary the refractive index. GRIN (Graduated Indices Lens) arrays are available from various suppliers and can be attached to the LCOS substrate as a window. Figure 6E Another alternative is shown in which a substrate 280 with microlenses 264 is attached to the surface of LCOS 260 via an adhesive medium 284 having a different refractive index than the microlenses 264. This example provides structural integrity and ease of assembly, fixes the MLA relative to the SLM, and provides a flat, outward-facing surface.

[0118] Figure 6F It shows Figure 6EA variation thereof, in which the substrate is part of a field lens extending across the SLM. In the example shown here, the field lens is a doublet lens with components 286 and 288, thereby introducing optical power, as described above. Figure 4A As shown. Most preferably, the outermost portion of the lens provides a flat, outward-facing surface to simplify the components in the optical system.

[0119] Figure 6G An equivalent dispersive MLA in which each microlens has negative optical power is shown. Preferably, this configuration is used with minimal gap between the active pixel elements of the SLM, thereby minimizing light loss.

[0120] Conversely, using an MLA with positive optical power for each microlens can help modify the SLM design to reduce crosstalk between pixels. Figure 7 A more detailed diagram of the LCOS SLM structure is shown, where 304 is a silicon substrate plane, 306 is a pixel electrode, and 307 and 308 are alignment layers. Layer 308 also includes a top electrode. The liquid crystal is 309. An outer window 310 supports an MLA 312 (which can be based on refraction, diffraction, graded refractive index, or other elements and according to…). Figures 6A to 6C , Figure 6E or Figure 6F (Any implementation of the structure shown). In this configuration, a smaller area of ​​each pixel element is illuminated due to the presence of the microlens 312. This allows the electrodes 306 to be spaced apart to prevent cross-field interactions (also known as "edge field effects"). This electrode spacing is possible due to the way the non-focused MLA312 is implemented.

[0121] Suppressing the "edge field effect" helps in the implementation of the "Vertical Alignment Mode" (VA) SLM structure, which has excellent contrast but would otherwise suffer from a high "edge field effect." Using the VA configuration, a large electrode spacing enables high contrast with minimal "edge field effect."

[0122] Figure 8 This illustrates an equivalent implementation of the same non-focused MLA on both sides of a transmissive LCD to achieve equivalent transmission efficiency. In some implementations, implementing the MLA on only one side of the transmissive LCD may be sufficient.

[0123] Turning now to another aspect of the invention, which relates to various improved techniques for the synchronous operation of SLM and two-dimensional scanning illumination systems. The scanning illumination device 14 typically has a fast scanning axis (also referred to as a “resonant axis”) that oscillates at, for example, about 10 kHz, and a slower quadrature axis (also referred to as a “linear axis”) that typically oscillates at several hundred Hz. Figure 9AThe schematic description illustrates an example of a planar SLM 20. By way of example, consider an image generation matrix of an SLM 20 with 1000×1000 pixels operating at a frame rate of 100Hz. Each color frame consists of three separate frames of three colors, illuminated sequentially by red, green, and blue (RGB) illumination to form the visual perception of a color image updated at 100Hz. In this example, the matrix (e.g., LCOS) modulation frame rate should be 300Hz. This rate is controlled by the linear mirror oscillation rate, while the resonant mirror generates a lateral scan much faster than the linear scan.

[0124] Assuming the resonant mirror oscillates at 10kHz, the width of the light spot 120 is ( Figure 9A The vertical line in the middle should be:

[0125] 300Hz × 1000 [lines / frame] / 10kHz = 30 [lines / spot]

[0126] The optical scan trajectory 121 has the following per-frame scan (defined as a scan direction):

[0127] 1000 lines / frame / 30 lines / spot = 33 horizontal scans / frame

[0128] The laser power is modulated to illuminate the desired pattern. The modulation rate determines the length of the laser spot size. Figure 9A (Level in) Figure 9A In the above, assuming fast modulation, an elliptical spot 120 with a ratio of 30:1 is generated, wherein single-pixel modulation is achieved in the scanning direction of the resonant axis.

[0129] This 2D laser scanning speed is suitable for the simplest sequences, including loading an image onto an LCOS and then scanning only the modulated laser onto the LCOS.

[0130] More preferably, the image is loaded onto the LCOS in a rolling mode, wherein data is loaded line by line in sequence, synchronously with the illumination scan. Figure 9A The newly loaded line 122 is shown as a dashed line, with the progress of line loading represented by thick arrows. Line loading proceeds before the scan progresses while maintaining an interval of 124. This interval is chosen to accommodate the response time of the liquid crystal molecules to their own redirection, assuming that switching of the liquid crystal is performed immediately upon receiving pixel data. Figure 9BThe contrast achieved by the liquid crystal pixel as a function of the time since self-excitation is shown. The time required for the crystal molecules to rotate to their optimal orientation makes immediate illumination after pixel activation impossible. Therefore, the interval 124 is chosen to correspond to a time difference 125 sufficient for molecule reorientation. By forming a wide gap 124 between the pixel drive 122 (when it moves downwards) and the illumination scan mode 121 (when it also moves downwards), the liquid crystal molecules have more time 125 to reorient themselves.

[0131] Assuming that the roll-up loading process can only be performed in one direction, the linear scanner will... Figure 9C The zigzag distribution operation is shown. The linear portion of the scan is used for laser illumination of the frame, while between these scans, the scanner performs backtracking without illumination to begin illumination for the start of the matrix for the next frame.

[0132] This rolling shutter method can be implemented using a wide-range SLM image generator with different data loading modes. As another example, Figure 10A and Figure 10B An alternative scheme for bidirectional pixel roll-up loading is shown. Therefore, a linear scan is also performed bidirectionally, first with a downward scan ( Figure 10A ), and then an upward sweep ( Figure 10B ), where the entire scanning motion is as follows Figure 10C As shown in the curve, laser illumination is excited while scanning in both directions. Roller-up loading in both directions can be achieved by loading each line in different sequences (e.g., from different lines on opposite sides of the line) or by activating each second line by different lines (between lines).

[0133] Figures 11A to 11C Another variation is shown, in which the roll-up loading of pixel data to the SLM 122a is performed before the illumination scan mode 120 (as described above), and additionally at 122b after the illumination scan has passed. Therefore, when the linear scan mode scans up and back across the SLM ( Figure 11B The pixel elements have been loaded with the appropriate image. During this back-and-forth scan, the time elapsed from stimulating the pixels with image data until the illumination scan reaches those pixels is different for each line. To address this issue, as... Figure 11C As shown, some delay may need to be introduced between forward and reverse scanning to allow more time for the liquid crystal molecules to rearrange in the correct orientation.

[0134] To reduce contrast variations during backward scans, forward roll-up loading can be instantaneous, i.e., each row is activated as soon as its data is loaded (but a constant margin of 124 is maintained after loading until illumination reaches each row), whereas for backward scans, global triggers activate the entire matrix simultaneously, regardless of the order in which it is loaded. This results in reduced temporal variations across the matrix.

[0135] Figure 12 shows the segmented LCOS matrix, where each half of the roller shutter loading is in opposite directions. The two parts are shown as being separated by a dashed line. Figure 12A In the middle, a downward scan is performed, where 122a1 is an instantaneous activation of the downward roll-up loading, i.e., each row is activated after loading. Therefore, the contrast of these pixels is uniform with a constant time difference 125. The roll-up loading 122b1 after the scan can be an instantaneous activation or a global activation after the scan of the upper portion. Figure 12B A continuous illumination scan through the lower portion, which was previously loaded and activated by 122b2, is shown here.

[0136] exist Figure 12C In the middle, used with Figure 12A The same roll-up loading (represented as 122a2 and 122b2) performs an upward scan. Figure 12D This again demonstrates the equivalent of Figure 12B The optical scan, in which the image has been loaded by 122b1, and the time interval is long enough to achieve stable and uniform contrast in this section.

[0137] The activation of the portions following the laser scan (122b1 and 122b2) can be instantaneous (line-by-line) or triggered global activation.

[0138] Turning now to a particularly preferred feature of certain preferred implementations of the invention, in order to avoid capacitance accumulation (since a constant electric field can adversely affect the properties of the liquid crystal material), the voltage applied to each pixel is preferably reversed between illumination cycles. By combining this reverse excitation with a rolling shutter excitation scheme, it is possible to achieve faster switching and faster frame rates than in other possible approaches.

[0139] For this implementation, the display controller 15 (in) Figure 2A (As shown in the document and implied throughout the specification) is preferably configured as follows:

[0140] (i) Stimulate the SLM by rolling up the pixel data of the pixel row that moves slowly along the scanning mode.

[0141] (ii) Driving the scanning illumination device to illuminate the pixel row that is advancing slowly in the direction after a roll-up update of the pixel data; and

[0142] (iii) As the pixel row following the illumination beam is updated in a rolling fashion, a reverse excitation is applied to the pixel elements of the SLM.

[0143] Figures 13A to 13C A preferred sequence of operations according to this aspect of the invention is illustrated schematically. Figure 13A The time series across a given pixel is shown. Initially (time period 146), the pixel capacitor is charged according to the desired pixel image. Then, an excitation voltage 148A is activated on the pixel, causing the liquid crystal to respond according to a contrast response distribution 150A (according to the SLM design, the contrast response distribution 150A can transmit the state of the pixel (making the pixel state bright) or block the state of the pixel (making the pixel state dark) by polarization rotation). During this time period, the pixel is illuminated (illumination intensity 152). After (or during) the pixel is illuminated, the voltage on the pixel stops and the LCD begins to decay 150B. At this time, a negative voltage 148B is applied (with or without a gap between 148A and 148B), and the liquid crystal decays to a zero contrast response.

[0144] Figure 13B The same process as the spatial progression of matrix 118 across LCOS is illustrated, where sub-controller 152 controls the activated line and the voltage applied to that line, while sub-controller 154 drives information for the selected row (charging the pixel's capacitor). Here, line 156A represents the row currently loading data into the pixel capacitor (in... Figure 13A The time interval is 146 (represented as pixel cycle time). Region 158A represents the region currently provided with a positive voltage (pixel cycle time interval 148A), and region 160 represents the reverse voltage activated after illumination scan 120 has illuminated a specific region (pixel cycle time interval 148B). Activation of load 156B and the next frame 158B can begin immediately after 160, thus giving the liquid crystal more time to respond to the next frame. This configuration is suitable for excitation schemes that include multiple loads per scan frame (e.g.,...). Figure 11A (as in the example).

[0145] exist Figure 13C A similar but slower process is described. Here, 161 represents the long activation of the LCOS before the scanning illumination 120 passes through it, and 157 is the loading after the reverse voltage 160. This configuration is suitable for a single scan per frame (e.g., Figure 10A middle).

[0146] The fact that illumination on the image plane is controllable and can be selectively applied helps reduce the power consumption of additional operating mode LCOS by activating fewer pixels. For example, when no voltage is applied to a pixel, the LCOS operating in a Mixed Mode Twisted Nematic (MTN) configuration is typically off (bright). Blocking pixels (darkening them) requires power. This typically results in an image with a relatively small number of bright pixels, and most of the image is dark (as is common in augmented reality applications), which is highly energy-intensive.

[0147] Figure 14A The desired image intensity distribution along the pixel rows of the image is shown, corresponding to a single bright pixel 230 in a dark pixel row in the image plane. Superimposed on the desired intensity distribution is the intensity distribution of the laser illumination spot (fixed) used for scanning across LCOS at the image plane (distribution 232). A conventional method for generating the desired image intensity distribution is to excite all pixels in the row to direct them toward the blocking (dark) pixels, which consumes a large amount of energy.

[0148] As an alternative method according to aspects of the present invention Figure 14B A preferred form of excitation for the pixel row of an MTN-based LCOS is shown. Pixels at appropriate locations 234 are set to be off, while pixels adjacent to the desired pixels in areas requiring darkness and within the coverage area of ​​the laser illumination spot distribution 236 are set to block light as much as possible. These activated dark pixels 236 consume system power. Pixels located in region 238 far from the desired bright pixels are not activated and thus remain "bright" without consuming energy. However, the area of ​​the output image corresponding to pixel region 238 will remain dark because the controller does not excite the laser illumination when the illumination system is scanning in that area. Therefore, dark areas remain dark but will not consume power because these LCOS pixels are not activated. Alternatively, for partial power savings, pixels in region 240 may be only partially blocked because the light power in that area is low, and even partial pixel blocking will cause the output illumination intensity in that area to be below an observable level.

[0149] To achieve high contrast, light in the dark segments of the image must be sufficiently attenuated. Generating high-contrast LCOS pixels requires a relatively long response time, which reduces performance. However, due to selective illumination activation, the lower contrast of the LCOS is acceptable because the final image contrast (considering the unilluminated dark segments) is high. Therefore, the liquid crystal orientation in the LCOS can advantageously be set to maximum speed rather than maximum contrast (a practice known in the art). This may result in segment 236 having some residual transmittance, which, after illumination, will give an output illumination intensity distribution with a residual margin 242 on each side of pixel 230, such as... Figure 14C As shown in the image.

[0150] The above-described operating mode for displaying a portion of an image with dark areas is an example of an operating mode that only partially excites the SLM. This invention provides many other useful operating modes for exciting the SLM only on one or more sub-regions of a pixel array, which will be discussed further below.

[0151] The scanning pattern of mirror 14, generated by slower linear motion superimposed on faster oscillations, typically produces a nonlinear and non-parallel scan path across LCOS (image plane) 218. For various reasons, it may be preferable to at least partially compensate for this nonlinearity and / or angular spacing of the scan path, for example, by making the time interval 125 (image interval 124) more constant across the scan line, thereby causing the liquid crystal (...) Figure 9B or Figure 13A A more uniform response distribution along the line (150 in the middle). Figures 15A to 15C One method for mitigating this effect that linearizes illumination scanning is shown.

[0152] Figure 15A The light spot patterns of six lasers are shown, including two red (R1 and R2), two green (G1 and G2), and two red (R1 and R2). Figure 15B A green laser (shown only in green for clarity) is illustrated, where the spot is defocused 264 into a larger spot (shown here as elongated). Drawing 266 shows a distribution with only laser G1, 268 shows a spot where G1 and G2 are both active, and 270 shows a distribution where only G2 is active. It is evident that switching between lasers (G1 and G2, or more lasers in a row, if present) shifts the centroid of the illumination mode. Figure 15C The implementation of this property, which linearizes the scanning pattern, is illustrated. 272 ​​and 274 represent the defocused spots of G1 and G2. Pattern scan 276 represents the nonlinear scanning pattern generated by the biaxial motion of mirror 14. Line 278 represents the desired linear horizontal illumination line parallel to the roll-up loading line 256 of matrix 118. The linearization of illumination is approximated by switching illumination between G1 and G2. At the beginning of scan line 276, the lower laser 274 is illuminated (shown as a full ellipse), while at the end of the scan, the upper laser 272 is activated, and the linear transition between the lasers generates the approximate correction line 278. If used, a similar laser activation (but in the opposite pattern) is activated on the opposite scan 280.

[0153] Other scanning modes are also possible, and the same activation is advantageously applied to both the red and blue lasers. In all of the above, if the matrix includes color-filtered pixels, all lasers can be activated simultaneously without requiring frames for each color, as each pixel will only act on its corresponding color.

[0154] All of the above can be applied to a transparent LCD because its physical properties are the same as those of an LCOS.

[0155] Another aspect of the invention relates to an image projector with a novel approach that synergistically employs an SLM and scanning laser illumination to generate a high-quality projected image, and in some cases, is suitable for use with a binary switchable SLM having only two brightness levels per pixel.

[0156] Specifically, according to certain particularly preferred implementations of the present invention, the controller 15 (in) Figure 2A (As shown in the document and implicit throughout the specification) is configured to:

[0157] (i) Processing the input digital image to generate a first modified image, which substantially corresponds to a reduced-resolution version of the input digital image, with a resolution less than that of the SLM in at least one dimension;

[0158] (ii) Processing the input digital image to generate a second modified image, the second modified image including pixel modulation data for pixel elements of an SLM corresponding to high spatial frequency variations in the input digital image;

[0159] (iii) Excite the SLM based on the pixel modulation data of the second modified image; and

[0160] (iv) The scanning illumination device modulates the intensity of the illumination beam while scanning across a two-dimensional array to illuminate the SLM according to a first modified image, wherein the SLM thereby provides high spatial frequency modulation of the illumination to project an output image with pixel intensity corresponding to the digital image.

[0161] In other words, the modulation intensity of the scanning illumination device is used to generate a reduced-resolution rendering of the desired digital image, while the higher resolution of the SLM is used to "sharpen" the image. This method can be applied to a wide range of SLM types, but it is particularly attractive in the case of binary-excited SLMs, in which each pixel element can switch between a bright and dark state, and especially for ferroelectric liquid crystal displays.

[0162] exist Figure 16The diagram illustrates a possible implementation of image processing for an LCOS-based scanning laser system implemented by controller 15. A "nominal image" (input digital image) is injected into the image processing of both the laser and the LCOS. The image is processed differently for each component in the system, and the two processed images are injected into their appropriate drivers. The light generated by the laser is modulated by the laser driver while the scanning mirror scans across the image field. The light from the scanner illuminates portions of the LCOS image that are further modulated to generate a high-resolution image, which is then injected into an optical combiner (waveguide) or otherwise projected for viewing by a viewer.

[0163] Most applications for augmented reality near-eye displays require high resolution to render data such as letters, numbers, or symbols. This type of data is typically presented to the viewer as a binary image, where pixels are turned on or off using minimal grayscale modulation.

[0164] According to one embodiment of the invention, a fundamental simplification is achieved by setting the LCOS to activate pixels in a binary mode, where the pixels are either "open" or "closed." Ferroelectric liquid crystals, for example, can be used in such applications. Depending on the system's contrast requirements, the "closed" position can be partially closed. Here, for simplicity, "closed" will be considered to mean that the pixel does not exhibit transparency.

[0165] Figure 17A The following example is shown: where the laser illumination spot is larger than the pixel size (as previously stated), and therefore has rounded corners (solid lines), and cannot generate sharp edges (dashed lines) that also represent the edges of pixels. This illumination is approximated by implementing binary LCOS as follows: Figure 17B The desired illumination is shown. Here, in locations where there is no intensity in the nominal image, LCOS pixels are turned off. Figure 17C An example with a larger laser spot size is shown, and Figure 17D This shows that LCOS generates a sharper intensity distribution.

[0166] Figure 18 An example of this operating mode is shown below. The "digital image" is the image injected into the system. It mainly consists of black pixels, some gray, and some white. The width of the letters is one or two pixels. The "laser illumination image" is generated by a Gaussian distributed laser beam with a width of three pixels. The letters are indistinguishable and unobservable. After activating binary LCOS, the letters are recognizable and separable. The grayscale is almost imperceptible.

[0167] The same binary modulation can be achieved in sequential laser scanning illumination used for color. Figure 19AThe distribution in the matrix represents the different color illumination distributions applied sequentially along the rows of the matrix after the matrix has been properly set according to the corresponding color separation frames of each scan.

[0168] Figure 19B This method can be implemented for use with Figure 9A A conceptually similar two-dimensional scanning illumination mode utilizes the synchronous updating of each color. In this case, most preferably, a multi-laser source with spaced-out RGB lasers that can operate simultaneously and independently is used. As three scans travel simultaneously on the SLM, parallel synchronous raster scanning can be performed across the SLM using the rolling-up reset of pixels and the rolling-up update of pixel values ​​to the binary pixel values ​​required for the next color. By utilizing all three colors for scanning simultaneously, the linear scan speed can be slowed down, thereby increasing the effective resolution of the laser illumination system at a given overall frame rate.

[0169] Figure 19C It shows having with Figure 18 Examples of laser illumination with the same parameters, while Figure 19D The image is shown after the illumination has been modulated by a binary matrix.

[0170] For further grayscale resolution, sequential scanning of the same color can be achieved, such as... Figures 20A to 20D As shown. In Figure 20A In this process, three separate sequential illuminations with equal power are binary modulated to be perceived by the observer as Figure 20B Intensity distribution. Figure 20C Sequential illumination with different powers is shown to generate the same distribution, but only through two scans.

[0171] It should be noted that the various operating modes do not require the SLM to be fully activated at all times. Selective and partial activation of the SLM can provide the following advantages: energy saving (and thus extended battery life in battery-operated applications), reduced image processing load, and / or may contribute to faster frame rates.

[0172] Therefore, according to certain particularly preferred implementations of the present invention, the controller 15 (in) Figure 2A (As shown in the document and implicit throughout the specification) is configured to:

[0173] (i) Applying at least one criterion to determine the sub-region of the SLM required for output generation in the corresponding subfield of the output image, the controller is configured to activate the sub-region of the SLM while deactivating multiple pixel elements outside the sub-region; and

[0174] (ii) Driving the scanning illumination device to illuminate at least a sub-region of the SLM to project an output image having a pixel intensity corresponding to the digital image onto at least a corresponding subfield, the pixel intensity being determined by the modulation state of the pixel element and the illumination intensity from the scanning illumination device falling on the pixel element.

[0175] An example of such an application is when an image is selectively displayed only in a sub-region of the image plane, as referenced above. Figures 14A to 14C As described. In this case, the "standard" is that the desired image has only non-zero (or visible) values ​​within the sub-region, and the scanning illumination device is excited to transmit illumination outside the desired sub-region.

[0176] Other applications do indeed deliver illumination outside the excitation region. This is particularly relevant when pixel elements are bright when unexcited. In this case, controller 15 is configured to drive the scanning illumination device to modulate the intensity of the illumination beam while scanning both the interior and exterior of a sub-region of the SLM. The result is that the sub-region of the image in which the SLM is excited has the benefits of enhanced resolution, sharp edges, or other high spatial frequency characteristics provided by SLM modulation, while the rest of the image is limited by the inherent resolution of the scanning illumination system used to form the image. Therefore, the result can be considered a "mixed resolution" image.

[0177] To continue with the above example where the SLM has 1000x1000 pixels and the scanning illumination device operates at a resonant scanning frequency of 10 kHz, if the frame rate is reduced to 33 Hz, coverage of each color separation frame can be achieved using a laser spot covering a 10-row pixel matrix. If the modulation frequency is adjusted accordingly, the original resolution of the laser scanning projection will be equivalent to a 10x10 pixel area of ​​the SLM array. Therefore, as described above, the areas where the SLM is not excited will be projected at the laser scanner resolution, while one or more sub-regions where the SLM is excited will be enhanced through SLM modulation.

[0178] Several different criteria can be used as a basis for determining which sub-region(s) of an image should be excited by the SLM. In a first example, at least one criterion includes identifying regions of the input digital image that contain high spatial frequency content. For example, if the image contains sub-regions with textual content and other regions with gradually changing characteristics, the inherent resolution of the scanning laser projector may be sufficient for the non-textual content, and the SLM can be selectively excited in the textual regions to make the textual content sharp.

[0179] In another example, at least one criterion may include: determining a subfield of the output image corresponding to the viewer's current gaze direction (as determined by an eye-tracking system—not shown). This method relies on the fact that the peripheral vision of the human eye has a much lower resolution than the central (foveal) region, making the eye insensitive to a reduction in image resolution outside the current gaze direction. Therefore, the system can selectively excite the SLM in the region surrounding the current gaze direction to provide a "full-resolution" projected image while displaying the peripheral field at the inherent scanning laser projector resolution.

[0180] It should be understood that the above description is intended to be illustrative only, and many other embodiments are possible within the scope of the invention as defined in the appended claims.

Claims

1. An image projector for projecting an image via an exit aperture, the image being a representation of an input digital image, the image projector comprising: (a) A spatial light modulator (SLM) that provides a two-dimensional array of pixel elements defining the resolution of the SLM, each of which can be controlled to modulate the properties of light transmitted or reflected by the pixel element. (b) A scanning illumination device, which is deployed to scan the illumination beam across the two-dimensional array of the spatial light modulator in a two-dimensional scanning mode, while simultaneously modulating the intensity of the illumination beam; (c) A controller, which is electronically connected to the spatial light modulator and the scanning illumination device; as well as (d) A projection optics device comprising at least one optical element configured to project illumination from the spatial light modulator to generate an output image guided to the exit aperture. The controller is configured to: (i) Process the input digital image to generate a first modified image, the first modified image substantially corresponding to a reduced-resolution version of the input digital image whose resolution is smaller than the SLM resolution in at least one dimension; (ii) Processing the input digital image to generate a second modified image, the second modified image including pixel modulation data for the pixel elements of the SLM corresponding to high spatial frequency variations in the input digital image; (iii) Excite the SLM according to the pixel modulation data of the second modified image; and (iv) Driving the scanning illumination device to modulate the intensity of the illumination beam while scanning across the two-dimensional array to illuminate the SLM according to the first modified image, wherein the SLM thereby provides high spatial frequency modulation of the illumination to project an output image having pixel intensities corresponding to the digital image. The controller is further configured to: apply at least one criterion to determine a sub-region of the SLM required to generate high-resolution output in a corresponding subfield of the output image; the controller is configured to: excite the sub-region of the SLM while de-exciting a plurality of pixel elements outside the sub-region.

2. The image projector according to claim 1, wherein, The two-dimensional scanning mode has a fast direction aligned with the rows of the SLM and a slow direction aligned with the columns of the SLM, wherein the SLM is excited as the pixel rows advancing in the slow direction are updated in a rolling fashion in front of the illumination beam.

3. The image projector according to claim 2, wherein, The controller is also configured to apply a reverse excitation to the pixel elements of the SLM as the pixel row following the illumination beam is updated in a rolling fashion.

4. The image projector according to claim 1, wherein, The SLM is a ferroelectric liquid crystal display in which each pixel element can switch between a bright state and a dark state.

5. The image projector according to claim 1, wherein, The pixel element is lit when not excited, and the controller is configured to drive the scanning illumination device to modulate the intensity of the illumination beam when scanning both the interior and exterior of the sub-region of the SLM.

6. The image projector according to claim 5, wherein, The at least one criterion includes: determining a region of the input digital image that contains high spatial frequency content.

7. The image projector according to claim 5, wherein, The at least one criterion includes: determining a subfield of the output image that corresponds to the viewer's current gaze direction.

8. The image projector according to claim 1, wherein, The controller is configured to drive the scanning illumination device to illuminate only the sub-region of the SLM.

9. The image projector of claim 1, further comprising an angle beam expander deployed adjacent to the SLM.

10. The image projector of claim 1, further comprising a microlens array (MLA) deployed in a non-focused proximity to the SLM.

11. The image projector according to claim 10, wherein, The MLA includes a lens for each pixel element of the SLM, and each lens is aligned with the corresponding pixel element.

12. The image projector according to claim 10, wherein, The MLA is attached to the surface of the SLM.

13. The image projector according to claim 10, wherein, The MLA is integrated with a transparent substrate covering the SLM.

14. The image projector according to claim 13, wherein, The MLA and the SLM are facing each other, and the substrate provides a flat surface opposite to the MLA.

15. The image projector according to claim 14, wherein, The MLA is formed of a material having a first refractive index, and wherein a transparent adhesive fills the space between the MLA and the SLM, the transparent adhesive having a second refractive index different from the first refractive index.

16. The image projector according to claim 14, wherein, The substrate is part of a field lens that extends across the SLM.

17. The image projector according to claim 16, wherein, The field lens is implemented as a doublet lens, which presents a flat surface opposite to the SLM.

18. The image projector according to claim 1, further comprising: (a) An intermediate optical device with positive optical power, which is deployed in the optical path between the SLM and the collimator, the intermediate optical device refocusing the illumination from the SLM onto the image plane; as well as (b) An angular beam expander, deployed at the image plane, The projection optics are configured to project illumination from the image plane to generate an output image that is guided to the exit aperture.

19. The image projector according to claim 1, further comprising: (a) An angular beam expander deployed at the illumination image plane, wherein the scanning illumination device is configured to generate a modulated scanning illumination pattern at the illumination image plane; as well as (b) An intermediate optical device with positive optical power, which is deployed in the optical path between the illumination image plane and the SLM, the intermediate optical device refocusing the illumination from the illumination image plane at the SLM.

20. The image projector according to claim 18 or 19, wherein, The angular beam expander is a microlens array.

Images