Near-eye display with overlapping projector assemblies

Abstract

To provide a display and a method for providing an image to an eye of a viewer.SOLUTION: A display comprises at least two projector assemblies. Each projector assembly comprises a light-guide optical element (LOE), and an image projector arrangement for generating a partial image and being deployed to introduce the partial image into the LOE for coupling out towards the eye of the viewer. The at least two projector assemblies cooperate to display the image to the eye of the viewer with partial overlap. The display further comprises a controller associated with the image projector arrangements and configured to reduce a pixel intensity of selected pixels in a region of partial overlap between the first and second parts of the image so as to enhance a perceived uniformity of the image.SELECTED DRAWING: Figure 2C

Description

[Technical Field]

[0001] The present invention relates to displays, and more particularly to displays for presenting images to a viewer's eyes.

[0002] For applications such as near-eye displays, a projected image with a large field is desirable. This is typically achieved by introducing a large field image from a single image projector into a waveguide. The waveguide expands the aperture of the projected image, thereby illuminating the eye with the large field image.

[0003] However, to achieve such an aperture expansion, a large projector and / or large optics are typically required, which is a disadvantage for use in near-eye displays and other applications where the display must be small to be usable in the desired application. Furthermore, the angular size and outcoupling arrangement of the field from a given waveguide are limited by geometric optical considerations such as the range of angles that can be trapped within the waveguide for propagation by internal reflection, and by avoiding overlap between the image and its conjugate within the waveguide. Summary of the Invention

[0004] In accordance with the teachings of the present invention, there is provided a display for providing an image to an eye of an observer, the display including: (a) at least two projector assemblies, each projector assembly including: (i) a light-directing optical element (LOE) having a pair of parallel exterior surfaces; and (ii) an image projector arrangement for generating a partial image, the image projector arrangement being arranged to introduce the partial image from the image projector arrangement into the LOE for propagation within the LOE by internal reflection from the pair of parallel exterior surfaces, each projector assembly including an outcoupling arrangement relative to the LOE, the image projector arrangement being configured to outcoupling the partial image from the LOE to the eye of the observer, wherein the LOE of a first projector assembly of the projector assemblies projects a first partial image corresponding to a first portion of an image in overlapping relationship with the LOE of a second projector assembly of the projector assemblies; and (b) at least two projector assemblies, the second projector assembly being arranged to project a second partial image corresponding to a second portion of the image, with partial overlap of the first and second portions of the image, such that the at least two projector assemblies cooperate to display an image to an eye of an observer; and (c) a control device including at least one processor, the control device being associated with the image projector arrangements of at least the first projector assembly and the second projector assembly, and configured to reduce pixel intensity of selected pixels projected by at least one of the first image projector arrangement and the second image projector arrangement, the selected pixels being in the area of the partial overlap between the first and second portions of the image, to improve perceived uniformity of the image.

[0005] In accordance with further teachings of the present invention, a display for providing an image to an eye of an observer includes: (a) a projector assembly, the projector assembly including: (i) a light-directing optical element (LOE) having a pair of parallel exterior surfaces, and two non-parallel sets of mutually parallel reflective surfaces, the LOE configured for 2D aperture placement of images propagating therethrough; and (ii) at least two image projector arrangements that generate at least two partial images corresponding to at least a first portion of the image and at least a second portion of the image, respectively, the at least two image projector arrangements arranged to introduce the at least two partial images into the LOE for propagation within the LOE by internal reflection from the pair of parallel exterior surfaces; (b) a control unit including at least one processor associated with the at least two image projector arrangements and configured to reduce pixel intensities of selected pixels projected by at least one of the first image projector arrangement and the second image projector arrangement, the selected pixels being within the area of partial overlap between the first image portion and the second image portion to improve perceived uniformity of the image.

[0006] According to the teachings of the present invention, there is provided a method for providing an image to an eye of an observer, the method including: generating, by a first projector assembly including a first LOE and a first image projector arrangement, a first partial image corresponding to a first portion of the image for outcoupling to the observer; generating, by a second projector assembly including a second LOE and a second image projector arrangement, a second partial image corresponding to a second portion of the image for outcoupling to the observer, wherein the first LOE and the second LOE are arranged in an overlapping relationship such that the first and second portions of the image are outcoupled to the observer with partial overlap, and the projector assemblies cooperate to display the image to the observer's eye; determining, by a controller associated with the first image projector arrangement and the second image projector arrangement, a subset of pixels in the area of partial overlap; and reducing, by the controller, intensities of selected pixels in the subset of pixels, the selected pixels being projected by at least one of the first image projector arrangement and the second image projector arrangement to improve the perceived uniformity of the image.

[0007] According to some aspects of the invention, a display includes at least a third projector assembly, the at least third projector assembly including: (i) an LOE having a pair of parallel exterior surfaces; and (ii) an image projector arrangement for generating a third partial image corresponding to a third portion of an image, the image projector arrangement being arranged to introduce the third portion of the image from the image projector arrangement into the LOE for propagation within the LOE by internal reflection from the pair of parallel exterior surfaces; and the at least third projector assembly including an outcoupling arrangement associated with the LOE and configured to outcoupling the third partial image from the LOE towards an eye of a viewer. and a projector arrangement, wherein the LOE of the at least third projector assembly is arranged in an overlapping relationship with at least one LOE of the first projector assembly and the second projector assembly such that the at least three projector assemblies cooperate to display an image to the eye of a viewer, wherein the controller is further associated with the image projector arrangement of the at least third projector assembly and configured to reduce pixel intensity of selected pixels projected by the at least one image projector arrangement of one projector assembly, the selected pixels being an area of partial overlap between at least two portions of the image.

[0008] According to some aspects of the invention, the first sub-image and the second sub-image share a set of common pixels, and the selected pixels of reduced intensity are a subset of the set of common pixels.

[0009] According to some aspects of the invention, the controller varies the selection of a subset of the set of common pixels in response to an overlap region adjustment input.

[0010] According to some aspects of the invention, the overlap region adjustment input comes from a pupil position sensor.

[0011] According to some aspects of the present invention, the overlap region adjustment input is derived from a manual user input.

[0012] According to some aspects of the invention, the control device is configured to gradually decrease the intensity of the selected pixels projected by the first projector arrangement across the area of partial overlap and gradually increase the intensity of the selected pixels projected by the second projector arrangement across the area of partial overlap.

[0013] According to some aspects of the invention, the second projector assembly includes a second image projector arrangement arranged to generate a third partial image corresponding to a third portion of the image and introduce the third partial image into the LOE of the second image projector assembly such that said first portion, said second portion, and said third portion of the image have a partial overlap, wherein a control device is further associated with said second image projector arrangement and configured to reduce pixel intensities of selected pixels projected by at least one image projector arrangement of the at least one projector assembly, the selected pixels being in an area of partial overlap between at least two portions of the image.

[0014] According to some aspects of the invention, the LOEs of the at least two projector assemblies are arranged parallel to each other.

[0015] According to some aspects of the invention, the LOEs of the at least two projector assemblies are arranged non-parallel to each other.

[0016] According to some embodiments of the invention, the LOEs are positioned to extend around or partially encompass the observer or the observer's eyes, and the display further includes one or more index-matched media positioned around the observer between the LOEs that form an optically smooth transition with the edges of the LOEs. [Brief explanation of the drawings]

[0017] The present invention is herein described, by way of example only, with reference to the accompanying drawings. [Figure 1A] Figure 1A shows a schematic representation of a wide field projection onto the eyebox; [Figure 1B] Figure 1B shows a schematic representation of a narrow field projected onto the eyebox; [Figure 1C] Figure 1C shows a schematic representation of the projection of a narrow field combination onto the eyebox; [Figure 2A] FIG. 2A shows schematically the first projector assembly having an LOE and image projector arrangement; [Figure 2B] FIG. 2B shows schematically that the second projector assembly has an LOE and two image projector arrangements; [Figure 2C] FIG. 2C illustrates a schematic representation of a display of the present invention according to some embodiments; [Figure 2D] FIG. 2D is a cross-sectional view of the projector assembly and a schematic showing outcoupling towards different pupils; [Figure 2E] FIG. 2E shows a cross-sectional view of the projector assembly and schematic outcoupling towards different pupils; [Figure 2F] FIG. 2F shows schematically selected points in the projected field in angular space of the first pupil position; [Figure 2G] Figure 2G shows schematically selected points in the projected field in angular space at a second pupil position; [Figure 3A] Figure 3A shows an example of the angular distribution of power intensity in a projection image; [Figure 3B] Figure 3B shows an example of the angular distribution of power intensity in the projection image; [Figure 3C] Figure 3C shows an example of the angular distribution of power intensity in the projection image; [Figure 3D]Figure 3D shows an example of the angular distribution of power intensity in the projection image; [Figure 3E] Figure 3E shows an example of the angular distribution of power intensity in a projection image; [Figure 4A] FIG. 4A shows a schematic cross-sectional view of a different configuration of a display according to the invention; [Figure 4B] FIG. 4B shows a schematic cross-sectional view of a different configuration of a display according to the present invention; [Figure 4C] FIG. 4C shows a schematic cross-sectional view of a different configuration of a display according to the present invention; [Figure 5A] FIG. 5A shows a schematic representation of a first embodiment of a display configured for 2D augmented images; [Figure 5B] Figure 5B shows a schematic representation of the horizontally merged image; [Figure 5C] Figure 5C shows a schematic representation of the vertically merged image; [Figure 6] FIG. 6 shows a schematic representation of a second embodiment of a display configured for 2D augmented images; [Figure 7] FIG. 7 illustrates a schematic example functional block diagram of a display according to an embodiment; [Figure 8] FIG. 8 illustrates an example flowchart of a method for displaying an image to a viewer's eye according to an embodiment; [Figure 9A] FIG. 9A shows an example of the angular distribution of power intensity of a projected image according to an alternative embodiment. [Figure 9B] FIG. 9B shows an example of the angular distribution of power intensity of a projected image according to an alternative embodiment. [Figure 9C] FIG. 9C shows an example of the angular distribution of power intensity of a projected image according to an alternative embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0018] DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention provides a display for projecting large field images using small size optical systems by projecting multiple partial narrow field images to be combined and viewed by a viewer as a single large field image.

[0019] The term "field" as used herein should be understood to refer to the field of view of a projected image.

[0020] The term "eyebox" as used herein should be understood to refer to the general area where the pupil is expected to reside while viewing an image. It is expected that the actual pupil position in the eyebox will vary for different observers (e.g., based on interpupillary distance "IPD") and even for a given observer at different times (e.g., based on eye rotation).

[0021] The principles and operation of a display according to the present invention may be better understood with reference to the drawings and accompanying description.

[0022] Referring now to the drawings, FIG. 1A schematically illustrates projecting a wide field onto the eyebox. An image generator (10a) transmits light rays onto optics (12a), which collimate the light rays and illuminate the eyebox (14). As is evident from FIG. 1A, in order to generate an image with a large field, optics (12a) must be relatively large. Also evident from FIG. 1A, a substantial amount of light rays (16a) transmitted through optics (12a) falls outside the eyebox (14) and is therefore "wasted," in the sense that it is not seen by the pupil.

[0023] FIG. 1B schematically illustrates projecting a narrow field onto the eyebox. For a narrow field, the image generator (10b) and optical system (12b) can be smaller compared to the image generator and optical system required to project a large field (as in FIG. 1A). Additionally, fewer light rays (16b) fall outside the eyebox (14) associated with FIG. 1A, and most of the collimated light rays reach the eyebox (14). However, the field is relatively narrow compared to that of FIG. 1A (and even narrower than a person's natural view of the world), resulting in a smaller than desired image that the observer experiences.

[0024] FIG. 1C schematically illustrates the projection of a combined narrow-field image onto the eyebox. Multiple image generators (10c) are used in conjunction with each other, with separate image generators projecting narrow-field partial images, resulting in a much wider-field image reaching the eyebox (14) where the final image reaches. As is evident from FIG. 1C, the image generators (10c) and optical system (12c) can be small (as in FIG. 1B). This results in almost no light rays (16c) falling outside the eyebox (14), yet the observer still advantageously sees a wide-field image, i.e., the combined multiple narrow-field partial images. In FIG. 1C, the dashed lines represent overlapping image data (dashed lines from the center image generator, dashed lines from the left image generator), which are preferably implemented to create a perception of continuity for the observer. Due to physical constraints, this overlap cannot be generated by conventional optical systems without cross-obscuration. According to one aspect of the present invention, a light-directing optical element (LOE) (also referred to herein as a "waveguide") is used to create this overlap without cross-obscuration. A waveguide has a pair of parallel exterior surfaces for total internal reflection of light rays introduced into it.

[0025] Image Projection and Combination FIG. 2A schematically illustrates a first embodiment of an image projector assembly (5a) having an LOE (28a) and an image projector arrangement (20a). Images are shown as dashed lines representing light rays. The image projector arrangement (20a) generates, projects, and couples a partial image into a waveguide (28a). In some embodiments, the image projector arrangement includes a light source, a spatial light modulator such as a liquid crystal on silicon (LCOS), or "LCOS," and collimation optics. As known in the art, these components can be advantageously configured on the surface of a number of beamsplitter prisms, such as a polarizing beamsplitter (PBS) cube.

[0026] The image projector arrangement (20a) is positioned to introduce the partial image into the waveguide, propagating the partial image into the waveguide from a pair of parallel exterior surfaces by internal reflection. The introduction of the partial image into the waveguide is achieved by a suitable optical arrangement, referred to as an incoupling arrangement. The incoupling arrangement typically includes a prism with an appropriately angled surface associated with the LOE or a major surface of the LOE and / or one or more side edges of an incoupling reflector associated with one of those surfaces or that may be positioned inside the LOE. Details of the image projector arrangement, including the incoupling arrangement, are omitted from the schematic diagram to simplify the disclosure. An outcoupling arrangement (7a) (shown as a dashed rectangle on the LOE) associated with the LOE (28a) is positioned to outcoupling the partial image from the waveguide to the observer's eye.

[0027] In some embodiments, the projector arrangement (20a) can be a wide optical arrangement, or can include a distinct optical arrangement for a side aperture arrangement. The outcoupling arrangement (7) is typically implemented as one or more sets of angled, mutually parallel internal partially reflective surfaces or as a diffractive optical element, all as known in the art. The general area of the LOE from which image illumination is outcoupled toward the observer's eyes is indicated by the dashed line.

[0028] FIG. 2B schematically illustrates a second embodiment of projector assembly 5b having LOE 28b and two image projector arrangements 20b1 and 20b2. The two image projector arrangements 20b1 and 20b2 generate and project distinct partial images (shown as dashed lines representing light rays). The partial images are incoupled into waveguide 28b. As a result, each partial image is outcoupled toward the observer (via respective outcoupling arrangements 7b1 and 7b2). The partial images are incoupled into the waveguide at different angles relative to the waveguide. As a result, the outcoupled images do not overlap. It is evident from FIG. 2B that there are gaps between the apertures of the image projector arrangements, which leads to corresponding gaps in the outcoupled partial images.

[0029] FIG. 2C schematically illustrates an embodiment of a display (70) according to the present invention. While in principle there could be more than one projector assembly, the display (70) is implemented by a combination of a projector assembly (5a) (FIG. 2A) and a projector assembly (5b) (FIG. 2B). The projector assembly (5a) includes an image projector arrangement (20a) and an LOE (28a). The image projector arrangement (20a) is configured to generate and project a first partial image corresponding to a first portion of an image. The image projector arrangement (20a) is positioned to introduce the first partial image into the LOE (28a), thereby propagating the first partial image within the LOE by internal reflection from a pair of parallel external surfaces of the LOE. An outcoupling arrangement (7a) (shown as a dotted rectangle on the LOE) associated with the LOE (28a) is positioned to outcouple the first partial image from the waveguide to the observer's eye.

[0030] Projector assembly (5b) includes image projector arrangements (20b1) and (20b2) and LOE (28b). Image projector arrangement (20b1) is configured to generate and project a second partial image corresponding to a second portion of the image. Image projector arrangement (20b2) is configured to generate and project a third partial image corresponding to a third portion of the image. Image projector arrangements (20b1) and (20b2) are positioned to introduce the second and third partial images, respectively, into LOE (28b2), thereby propagating the partial images within the LOE by internal reflection from a pair of parallel exterior surfaces of the LOE. Outcoupling arrangements (7b1) and (7b2) (shown as dotted rectangles on the LOE) associated with LOE (28b) are positioned to outcouple the second and third partial images, respectively, from the waveguide to the observer's eye. Note that in reality, the outcombining arrangement (7a) is associated with projector assembly (5a), but in FIG. 2C it is shown above projector assembly (5b) to illustrate the effect of overlapping outcombined partial images.

[0031] In the embodiment shown in Figure 2C, the first partial image (projected by image projector arrangement (20a)) partially overlaps the second partial image (projected by image projector arrangement (20b1)) and the third partial image (projected by image projector arrangement (20b2)). LOEs (28a) and (28b) are positioned in an overlapping relationship with respect to one another such that projector assemblies (5a) and (5b) cooperate to display images to the observer's eyes.

[0032] It should be noted that while LOE 28a is shown as being located behind LOE 28b, in principle LOE 28a could alternatively be in front of LOE 28b. While an air gap or a layer simulating an air gap is typically necessary to maintain the light-guiding properties of the LOEs, preferably LOEs 28a and 28b should be as close to each other as possible. In some embodiments, it may be desirable to have image projector arrangements 20b1 and 20b2 extend against the sides of the LOEs if the image projector arrangement is wider than its associated waveguide, such that a portion of the image projector extends onto the side of the LOE.

[0033] Preferably, field and aperture continuity as well as pixel intensity uniformity should be maintained when the observer's pupil is at different positions within the eyebox.

[0034] While it should be readily apparent from FIG. 2C that the projector assembly outcombines overlapping partial images, it may be less obvious that not all of the overlapping pixels outcombined to the observer illuminate the pupil, as will be discussed in more detail below with reference to FIGS. 2D-2E.

[0035] Figures 2D and 2E schematically illustrate top-down cross-sectional views of projector assemblies (5a) and (5b), showing partially overlapping partial images outcombined toward the eyebox (14). Figure 2D shows ray directions corresponding to two pixels in the left half of the overall field of view, generated by projector arrangements (20a) and (20b1), and outcombined by outcombining arrangements (7a) and (7b1), respectively. Figure 2E shows ray directions corresponding to two pixels in the right half of the overall field of view, generated by projector arrangements (20a) and (20b2), and outcombined by outcombining arrangements (7a) and (7b2), respectively. These pixels were chosen to facilitate understanding of certain aspects of the present invention, but it will be understood that in use, all pixels of the overall image are simultaneously outcombined to the observer. Two possible pupil locations (15a) and (15b), respectively, are shown in Figures 2D and 2E.

[0036] Figures 2F and 2G correspond to Figures 2D and 2E and illustrate, respectively, selected points (pixels) within the projected field in the angular interval as viewed by the pupil at pupil position (15a) (shown in Figure 2F) and pupil position (15b) (shown in Figure 2G). Figures 2F and 2G illustrate the variation in the observed image with the observer's pupil position.

[0037] "Overlap region," "region(s) of overlap," and "region of partial overlap" may be used to refer to image data that is simultaneously projected by one or more image projection configurations. As noted, typically, a subset of pixels within the overlap region will illuminate the pupil from both projectors at any one time (other pixels reaching the eye from only one projector, while light from the other projector falls to the left or right of the pupil).

[0038] Referring now to Figures 2F and 2G, pixels 1000F, 1002F, 2002F, and 2000F are generated by rays 1000, 1002a / b, 2002a / b, and 2000, respectively (shown in Figures 2D-2E). In both Figures 2F and 2G, like-numbered pixels correspond to the same image information, as indicated by their location at the same position within the image field.

[0039] Referring now to FIG. 2F, pixel 1002F is simultaneously outcoupled towards the observer by light ray 1002a and light ray 1002b (emanating from image projector arrangements 20a and 20b1, respectively). Both of these light rays illuminate the pupil. On the other hand, pixel 2002F is also simultaneously outcoupled towards the observer by two light rays, light ray 2002a and light ray 2002b (emanating from image projectors 20a and 20b2, respectively). However, in this case, only light ray 2002b illuminates the pupil.

[0040] In contrast, the opposite is true when the pupil is at pupil position 15b, where only ray 1002b illuminates the pupil due to pixel 1002F, while both rays 2002a and 2002b illuminate the pupil due to pixel 2002F.

[0041] Thus, for pupil location 15a, the "selected pixels" within the region of overlap preferably include pixel 1002F, not 2002F. For pupil location 15b, the selected pixels within the region of overlap preferably include pixel 2002F, not 1002F.

[0042] Note that at both pupil positions (15a) and (15b), neither pixel (1000F) nor (2000F) falls within the overlap region because each of these pixels originates from a single image projector arrangement.

[0043] This illustrates that although the image overlap area is fixed by the configuration of the projector assembly, typically only a subset of pixels within the overlap area can be illuminated by the two projectors at a given time, based on the observer's pupil position.

[0044] Pixel Intensity Decrease It can be seen that light rays reaching the pupil from two sources will produce pixels with approximately twice the intensity compared to other pixels produced from a single source. This results in a noticeable non-uniformity in the viewed image. To address this non-uniformity, it is desirable to reduce the intensity of these pixels. However, as already pointed out above, the number of projector locations whose illumination reaches the observer's pupil for different pixels in the region of overlap between partial images will vary depending on the pupil position across the eyebox. Intensity correction according to embodiments of the present invention is therefore preferably performed only on a selected subset of pixels within the region of overlap between partial images, as detailed herein.

[0045] Thus, in some embodiments, the pixel intensity of selected pixels in the region of overlap is reduced (e.g., by a control device as described further below) to improve the perceived uniformity of the image when viewed by an observer.

[0046] 3A and 3D illustrate examples of angular distributions (horizontal axis only) of power intensities of partial images generated by separate image projector arrangements (20b1), (20a), and (20b2) (labeled (a), (b), and (c) respectively) after reduction of pixel intensities in portions of the partial image overlap regions. When the partial images are combined, FIGS. 3B, 3C, and 3E illustrate examples of lateral angular distributions of pixel intensities. It should be noted that FIGS. 3A-3E illustrate theoretical pixel intensity distributions across the field. However, in practice, the intensity distribution of a given projector arrangement is typically non-uniform across the illuminated field and gradually falls off toward the end of the field.

[0047] Figure 3A is optimized for an observer's pupil positioned at pupil position (15a) (see Figures 2D-2H). In that case, the subset of pixels in image region (50) arriving at the central pupil position from two projectors is reduced to half intensity. As a result, after combining the images from all image projector arrangements, the pixel intensity reaching the eye is uniform across the entire image, as shown by the dashed line in Figure 3B.

[0048] However, when the observer's eye is changed to pupil position (15b) (see Figures 2D-2H), the intensity of the combined image is no longer uniform, as shown in Figure 3C, due to the variation in the subset of pixels from the overlap region that reach the eye from the two projectors. The variation in pixels that reach the eye due to different pupil positions was discussed above with reference to Figures 2F-2G, where point (1002F) moved from being seen by two projectors to being seen by only one projector, and point (2002F) moved from being seen by one projector to being seen by two projectors.

[0049] Thus, according to some embodiments, the controller may vary the subset of pixels whose intensity is reduced based on overlap region adjustment inputs, e.g., the expected or known position of the observer's pupil. In some embodiments, the overlap region adjustment inputs may be derived automatically, e.g., by a pupil sensor. In some embodiments, the overlap region adjustment inputs may be provided to the user by manual input. For example, a test image may be displayed to the user with overlapping portions. For example, the user may be asked to view different portions of the image and provide input to reduce the intensity of selected pixels by activating a knob or lever connected to the controller when the image appears uniform. Alternatively, the user may provide feedback to the adjustments made by the controller, e.g., during a calibration process. The controller, receiving such feedback, may vary the subset of pixels whose intensity is reduced until the best approximation for a uniform perceived image is achieved.

[0050] As an example, Figure 3D illustrates the angular distribution of power intensity based on the observer's eye at pupil position 15b after reducing the intensity of pixels corresponding to image region 50'. Note that image region 50' for pixel intensity reduction shown in Figure 3D is slightly different from image region 30 in Figure 3A due to the different pupil position. After combining the separate images, the intensity across the combined image becomes uniform, as shown in Figure 3E.

[0051] It should be noted that when an observer looks in different directions, e.g., due to rotation of the eye about its center of rotation, the position of the pupil changes in different parts of the projected image. Typically, the sensitivity of the human eye to changes in image intensity is much greater in the central field of view, while people are much more tolerant of image intensity changes in their peripheral vision. Therefore, it is typically sufficient to make adjustments that optimize the area of intensity correction for each "seam" (area of overlap) for the pupil position corresponding to the gaze direction facing that "seam." Thus, for example, the aforementioned manual user adjustments may be conveniently performed as part of a software-guided calibration process in which the user is first instructed to view a projected calibration image measuring a first seam, e.g., to the left, and make manual adjustments until the calibration image appears uniform, and then view a projected calibration image measuring a second seam, e.g., to the right, and make manual adjustments until the calibration image appears uniform. These settings can be used continuously to project subsequent images, regardless of the instantaneous pupil position, with the understanding that the seam regions of the field of view will be of high quality, while the user is viewing them in the user's central vision and may be somewhat uneven in the peripheral vision.

[0052] In some embodiments, the pupil sensor can be positioned to dynamically detect eye rotation (e.g., as a function of deviation from a predetermined center of rotation). Based on the detected eye rotation, the controller can determine a subset of pixels to reduce in intensity and make appropriate adjustments, thereby providing optimization of uniformity across the entire field for each instantaneous position of the pupil.

[0053] 4A-4C schematically illustrate cross-sectional views of different configurations of a display according to the present invention.

[0054] In some embodiments, the display can include a separate waveguide for each image projector arrangement. In a particularly preferred embodiment, as shown in FIG. 4A , the display includes three projector arrangements and three corresponding waveguides. A three-projector arrangement configuration accommodates an observer gazing at the center of the image (as opposed to facing the side), advantageously allowing for a central field of view generated only by the central projector arrangement, which is generally non-overlapping and naturally uniform. Parenthetically, both in this section and all other implementations described herein, the fields of view of the different projector arrangements need not be equal. In certain instances, it may be advantageous to provide a projector arrangement with a relatively larger field of view for the central portion of the FOV, while lateral regions of the overall FOV can be served by projector arrangements projecting smaller FOVs.

[0055] FIG. 4B schematically illustrates an alternative embodiment in which a two-projector arrangement is used with corresponding waveguides. Due to the reduced number of components, this configuration is advantageously relatively simpler to manufacture (as well as to operate). Additionally, in this configuration, there is advantageously only one overlap area, which requires pixel intensity adjustment, as opposed to a three-projector arrangement that creates two different overlap areas. The use of different sized FOVs for the two projectors would allow for offsetting the outer seam area of the center section.

[0056] FIG. 4C schematically illustrates an embodiment with non-parallel waveguides 41a, 41b, and 41c. The non-parallel waveguides further expand the image field by extending the waveguides around or orienting them to partially encompass the observer (or the observer's eye). It should be noted that in this embodiment, the edges of the waveguides may be within the observer's field of view and therefore may cause scattering and / or perturbations in the viewed image. These effects can be at least partially suppressed or eliminated by introducing an index-matched medium 43 (such as a conformal plastic) between these ends, thereby forming an optically smooth transition to the edges of the LOE.

[0057] Furthermore, this embodiment can be extended to multiple light guide panels optically replicated in two dimensions to encompass all desired angles around the observer and provide an overall concave display, which can be extended to form viewing domes or the like.

[0058] 5A schematically illustrates an embodiment of a display configured for 2D augmented images. In this embodiment, the display has two projector arrangements (24) and (26) arranged to intercombine partial images into a single 2D LOE (28) for 2D image augmentation. The LOE (28) has two non-parallel sets (30) and (32) of mutually parallel facets or diffractive elements. LOEs configured for 2D augmented images are further described in WO 2019 / 016813 (see, e.g., FIGS. 5A and 6 therein).

[0059] Projector arrangements (24) and (26) project images at two different angles into the LOE (28). Light from both projector arrangements is first reflected by facet (30) (thereby expanding the aperture in one dimension, e.g., vertically) and then reflected outward toward the observer by facet (32) while simultaneously expanding the aperture in another dimension, e.g., horizontally. Each projector arrangement generates a partial image, which is then outcombined toward the observer, who sees the outcombined image. Note that the area of overlap between the partial images can be in a side-by-side horizontal configuration, as shown in Figure 5B, or in a top-to-bottom vertical configuration, as shown in Figure 5C. The angle of horizontal or vertical tilt between projector arrangements (24) and (26) determines the offset between the optical axes of the two projectors. Accordingly, the degree of vertical and horizontal overlap present in the image viewed by the observer. Note that the actual positioning of projector arrangements (24) and (26) is typically not critical, since a two-dimensional aperture arrangement is performed by facets (32) and (30) on the light from both projector arrangements. Note that in the overlap region of the field, the intensity projected by the two projector arrangements must be managed to maintain uniform intensity. In this case, the variation in intensity across the eyebox will be reduced compared to a multiple waveguide configuration.

[0060] 6 schematically illustrates a second embodiment of a display configured for 2D image enhancement. This embodiment uses a four-projector arrangement. Projector (34) incoupling a partial image of the LOE (28) for reflection and aperture placement with facet (30) and then reflection only with facet (32). On the other hand, projector arrangement (36) incoupling a partial image of the LOE (28) for reflection and aperture placement with facet (32) and then reflection only with facet (30).

[0061] Projector arrangement 38 is directed primarily to reflect from facet 32, while projector arrangement 40 is directed primarily to reflect from facet 30. Light from both projector arrangements 38 and 40 experiences several back-and-forth reflections between the perpendicular sets of facets 30 and 32, causing aperture expansion in both the vertical and horizontal dimensions.

[0062] 7 schematically illustrates an example of a functional block diagram of a display (70) in accordance with certain embodiments. The display (70) includes a controller (74) and two or more projector assemblies (5-1)-(5-n).

[0063] Each projector assembly (5) includes at least one image projector arrangement (20) and at least one LOE (28) having a pair of parallel exterior surfaces. The image projector arrangement (20) is configured to generate and project a partial image and is positioned to introduce the partial image into the LOE (28). The LOE (28) is configured to propagate the partial image within the LOE by internal reflection from the pair of parallel exterior surfaces. In some embodiments, each projector assembly includes an outcoupling arrangement (7) associated with the LOE (28) and configured to outcoupling the partial image from the LOE to an observer's eye.

[0064] In some embodiments, the LOEs (28) of each projector assembly are arranged in an overlapping relationship with one another so that each projector assembly projects a respective partial image corresponding to a respective portion of the image displayed to the observer, the respective parts of the image having a partial overlap such that two or more projector assemblies cooperate to display the image to the observer.

[0065] The controllers (74) are each associated with an image projector arrangement of the projector assembly. The controllers (74) include at least one processor (76) associated with a memory (78). The processors (76), in combination with the associated memory (78), are configured to execute one or more functional modules stored in the memory (78) for controlling the display (70), including, for example, reducing pixel intensity of selected pixels projected by the at least one image projector arrangement, the selected pixels being located in areas of partial overlap between portions of the image, thereby improving the perceived uniformity of the image displayed to a viewer.

[0066] In some embodiments, the controller may be configured to vary the pixel intensity of selected pixels within the region of overlap to account for pixel intensities of projector arrangements projected across the field and any variations in the observer's pupil position within the eyebox.

[0067] In some embodiments, the controller may be configured to gradually decrease the intensity of selected pixels projected by one projector arrangement across the region of partial overlap, and gradually increase the intensity of selected pixels projected by the second projector arrangement across the region of partial overlap.

[0068] In some embodiments, the controller (74) may be coupled to a user input device (not shown) configured to provide user input to the controller (74), for example, as described above with respect to Figures 3A and 3D. In some embodiments, the controller may be physically located in the same housing as other components of the display (70) or in a different housing. In some embodiments, different components of the controller may be physically located separately from one another. In some embodiments, the controller is preferably, but not exclusively, implemented in a head-mounted display, and most preferably in a glasses-type display.

[0069] In some embodiments, the display (70) includes a pupil sensor (72) configured to detect a further pupil position of the viewer and update the control unit (74) with data indicative of the current pupil position.

[0070] In some embodiments, the controller (74) is configured to determine a subset of common pixels between the sub-images based on data obtained from the pupil sensor or based on user input, and to implement intensity reduction for that subset of pixels, so that image illumination arrives at the pupil from the two projectors simultaneously. In some embodiments, the controller is further configured to vary the selection of the common pixel subset in response to an overlap region adjustment input, which may come from the pupil sensor or manual user input.

[0071] In some embodiments, the controller is configured to cooperate with the pupil sensor (72) to obtain calibration data and store the obtained calibration data in the memory (78), and can determine appropriate overlap region adjustments for all pupil positions based on the stored calibration data.

[0072] 8 illustrates an example flowchart of a method for displaying images to an observer's eyes according to a particular embodiment when a sensor is used to detect the observer's pupil position, either as a one-time calibration process or as an ongoing real-time adjustment. All steps are performed by processor 76 unless otherwise noted.

[0073] In step 86, the pupil sensor detects the pupil position of the observer's eye.

[0074] Step 88 determines a subset of pixels in the overlap region of the images displayed to the observer at each different pupil position.

[0075] In step (90), the intensities of pixels within the determined subsets are reduced to improve the uniformity of the image displayed to the observer's eye. This intensity reduction is typically performed by modifying the image data sent to the projectors to reduce the pixel intensity values for the associated subsets of pixels sent to both projectors. For example, from the region of recognized overlap, a pixel with RGB values (200, 80, 168) could be sent to both projectors as if the pixel data were a fainter pixel of the same color, such as (100, 40, 84), assuming an ideal linear response of the projectors. In practice, the correction may need to be calibrated depending on the specific hardware characteristics of the projector assemblies. Furthermore, as discussed above, the output intensities of different projector assemblies are typically not uniform across the field, and the intensity correction should preferably take these non-uniformities into account.

[0076] Note that although the intensity reduction profile is illustrated herein as a step function with 50% intensity contributed by each projector in the region of allowed overlap, the intensity subdivision between the two projectors need not be equal for every given pixel, and the smoothness of the resulting image would typically be greatly enhanced by the use of linear tapering or otherwise smoothed transition profiles.

[0077] 9A illustrates an embodiment in which the visible image intensity is continuously passed between two adjacent projectors across the permitted overlap region, preferably starting at over 80% intensity at the beginning of the permitted overlap region, passing through 50:50 somewhere in the middle, and reaching a contribution of less than 20% to the associated pixel intensity for each projector at the outer extremities of the permitted overlapping pixels. This progressive change is preferably monotonic and occurs gradually across the transition region.

[0078] FIG. 9B illustrates the intensity distribution in the combined image after linear tapering correction as described in FIG. 9A. In some embodiments, when intensity correction is performed by linear or stepwise tapering as described above, preferably with a more gradual slope and larger transition range, the corrected image may be adequate within viewing tolerance limits for any pupil position. This avoids the need to detect pupil position and perform dynamic correction, as illustrated in FIG. 9C. In the case of manual user input adjusting the intensity falloff region, the intensity falloff profile may advantageously be temporarily switched to a step function profile during calibration to make intensity non-uniformities more noticeable, and then revert to a progressive change during normal operation. Additionally, when the projector arrangement is of the type with gradual spatial intensity falloff, the progressive intensity falloff described herein can be modified according to the attenuation characteristics of each projector arrangement, as well as the pupil position within the eyebox, to maintain uniform image intensity as seen by the eye.

[0079] It should be understood by those skilled in the art that the displays provided herein may be implemented in both virtual reality and augmented reality applications (i.e., where virtual display elements are combined with a direct view of the real world).

[0080] It will be appreciated that the above description is intended to serve only as an example, and that many other embodiments are possible within the scope of the invention as defined in the appended claims.

Claims

1. 1. A display for providing an output image to an eye of an observer, said display comprising: (a) a first light-directing optical element (LOE) having a pair of mutually parallel planar surfaces that support the propagation of light by internal reflection; (b) a first image projector device that generates a first partial image, the first image projector device being arranged to introduce the first partial image into the first LOE by internal reflection at the pair of planes for propagation within the first LOE; and (c) a first coupling-out device associated with the first LOE and configured to couple out the first partial image so that the first partial image is directed toward the viewer's eye; (d) a second light-directing element (LOE) having a pair of mutually parallel planar surfaces that support the propagation of light by internal reflection; (e) a second image projector device that generates a second partial image, the second image projector device being arranged to introduce the second partial image into the second LOE so that the second partial image propagates within the second LOE by internal reflection at the pair of planes; and (f) a second coupling-out device associated with the second LOE and configured to couple out the second partial image so that the second partial image is directed toward the observer's eye; the plane of the first LOE is non-parallel to the plane of the second LOE, a first portion of the first LOE overlaps a first portion of the second LOE, and the first and second partial images provide overlapping subregions of the output image that contribute to a continuous output image viewed by the observer's eye; A display wherein the first LOE has a first transparent extension that is continuous from an edge of the first portion of the first LOE in an overlapping relationship with a second portion of the second LOE, and the first transparent extension is non-planar.

2. The display of claim 1 , wherein the first transparent extension is index-matched to the first LOE.

3. 2. The display of claim 1, wherein the second LOE has a second transparent extension that is continuous from an edge of the first portion of the second LOE in an overlapping relationship with the second portion of the first LOE, and the second transparent extension is non-planar.

4. (a) a third light-directing element (LOE) having a pair of mutually parallel planar surfaces that support the propagation of light by internal reflection; (b) a third image projector device that generates a third partial image, the third image projector device being arranged to introduce the third partial image into the third LOE so that the third partial image propagates within the third LOE by internal reflection at the pair of planes; and (c) a third coupling-out device associated with the third LOE and configured to couple out the third partial image so that the third partial image is directed toward the observer's eye; a plane of the third LOE is non-parallel to the planes of both the first LOE and the second LOE, a first portion of the third LOE overlaps a third portion of the first LOE, and the first, second, and third partial images provide overlapping subregions of the output image that contribute to a continuous output image viewed by the observer's eye; The display of claim 3 , wherein the second transparent extension bridges between the second LOE and the third LOE.

5. 5. The display of claim 4, wherein the first LOE has a third transparent extension that is continuous from an edge of the third portion of the first LOE to an overlapping relationship with the second portion of the third LOE, and the third transparent extension is non-planar.

6. 2. The display of claim 1, further comprising a controller having at least one processor, the controller being associated with the first and second image projector devices and configured to reduce pixel intensities of selected pixels projected by at least one of the first and second image projector devices, the selected pixels being in areas overlapped by the first and second partial images to increase the perceptual uniformity of the output image.

7. 7. The display of claim 6, wherein the controller is configured to gradually decrease the intensity of the selected pixels projected by the first image projector device across the overlapping region and to gradually increase the intensity of the selected pixels projected by the second image projector device across the overlapping region.

8. 2. The display of claim 1, wherein each of the first and second coupling-out devices is implemented as one or more sets of obliquely angled, parallel internal partially reflective surfaces disposed within the first and second LOEs, respectively.

9. The display of claim 1 , wherein the first and second coupling-out devices are each implemented as a diffractive optical element.

Images