Waveguide combiner system and method

Abstract

A waveguide combiner system comprises a plurality of incoupling regions arranged to incouple light corresponding to a respective one of a plurality of input processed images relating to a light field. The system comprises a plurality of sets of sparse outcouplers, each set being coupled to a respective incoupling region. Each set of sparse outcouplers defines a respective outcoupling region, and the plurality of outcoupling regions at least partially overlap spatially. The system also comprises one or more waveguide combiners, where at least one of the waveguide combiners comprises at least two of the incoupling regions and corresponding sets of outcouplers. Each set of sparse outcouplers is configured to replicate light corresponding to a respective input processed image to generate a respective set of replicated processed images, each set of replicated processed images combining at a viewing area to form a component of a light field image.

Description

[0001] WAVEGUIDE COMBINER SYSTEM AND METHOD

[0002] Technical Field

[0003] The present invention relates to waveguide combiner systems and methods for displaying light field images using waveguide combiner systems.

[0004] Background

[0005] Focus rivalry is a visual phenomenon that occurs when a viewer’s eyes attempt to focus on two different distances simultaneously. This can lead to visual discomfort and cognitive dissonance in the viewer. Focus rivalry is a well-known problem in Augmented Reality (AR) display systems where digital content is superimposed on top of the real world. For example, the information presented by an AR display may be at a different focal depth than the real world object on which it is superimposed or adjacent.

[0006] Displays that aim to mitigate the effects of focus rivalry, such as light field displays, may not be suitable for smaller form factors, such as a pair of AR-ready glasses.

[0007] It would be useful to provide an improved display which can reduce focus rivalry and potentially suitable for a small form factor, such as a wearable device.

[0008] Summary

[0009] Examples discussed herein propose a new waveguide combiner system for use in displaying light field images. Accordingly, in a first aspect there is provided a waveguide combiner system for a light field display. The waveguide combiner system comprises a plurality of incoupling regions, each incoupling region arranged to incouple light corresponding to a respective one of a plurality of input processed images relating to a light field; a plurality of sets of outcouplers, each of the sets of outcouplers coupled to a respective incoupling region and each set of outcouplers defining a respective outcoupling region, the plurality of outcoupling regions at least partially overlapping spatially; and one or more waveguide combiners, at least one of the one or more waveguide combiners comprising at least two of the incoupling regions and corresponding sets of outcouplers. Each set of outcouplers is configured to replicate light corresponding to a respective input processed image to generate a respective set of replicated processed images, each set of replicated processed images combining at a viewing area to form a component of a light field image, such that the components of the light field image combine at a viewing area to form the light field image. An area covered by a set of outcouplers is less than half an area covered by the respective outcoupling region.

[0010] A “light field display” is a display system capable of projecting a light field depicting objects at more than one depth, on which a viewer’s eye can focus naturally. In some examples, the light field is a 4D light field that specifies the intensity of light as a function of x, y, 0, (p, wherein x, y are spatial coordinates at the viewing area, and 9, (p are angular coordinates of light at that viewing area. In other examples, the light field display may be operated in a 2- dimensional mode at a single depth. It will be appreciated that the same light field display may be operable in either 4D and 2D modes.

[0011] The plurality of input processed images relating to the light field image may be generated by an optical system that may be external to, or incorporated in the same system as, the waveguide combiner system as will be discussed in detail below.

[0012] The outcoupling region may be an area or volume that encloses a respective set of outcouplers. Thus, the outcoupling region is a physical property of a waveguide combiner as opposed to a visual parameter such as field of view. In some examples, each outcoupling region may be defined as a convex hull of the respective set of outcouplers. The outcouplers in each set of outcouplers are sparse, or relatively well-spaced, at the outcoupling region. The sparseness of the outcouplers within each set of outcouplers may be determined, or selected, such that for a single image point at a viewing area, light corresponding to a relatively low number of replicated processed images reach a viewer’s pupil. The outcouplers themselves are optical structures that extract light from the waveguide, and may operate by diffraction and / or reflection. The result of light extraction by an outcoupler is a beam of “outcoupled” light. The outcouplers may comprise one or more of grating regions, holographic optical elements (HOEs), micromirrors and surface relief patterns. An outcoupler may be formed from individual nano-structures (for example, in the case of surface relief gratings), but those individual nano-structures are not themselves outcouplers. In other words, light incident on an outcoupler is extracted as an outcoupled beam, whereas light incident on, for example, a single diffraction grating tooth (an example of a nano-structure) will have a minimal and highly localised effect. A difference between the outcouplers and the nano-scale elements that form them is the different scales between them. For example, the spacing between nearest outcouplers in the present disclosure may be of the order of 10'3m, such as between around 3 mm and around 7mm. This is orders of magnitude higher than from diffraction grating periods (the spacing between nano structures making up the outcoupler), which are of the order of 10’7m.

[0013] Considering a beam of a single input processed image relating to a given virtual image point, and the set of replicated beams of the set of replicated processed images relating to that input processed image: in an ideal scenario, at a selected viewing area, at most one of such replicated beams relating to a single input processed image reaches the viewing area. This is because the resulting light field image at the viewing area will then have at most, one copy of each virtual image point from each input processed image relating to the light field image, so that there will not be spurious copies of the virtual image points of the input processed images that manifest as artefacts in the light field image. However, acceptable light field imagery may be achieved where, for a small fraction of the virtual image points, one, two or possibly even more spurious copies of a virtual image point reach the viewing area, and so the sparseness of the outcouplers in each set of outcouplers may be defined accordingly.

[0014] Furthermore, a number of beams of replicated processed images that reach a viewer’s pupil for a single image point depends on the particular viewer and the viewing conditions (which determine how dilated a viewer’s pupil is). For example, a maximum, minimum and average pupil width of a viewer will affect how many replicated beams, each of a different replicated processed image from a set of replicated processed images, are receivable at that viewer’s pupil in different lighting conditions. The sparseness may be determined based on an average human pupil width for wide applicability to human viewers. In other examples, a viewer may undergo a fitting procedure that determines an optimal sparseness for that viewer and so the waveguide combiner system may be custom-produced for that viewer.

[0015] The sparseness may be defined as: an area covered by a set of outcouplers at a respective outcoupling region is less than half an area covered by the respective outcoupling region. Other examples may occupy a smaller area of the outcoupling region with the set of outcouplers, such as less than 25%, less than 5%, less than 2.5% or less than 1%. In some examples, an area covered by any set of outcouplers of the sets of outcouplers, or all the sets of outcouplers, is less than half, less than 25%, less than 5%, less than 2.5% or less than 1% of an area covered by their respective outcoupling region. This is in contrast to waveguide combiner systems that aim to improve image uniformity by replicating an input light field on a dense grid at a single outcoupling region. Furthermore, the set of outcouplers may be spread over the outcoupling region substantially evenly, for example so that a spacing between outcouplers in the set of outcouplers is substantially constant. In some examples, the outcouplers from any set may not overlap any outcouplers from any other set. In other words, each outcoupler may occupy an area of the waveguide combiner system that does not overlap with areas occupied by any other outcoupler.

[0016] The viewing area may be a predetermined position at which a viewer’s pupil can be expected to be located when the waveguide combiner system is in use. The viewing area may therefore be predetermined in software, independently of any actual pupil position. In some examples, the viewing area may be determined based on a detected position of a viewer’s pupil. In further examples, the viewing area may be determined based on a calibration procedure, or routine of calibration procedures. In some examples, the viewing area is positioned at a certain distance from the outcoupling regions of the waveguide combiner system. For example, where the waveguide combiner system is implemented in a pair of spectacles, the viewing area may be at an expected position of a viewer’s pupils when wearing the spectacles. In some examples, the viewing area may be positioned at, or very near to, the outcoupling regions. For example, where the waveguide combiner system is implemented in a contact lens, the viewing area may be aligned with a surface of the waveguide combiner system that contacts the viewer’s pupil.

[0017] The waveguide combiner system may comprise a plurality of exit pupil expander regions arranged to: receive light from a respective one of the incoupling regions, replicate the received light, and transmit the replicated received light to a respective one of the outcoupling regions. In some cases, exit pupil expander regions may not be present, for example where the display system already has a large etendue in one direction so that exit-pupil expansion is not required.

[0018] In some examples, centres of nearest outcouplers in a set of outcouplers are separated by at least 3mm at the outcoupling region, for example separated by 4mm or separated by 7mm. Larger separations may allow for eye motion without requiring pupil tracking, but at the cost of reducing the density of the outcouplers.

[0019] Additionally, to display a component of a light field image with greater depth of field, each replicated processed image may be coupled out of a small exit pupil, meaning that the outcouplers have small widths. This results in outcoupled beams having small widths at the viewing area. In some examples, a sufficient depth of field is achieved when the width of each outcoupler is less than 1 mm, or 0.5mm or less. Combined with the sparseness of the outcouplers, this would normally result in significant gaps in the field of view, as well as image artefacts due to the very small pupil size. However, as described above, the waveguide combiner system comprises a plurality of outcoupling regions, each coupled to a respective incoupling region. That is, the “gaps” in the field of view can be filled by additional outcouplers from different sets of outcouplers. Thus, while the outcouplers from a set may be sparsely arranged at the outcoupling region, the union of all outcouplers from all sets of outcouplers forms a denser arrangement of outcouplers.

[0020] The waveguide combiner system utilises k-space multiplexing to avoid cross talk between the plurality of views at the outcoupling regions. The coupling between each incoupling region and corresponding outcoupling region may be such that light corresponding to each incoupling region received at an outcoupler occupies a region of k-space different from others of the incoupling regions. In some examples, each of the sets of outcouplers comprise gratings having an associated k-vector, wherein a magnitude of a difference between any two k-vectors is greater than a threshold amount. The threshold amount may be determined based on an acceptable amount of light that is allowed to be outcoupled at a non-corresponding outcoupling region, and can depend on the relative angle and magnitude of the k-vectors. Where the magnitudes of the k-vectors are the same, this condition may be stated as the dot product between any two k-vectors is less than the magnitudes of those k-vectors multiplied together (i.e. there is some angle between them). Where two k-vectors are parallel, their magnitudes may be different such that spurious replicated beams do not reach the viewing area. In some examples, the threshold amount may be the amplitude of a k-vector describing an edge of a field of view in free-space. This is equivalent to the light corresponding to each incoupling region received at an outcoupler occupying a region of k-space different from others of the incoupling regions. Techniques for describing the action of other types of coupling features in k-space are known in the art (for example, coupling features in a reflective waveguide combiner).

[0021] The k-space multiplexing, in this context, defines how the outcoupling regions affect light waves incident on the respective regions. This means that each input processed image of the plurality of input processed images can be coupled into the waveguide combiner system at a respective different angle, meaning that in some examples, optical engines generating respective input processed images can be positioned around the edges of the waveguide combiner system. In some examples they may be spaced apart from each other and / or distributed around a perimeter or periphery of the waveguide combiner system.

[0022] The respective angles of the k-vectors may be determined to outcouple light received from a specific range of angles towards the viewing area. Light received at a given outcoupling region from a non-corresponding incoupling region may then be received at a different angle such that it does not interact with the given corresponding set of outcouplers, or if it does interact, is outcoupled in a direction away from the viewing area. This allows for multiplexing of light received from multiple spatially separated sources, for example, allowing for an increased density of outcoupled light at the viewing area while still retaining the sparseness of each set of outcouplers.

[0023] Each of the k-vectors corresponding to the plurality of sets of outcouplers may form an edge of a triangular tiling. Such an arrangement may ensure that any spurious outcouplings, whilst having inconsistent focus due to being outcoupled out of the incorrect outcoupling region, still have the correct output k-vectors. This means that the spurious output image is as closely matched to the desired image as possible, rather than having multiple significantly shifted spurious image copies. Other tilings may also be used, for example each of the k-vectors corresponding to the plurality of sets of outcouplers may form an edge of a square tiling.

[0024] A waveguide combiner system having these properties allows for implementation in a thin form factor. The waveguide combiner system may have a thickness such that it is indistinguishable from a typical lens of a pair of glasses. In some examples, the thickness of each waveguide combiner may be less than 2mm, such as 1mm or 0.5mm. A light field display comprising such a waveguide combiner system and appropriately small image sources, or optical engines, therefore allows for a compact 3D-capable display form factor. Suitable smallsized image sources include micro LED.

[0025] The waveguide combiner system may comprise a single waveguide combiner, also referred to as only one waveguide combiner, in which case that waveguide combiner will comprise the plurality of incoupling and outcoupling regions. As used herein, a waveguide combiner refers to a single layer, waveguide, or plate, whereas the waveguide combiner system refers to the system comprising at least one such waveguide combiner. In some examples, the waveguide combiner system may comprise a plurality of waveguide combiners. The first waveguide combiner may comprise a first subset of the plurality of incoupling regions and corresponding outcoupling regions, and a second waveguide combiner may comprise a second subset of the plurality of incoupling regions and corresponding outcoupling regions, different from the first subset. While the outcouplers corresponding to the first and second subsets of outcoupling regions may not overlap, incoupling regions from the first and second subsets of incoupling regions (that is to say, incoupling regions of different waveguide combiners, when present) may overlap. In an example, the second waveguide combiner may comprise a single, also referred to as only one, incoupling region and corresponding set of outcouplers, different from the at least two of the incoupling regions and corresponding sets of outcouplers of the first waveguide combiner. The plurality of waveguide combiners may be stacked on top of one another. The thickness with which each waveguide combiner can be manufactured to give the required properties is low, meaning that even when multiple waveguide combiners are stacked on top of one another, the thickness of the waveguide combiner system is still low (e.g. low enough for use in compact form factors such as a pair of glasses). Utilising a plurality of waveguide combiners allows the viewing area to be more densely populated with outcouplers, further filling the gaps between outcouplers in a single set.

[0026] The waveguide combiner system may be configured so that the viewing area (where each set of replicated processed images combine to form a component of a light field image, and the components in turn combine to form the light field image) is positioned at a rotational centre of a viewer’s eye. This potentially avoids the need to have a sufficiently large viewing area to handle gaze-dependent eye pupil location, or alternately the need to track pupil location with changing gaze, and to move the viewing area accordingly. The positioning of the viewing area at a rotational centre of a viewer’s eye may also be applied more generally to displays with pupil or eye tracking, including light field and CGH displays, and is not limited to displays with the waveguide combiners described herein.

[0027] According to a second aspect, there is provided a light field display system comprising an optical system configured to generate a plurality of input processed images relating to a light field image, and the waveguide combiner system according to the first aspect. The waveguide combiner system is arranged in an optical path between the optical system and a viewing area.

[0028] It is possible to calculate a set of input processed images for display by the optical system in order to generate a desired target light field image at the viewing area. For example, for each of the input processed images, a respective set of construction rays may be determined, where each construction ray has an end at one of a set of virtual image points to be displayed and an end at an intersection with the viewing area, and each construction ray additionally passes through the centre of an outcoupler of a set of outcouplers coupled to the incoupling region that will receive the input processed image. The optical system may then be configured to display the input processed images, where each of the input processed images comprises a set of beams having angles relating to their respective set of construction rays.

[0029] This may be achieved in any desired way. For example, the construction rays may be determined for each possible or desired virtual image point. These may be summed and used to calculate the input processed images. In another example, a set of cones representing the allowable volumes in which the virtual image points can be positioned is determined. Each cone may then be treated as an independent set of virtual images which can be summed and used to calculate the input processed images. Other examples are possible.

[0030] The optical system may comprise a plurality of optical engines, or image sources, wherein each of the optical engines is configured to generate light corresponding to at least one input processed image of the plurality of input processed images. An “optical engine” or “image source” as used herein comprises display system component(s) which produce an input processed image for display. The optical engine may comprise one or more illumination sources and may use any suitable technology, including but not limited to currently known technologies and those that may become available. The optical engine may further comprise one or more optical components, for examples lenses or mirrors. In some examples, an optical engine may comprise a self-emissive system, such as a microLED or OLED image source. In some examples, an optical engine may also, or alternatively, comprise a Liquid Crystal on Silicon (LCoS) engine with lasers or LEDs. It is understood that an “optical engine” may refer to spatially separate monochrome engines, for example Red, Green and Blue engines, which together behave as a single optical engine. The separate monochrome engines may each be incoupled by different spatially separated coupling features of an incoupling region. In some examples, the plurality of optical engines comprises at least three optical engines, at least four optical engines, at least five optical engines, at least eight optical engines or at least twelve optical engines. Specific examples may comprise three, four, five, eight or twelve optical engines. Determining the plurality of input processed images in this example may involve determining the light to be displayed by each optical engines in order to produce the target light field image at the viewing area.

[0031] The optical engines may comprise a display having some resolution, and the overall apparent spatial and tonal resolution of the light field display system may be higher than the individual spatial and tonal resolution of each optical engine by spatially offsetting the pixels of each optical engine by a pixel amount relative to the other optical engines. This is known as pixel shifting. In addition, or alternatively, to pixel shifting, display rotations may be employed, wherein each optical engine is rotated by a different amount relative to the other optical engines to achieve similar resolution enhancement effects. The offsets between optical engines need not be controlled or regular offsets, and may instead comprise random offsets relative to the other optical engines. Rather than, or in addition to, providing higher resolution in terms of pixels per degree, such spatial offsetting and / or rotation of the optical engines may provide improved apparent rendition of thin lines, reduced pixelations, and reduced “screen door” artefacts. Additionally, any inherent pixel misalignment can be mitigated by calibrating the display in software.

[0032] In some examples, the light field display system may operate in a super-resolution mode where the enhanced resolution effects are present for images at a privileged depth. The privileged depth may be set at infinity, or may be defined at a particular depth such as 1 metre, or other predetermined distance, depending on the intended application of the display system.

[0033] The optical system may be configured to generate light at a plurality of wavelengths. In some examples, the waveguide combiner system comprises a single waveguide combiner, meaning that the light at a plurality of wavelengths is coupled into and out of that waveguide combiner. As discussed above, the waveguide combiner system may comprise a plurality of waveguide combiners. In some examples, each waveguide may be configured to in-couple light at one or more wavelengths of the plurality of wavelengths. In some examples, there may be a waveguide combiner per wavelength of light, or per range of wavelengths, or per colour channel. For instance, in a waveguide combiner system comprising three waveguide combiners, a first waveguide combiner may be configured to in-couple light at a first wavelength, or range of wavelengths (e.g. red, around 620 to 750 nm), a second waveguide combiner may be configured to in-couple light at a second wavelength, or range of wavelengths (e.g. green, around 510 to 570 nm), and a third waveguide combiner may be configured to incouple light at a third wavelength, or range of wavelengths (e.g. blue, around 450 to 495 nm).

[0034] In other examples, a first waveguide combiner may be configured to incouple light at two or more wavelengths / ranges of wavelengths, while a second waveguide combiner may be configured to incouple light at one or more wavelengths / ranges of wavelengths. Other examples are possible. For instance, where the optical system is configured to generate light corresponding to red, green and blue colour channels, and the waveguide combiner system comprises two waveguide combiners, a first of the waveguide combiners may be configured to incouple light corresponding to red and green, while the second may be configured to incouple light corresponding to blue. In another example, the first waveguide combiner may be configured to incouple light corresponding to blue and green and the second waveguide combiner may be configured to incouple light corresponding to red. In another example, the first waveguide combiner may be configured to incouple light corresponding to red and green, and the second waveguide combiner may be configured to incouple light corresponding to blue and green, i.e. light corresponding to green is incoupled to both the first and second waveguide combiners. Other combinations and permutations are possible.

[0035] Where input light of substantially the same wavelength is coupled to two or more waveguide combiners, selective coupling into a desired waveguide combiner can be achieved by varying polarisation of the input light such that light at a certain polarisation is only coupled into a corresponding incoupling region. Spatial multiplexing of the input light may also be performed to selectively couple light of a given wavelength into a corresponding one of the waveguide combiners.

[0036] In some examples, the optical system may utilise time multiplexing of an optical engine to generate two or more of the plurality of input processed images. For instance, an optical engine of the optical system may be configured to, at a first time, generate light having a first characteristic and corresponding to a first input processed image of the plurality of input processed images, wherein the first characteristic causes the light corresponding to the first input processed image to be coupled to a first incoupling region. Then, at a second time, the optical engine may be configured to generate light having a second characteristic and corresponding to a second input processed image of the plurality of input processed images, wherein the second characteristic causes the light corresponding to the second input processed image to be coupled to a second incoupling region.

[0037] The first and second characteristic may be respective polarisations, wavelengths, ray angles (e.g. spatial multiplexing) etc. For example, the first incoupling region may have a property that allows light with the first characteristic to be coupled into the waveguide combiner system, but not allow light having the second characteristic to be coupled in.

[0038] The light field display may comprise a controller configured to determine the plurality of input processed images based on a determined viewing area. The determined viewing area may be based on a determined pupil location. The determined viewing area may be predetermined, for example as part of a calibration procedure. The controller may also be configured to cause the optical system to generate the light corresponding to the plurality of input processed images. That is, the controller may define a viewing area, and may then update the content displayed by the optical system such that the light field image is displayed at the viewing area. The viewing area may be a predetermined area in which a viewer’s pupil is to be located to view the light field image. For example, the viewing area may be a region in which a viewer’s pupil can expected to be located when the light field display is in use. In some examples, the viewing area is based on a determined pupil position. In some examples, the determined pupil position may be predetermined, such as during an initial calibration, or detected in real time.

[0039] To determine / detect the pupil position, the light field display may comprise a pupil tracking system configured to determine the determined pupil position. The pupil tracking system may be further configured to determine a width of a pupil. The controller may be further configured to determine the plurality of input processed images such that each input processed image is the same input processed image when the determined width of the pupil is greater than a threshold width. In other words, the controller may cause the light field display system to operate in a 2D mode when the pupil is wider than the threshold (e.g. dilated). The threshold may be set at a width at which a maximally acceptable number of replicated beams of a set corresponding to a single virtual image point reach the viewer’s pupil, as discussed above. Example maximally acceptable numbers of replicated beams may be as small as one, but two, three or more for a small fraction of the virtual image points may also give acceptable image quality. Example threshold widths include 5mm, 6mm or 7mm. A pupil wider than the threshold will potentially receive more than a desirable number of outcouplers from a set of outcouplers, resulting in artefacts in the image. In such cases, it may be advantageous for image quality to fallback to operating in 2D mode. The light field display system may also be operated in 2D mode where reduced computational requirements are desired.

[0040] In the discussion above, each input processed image being the same input processed image refers to the same image content being displayed at the same focal depth across the plurality of input processed images. However, the input processed images may still differ from one another in at least one aspect. For example, the input processed images may incorporate software corrections due to pixel-shifting alignment differences, as well as super-resolution- based pixel-shifting as described above. The input processed images may incorporate software correction(s) for colour uniformity, which may be applied individually to each optical engine such that each input processed image has its own colour gradient correction based on known waveguide geometry, or may comprise an overall colour correction applied across the plurality of optical engines. The input processed images may incorporate overlap correction(s), where the content of each input processed image is adjusted based on a calculated amount of that input processed image that will be visible to the viewer’s eye, taking into account the spatial overlap between different sets of replicated processed images at the viewing area.

[0041] The light field display may comprise a push-pull lens arranged to reduce a difference in focal power between the components of a light field image and a virtual object represented in the light field image. Alternatively, the light field display may comprise a single lens rather than a push-pull lens arrangement, and utilise the controllable focus properties of the light field display to correct the virtual display content. For example, the light field display may comprise a lens positioned in front of the waveguide combiner system that affects the focal power of real world objects viewed through the display but does not affect the virtual objects displayed by the light field display (for example, a vision-correcting lens). The focus of the virtual objects may then be controlled by the light field display system to match the modified focus of the real world. In another example, the light field display may comprise a lens positioned behind the waveguide combiner system that affects the focal power of both real world objects and virtual objects, with the focus of the virtual objects being controllably adjusted by the light field display system to compensate for the lens effect. Reducing to a single lens rather than a push- pull lens arrangement may reduce production costs.

[0042] According to a third aspect, there is provided a head-mounted display comprising the light field display system according to the second aspect.

[0043] According to a fourth aspect, there is provided a pair of glasses. The pair of glasses comprise a first light field display system according to the second aspect and corresponding to a viewer’s left eye, and a second light field display system according to the second aspect and corresponding to a viewer’s right eye.

[0044] According to a fifth aspect, there is provided a display system comprising: a waveguide combiner system having a perimeter; and a plurality of optical engines positioned along the perimeter. The waveguide combiner system comprises: a plurality of incoupling regions, each incoupling region arranged to receive light from a respective one of the optical engines; a plurality of outcoupling regions, each outcoupling region coupled to a respective incoupling region and configured to output light received from the respective incoupling region towards a viewing area; and at least one waveguide combiner comprising at least two of the incoupling regions and corresponding outcoupling regions. In some examples, the optical engines may be spaced apart from each other and / or distributed around the perimeter. The waveguide combiner system of this aspect may also include any of the features discussed above for the first aspect.

[0045] According to a sixth aspect, there is provided a pair of glasses comprising: a frame delimiting at least two eye apertures; a first at least partially transparent element within a first of the two eye apertures; a second at least partially transparent element within a second of the two eye apertures; a first plurality of optical engines embedded in the frame around a first of the at least two eye apertures; and a second plurality of optical engines embedded in the frame around a second of the at least two eye apertures. At least two optical engines of each of the first and second pluralities of optical engines are at least 25mm apart, and the first and second at least partially transparent elements comprise respective waveguides arranged in optical paths between the respective plurality of optical engines and a respective viewing area . One or more of the waveguides may be the waveguide combiner system according to the first aspect.

[0046] According to a seventh aspect, there is provided a method of displaying a light field image. The method comprises determining a viewing area, and determining a first and second set of construction rays, each construction ray of the first and second set of construction rays having a first end at one of a set of virtual image points to be displayed and a second end at an intersection with the viewing area. Each construction ray of the first set of construction rays passes through a centre of an outcoupler of a first set of outcouplers of the waveguide combiner system according to the first aspect, and each construction ray of the second set of construction rays passes through a centre of an outcoupler of a second set of outcouplers of the waveguide combiner system. The method further comprises displaying, by an optical system, first and second input processed images, wherein: the displayed first input processed image comprises a first set of beams having angles relating to the first set of construction rays, and the displayed second input processed image comprises a second set of beams having angles relating to the second set of construction ray. The method further comprises replicating, by the waveguide combiner system arranged in an optical path between the optical system and the viewing area, the first and second input processed images to produce first and second sets of replicated processed images. Light from the first set of replicated processed images combines at the viewing area to form a first component of a light field image, and light from the second set of replicated processed images combines at the viewing area to form a second component of a light field image The first and second components of the light field image combine at the viewing area to form the light field image.

[0047] The light corresponding to the first and second input processed images may be generated by an optical system as in the light field display according to the second aspect.

[0048] The viewing area may be predetermined, for example during a calibration procedure or may be determined based on a detected position of a viewer’s pupil received from a pupil tracking system, such as the pupil tracking system described with regards to the second aspect above.

[0049] In some examples, the method comprises determining a width of the viewer’s pupil is greater than a threshold width, and when the width is greater than the threshold, the same input processed image is used for both the first and second input processed images.

[0050] According to an eighth aspect, there is provided a method of displaying a target light field using a pupil-replicating optical system. The method comprises determining a target light field to be displayed at a rotational centre of a viewer’s eye. This is different from other display systems in that the target light field is typically determined to be displayed at the pupil location. The method further comprises determining an input light field to be displayed at an input location such that the target light field is produced at the rotational centre of the viewer’s eye, and displaying the input light field at the input location. The input light field may be determined in any suitable manner. For example, the input light field may be determined based on the method according to the seventh aspect. In some examples, the input light field may be determined using a transfer function method, wherein a transfer function describing the propagation of light through the pupil replicating optical system may be calculated, or determined, based on the techniques such as those described in WO2023 / 057543, incorporated herein by reference for all purposes.

[0051] The target light field may, for example, be a holographic or a 4D light field. Determining the target light field to be displayed at the rotational centre of the eye potentially avoids the need to have a sufficiently large viewing area to handle gaze-dependent eye pupil location, or the need to track pupil location with changing gaze, and to move the viewing area accordingly. To understand this, consider that the target light field is determined to be positioned at a location referred to as a viewing area, which is located at a rotational centre of the viewer’s eye. The viewing area may be the eyebox wherein incident light appears to come from a desired virtual object point. The nature of the pupil replicating optical system is to generate multiple replications of the input light field. Where sets of replications combine, an eyebox is formed where the target light field can be viewed. When the viewing area is formed at the rotational centre of the viewer’s eye, some spurious copies of an image point (from different replications) will be received by the pupil because the valid eyebox is positioned behind the pupil. However, spurious copies of the specific image point being looked at are not seen, and only spurious copies of an image point not being looked at are seen. This remains true as the eye rotates about its centre, meaning acceptable image quality is achieved without needing to have a sufficiently large viewing area to support gaze-dependent pupil location. It will be understood that this eighth aspect is not limited to the waveguide combiner of the first aspect as the pupil replicating optical system, and can be applied more generally to displays which use pupil and / or gaze tracking.

[0052] The rotational centre of the viewer’s eye may be determined prior to determining the target light field. This may be achieved as part of an initial calibration procedure that locates the centre of the viewer’s eye, and which is fixed until a later calibration occurs. This could also be achieved using pupil tracking, such as by extrapolating from observed pupil movements the position of the rotational centre. Because the target light field is located at the rotational centre of the viewer’s eye, data generated by the pupil tracking can be obtained at a lower rate, meaning less computational resource utilisation is needed to produce acceptable image quality.

[0053] The target light field may comprise a 4D light field comprising a plurality of replicated processed images corresponding to a light field image. Displaying the input light field at the input location may comprise, displaying, via respective light sources, light corresponding to a plurality of input processed images. Further features and advantages of the invention will become apparent from the following description of preferred embodiments of the invention, given by way of example only, which is made with reference to the accompanying drawings.

[0054] Brief Description of the Drawings

[0055] Figure 1 an example arrangement of output beams for a given input beam using a waveguide combiner according to an example;

[0056] Figure 2A shows an example waveguide architecture that achieves sparse out-coupling;

[0057] Figure 2B shows a further example waveguide architecture that achieves sparse out- coupling;

[0058] Figure 3 shows an example arrangement of an input processed image with a given large field of view and corresponding ranges of angles of replicated processed images using a waveguide combiner according to an example;

[0059] Figure 4 shows an example arrangement of output beams for two input beams using a waveguide combiner according to an example;

[0060] Figure 5 shows a light field display system according to an example;

[0061] Figure 6 illustrates an example display arrangement showing a first input processed image;

[0062] Figure 7 illustrates an example display arrangement showing a second input processed image;

[0063] Figure 8 illustrates an example display arrangement showing the combination of the first and second input processed images;

[0064] Figure 9 shows a waveguide k-space diagram illustrating how light interacts with a waveguide combiner according to an example;

[0065] Figure 10 shows a waveguide k-space diagram illustrating how light corresponding to five input processed images interacts with a waveguide combiner according to an example;

[0066] Figure 11 A shows a k-space diagram illustrating an example spurious out-coupling;

[0067] Figure 11B shows a k-space diagram illustrating a further example spurious out- coupling;

[0068] Figure 12 shows a waveguide combiner arrangement according to an example;

[0069] Figure 12A shows a close up of the waveguide combiner shown in Figure 12;

[0070] Figure 13 A shows a pair of AR glasses according to an example;

[0071] Figure 13B shows a pair of AR glasses according to a further example; Figure 14 shows a diagrammatic representation of an example light field display system;

[0072] Figure 15 depicts a flow chart of an example method for displaying a light field image;

[0073] Figures 16A and 16B show an optical output according to an example; and

[0074] Figure 17 depicts a flow chart of an example method for displaying a target light field.

[0075] Detailed Description

[0076] Optical waveguide combiners, also referred to as waveguide combiners or image replicating combiners, are physical structures that guide electromagnetic waves in the optical spectrum redirecting and possibly expanding a size of a viewable area of an image for a viewer. They comprise an input surface, also known as an incoupling region, in-coupler or an entrance pupil, to receive light rays corresponding to at least a portion of an input image. The notion of an entrance pupil corresponds to the limiting aperture in an input of the waveguide combiner. The incoupling region is an area, or volume, comprising one or more coupling features that couple light waves, propagating externally, to the inside of the waveguide combiner. Each coupling feature may be referred to as an incoupler in this context. A coupling feature may be, for example, a mirror, a prism, a diffraction grating or a hologram. The incoupling region may comprise two or more spatially separated coupling features, for example two or more coupling features incoupling light corresponding to different ranges of wavelengths. It is understood that the two or more spatially separated coupling features may together be referred to as an incoupling region when they incouple light corresponding to the same input processed image. A waveguide combiner further comprises an output surface, also called an outcoupling region to output light corresponding to the (at least a portion of the) input image. The outcoupling region is an area, or volume, comprising one or more further coupling features, which may use the same technology as the input surface. The coupling features may be referred to as outcouplers in this context.

[0077] Waveguide combiners may be manufactured from materials with high-refractive indices that support total internal reflection over a wide range of internal incidence angles. Lanthanum dense flint glass, for example N-LASF46 manufactured by Schott™, has a critical angle of ()c= 31° at wavelength A = 530nm. A waveguide combiner will propagate waves by total internal reflection at all internal angles above the critical angle.

[0078] In an example, a waveguide combiner takes the form of a substantially planar sheet. The sheet may be constructed from a transparent material, such as glass. In this case, one arrangement for the incoupling and outcoupling regions is to position them on the same side of the sheet, such that light enters and exits at the same side of the sheet. In another arrangement, the incoupling and outcoupling regions may be positioned on opposite sides of the sheet. The particular arrangement may be selected based on a function of the waveguide combiner. In other examples, a waveguide combiner takes the form of a non-planar sheet. Such waveguide combiners may find use as a lens in a pair of spectacles or glasses, for example. In non-planar waveguide combiners, the incoupling and outcoupling regions of the waveguide combiner may be on the same or opposite sides of the sheet depending on the function of the waveguide combiner, as described above. The waveguide combiner may comprise a curved waveguide. Because the output of each outcoupler can be corrected in software, this reduces the requirement for a planar or rigid waveguide. This allows for design possibilities such as the use of plastic waveguides that may be more easily formed into curved or flexible configurations, or even waveguides where the presence of surface or bulk irregularities can be mitigated by software corrections.

[0079] Waveguide combiners may replicate the input, so that internal rays or beams repeatedly split into reflected and transmitted rays or beams. Transmission occurs at select positions at the outcoupling region corresponding to the positions of the outcouplers. The copies of rays or beams that are transmitted may be referred to as replications. The outcoupling region may comprise, for example, an array of mirrors, an array of prisms, an array of diffraction gratings or an array of holograms. Further possible coupling features include embedded mirrors, microprisms, a surface relief slanted grating, a surface relief blazed grating, a surface relief binary grating, a multilevel surface relief grating, a thin volume hologram, a thin photopolymer hologram, a polarisation volume grating, a Holographic Polymer Dispersed Liquid Crystal (H- PDLC) volume holographic coupler, a thick photopolymer hologram, a resonant waveguide grating, a metasurface coupler and embedded half-tone mirrors.

[0080] The present disclosure is at least partially based on the observation that a light field display may incorporate a waveguide combiner comprising what will be referred to as “sparse outcoupling regions”. A sparse outcoupling region may be defined by the density of corresponding outcouplers, and therefore replications it generates for a given in-coupled field, which may be referred to as an input processed image in this context. In particular, the density of outcouplers may be such that for a given input processed image, light corresponding to a virtual image point is typically only seen through, at most, a single exit pupil, although some examples may have more than one copy of a virtual image point which is visible and maintain acceptable image quality. The ideal case is that at most a single copy of a virtual image point for a given input processed image is incident on a viewer’s pupil. However, this may be difficult to achieve due to the variation in pupil width in different lighting conditions as well as pupil width variation between different viewers in a population.

[0081] Components of a light field image are generally at a same fixed depth, which will in general be different to the depth of a virtual image point, causing a mismatch. To reduce the effect of this mismatch, it is desirable for each component of a light field image to have a greater depth of field, so that each component is approximately always in focus (i.e. is approximately in focus at any depth). This achieves a higher resolution light field image at a viewing area where the components combine. To achieve this, each replicated processed image may be coupled out of a small exit pupil More specifically, the displayed light field image may be coupled out of an array of multiple small exit pupils, provided that, for a given input processed image, a viewer will typically receive only one copy of a virtual image point, or a suitably small number, such as two or three copies of a virtual image point, for a small fraction of the total number of virtual image points that comprise the virtual image. This principle is illustrated in Figure 1 which shows an example waveguide combiner 100 comprising an incoupling region 102 and sparse outcoupling region 104.

[0082] A single input beam 106, corresponding to a beam of an input processed image, is incident on the incoupling region 102 and is coupled into the waveguide combiner 100. Inside the waveguide combiner 100, the in-coupled beam 106 undergoes total internal reflections until interacting with outcouplers of the sparse outcoupling region 104. The sparse outcoupling region 104 is configured (meaning the outcouplers are arranged) to generate a plurality of beams 108, each beam being of a different replicated processed image from a set of replicated processed images. The outcouplers are sparsely distributed over the outcoupling region 104, and may comprise any of the coupling features described above. Each beam 108 has a sufficiently small diameter to have greater, or extended, depth of field (e.g. is significantly smaller than a typical user’s pupil diameter), and the sparseness of the beams 108 is such that a maximum of one beam 108 may be seen by a pupil, as is the case for the dashed pupil position 110. However, due to the sparse spacing of the beams 108, certain pupil locations will not see any of the beams 108, such as the dotted pupil location 112.

[0083] The outcoupling region 104 illustrated in Figure 1 is defined as an area that contains all of the outcouplers making up the outcoupling region 104, or more particularly the convex hull of the set out outcouplers.

[0084] Example waveguide architectures 200, 250 that may be used to achieve the sparse outcoupling are illustrated in Figures 2A and 2B, respectively. Each waveguide architecture 200, 250 comprises an incoupling (IC) region 202, 252, an exit pupil expander (EPE) region 204, 254 and an outcoupling (OC) region, 206, 256. The EPE regions 204, 254 are configured to redirect light undergoing internal reflections from the IC regions 202, 252 towards the OC regions 206, 256.

[0085] The waveguide architecture 200 comprises a conventional EPE region 204 and an OC region 206 which is sparse in both directions. In particular, the OC region 206 is configured to generate replications on a triangular grid. The waveguide architecture 250 comprises an EPE region 254 that is sparse along a first direction and an OC region 256 that is sparse along a second direction.

[0086] The waveguide architecture 200 illustrated in Fig 2A may generally be preferred as it gives better defined output pupils, however the waveguide architecture 250 illustrated in Fig 2B may be useful if ever it is preferable for the EPE region 252 to be sparse for other reasons such as see through performance, and / or compatibility with a reflective waveguide design.

[0087] The skilled person is aware of other architectures that can be used to generate sparse replications. Note that in general, an EPE region may not be required where the display system already has a large etendue in one direction so that exit-pupil expansion is not required. That is, if the area of the OC region 206, 256 is no bigger than the area of the IC region 202, 252, or even if the projected width in some direction is no bigger than a width of IC region 202, 252, then no etendue increasing action is required at all. In other words, the input phase space could in some way be mapped to the output phase space with no replication, or replication in only one dimension. A cascaded design as used in a WaveOptics® waveguide may be used in some examples, in such constructions the out-coupling region also functions as an exit pupil expanding region, so that a separate exit pupil expanding region is not required.

[0088] The diameter of an individual outcoupler depends on resolution and focal range: too large an outcoupler results in components of a light field image having insufficient depth of field to be substantially ‘always in focus’, too small an outcoupler has insufficient diffraction limited resolution. Typically, an outcoupler diameter of around 0.5-lmm may be appropriate. The waveguide combiner may form part of a system with push-pull lenses to reduce a difference in focal power between the components of the light field image and virtual objects represented in the light field image.

[0089] The spacing between neighbouring outcouplers may be sufficiently large that it is uncommon for an eye pupil to intersect two beams 108 of two different replicated processed images from a set of replicated processed images, but sufficiently small that gaps in the field of view are relatively small. Centre-to-centre vertex spacing between neighbouring outcouplers of around 3mm to 7mm has been shown to provide a good balance based on the large range of possible human pupil sizes. For example, a spacing of 4mm has been found to provide a good compromise between image artefacts and required number of multiplexed outputs. Larger values may be desirable to allow for eye motion without requiring pupil tracking.

[0090] For an outcoupling region arrangement 104 as shown in Figure 1, Figure 3 illustrates a scenario where an input processed image, received by a waveguide combiner 300 (which may be the same waveguide combiner 100 as shown in Figure 1), has a large field of view. The range of angles of the input processed image is represented as cone 302. Light corresponding to the input processed image is replicated at the outcoupling region 304 to generate a set of replicated processed images. For each of the replicated processed images, the range of angles which is visible to a given eye pupil location 306 is represented as one of cones 308. With a large field of view, different outcouplers of the outcoupling region 304 contribute different patches of the field of view to a given pupil position 306. However, there will be regions of the field of view that are not seen from this pupil position 306 corresponding to the replications that miss the pupil. So, when an approximately collimated input beam 106 of an input processed image is received by the waveguide combiner 100 of Figure 1, the sparseness of the outcoupling region 104 is such that, at most, a single beam of one replicated processed image is received at the pupil position 110. Alternatively, when an input processed image having a large field of view is received by the waveguide combiner 300 of Figure 3, the sparseness of the outcoupling region 304 is such that several patches of a field of view (each of different replicated processed images from a set) are received at the pupil position 306.

[0091] In both cases though, there are some pupil positions (for example pupil position 112) that do not receive any beams of the replicated processed images. To avoid this, the density of replications can be increased by multiplexing one or more additional input processed images at the waveguide combiner 100, 300. An example of this is illustrated in Figure 4 which shows a waveguide combiner 400 comprising two incoupling regions 404, 408 and two spatially- overlapping outcoupling regions (illustrated by the dotted parallelograms 410, 412). Each of the outcoupling regions 410, 412 are similar to the outcoupling regions 104, 304 shown in Figures 1 and 3, in that they represent a region of the waveguide combiner 400 containing the sets of outcouplers corresponding to each outcoupling region 410, 412. Particularly, each of the outcoupling regions 410, 412 is a convex hull of their respective set of outcouplers.

[0092] Each of the incoupling regions 404, 408 is coupled to a corresponding one of the outcoupling regions 410, 412. A first incoupling region 404 is arranged to receive a beam 402 of a first input processed image and a first outcoupling region 410 is configured to output a first set of replicated beams 414 corresponding to the beam 402 of the first input processed image. Similarly, a second incoupling region 408 is arranged to receive a beam 406 of a second input processed image and a second outcoupling region 412 is configured to output a second set of replicated beams 416 corresponding to the beam 406 of the second input processed image. In Figure 4, the first set of replicated beams 414 is illustrated with solid lines and the second set of replicated beams 416 is illustrated with chained lines.

[0093] The outcouplers of the first and second outcoupling regions 410, 412 are spaced apart from one another. In other words, each outcoupler occupies a distinct non-overlapping region of the waveguide combiner 400. However, the outcoupling regions 410, 412 overlap, thereby increasing the density of replicated beams.

[0094] Comparing with the arrangement shown in Figure 1, the dotted pupil 418 (dotted pupil 112 in Figure 1) which received no replicated beams previously, now receives a replicated beam from the second outcoupling region 412. The example pupil position 420 continues to receive a replicated beam from the first set of replicated beams 414. As can be seen then, this arrangement results in a replicated beam configuration that ensures that at least one replicated beam will be incident on any pupil position, or alternatively, there are no pupil positions where no replicate beams are incident.

[0095] For a typical centre-centre spacing of 4-5mm for nearest outcouplers in any given set on a triangular tiling, each outcoupler serves a unit cell area of approximately 14-22mm2. For reasonable levels of image artefacts, it has been found that a light field may be distributed approximately every 1.5-3 mm2, so between around 5 to 15 such overlapping outcoupling regions would be desired in this scenario.

[0096] Multiplexing a plurality of beams 402, 406 of a plurality of input processed images not only increases the density of replicated beams 414, 416 outcoupled from the waveguide combiner 400, but allows the content of each input beam 402, 406 to be controlled accordingly, and hence for the outputs of the two sets of outcouplers defining outcoupling regions 410, 412 to be controlled accordingly. In particular, determining particular input angles and field content for the respective input processed images (as well as the sufficiently narrowness of each outcoupler) allows the waveguide combiner 400 to operate as part of a light field display. To see this, Figure 5 illustrates a side view of a similar arrangement 500 to that shown in Figure 4, but where the input processed images are shown as generated by respective image sources 504, 506. The image sources 504, 506 are configured to generate beams 508, 510 of first and second input processed images, respectively. In this context, the image sources 504, 506 may also be referred to as display modules or optical engines. The beams 508, 510 of the first and second input processed images are coupled into a waveguide combiner 502 (which may, or may not, be the waveguide combiner 400 shown in Figure 4) at respective incoupling regions 512, 514 as has been discussed above. Light generated by a first of the image sources 504 traverses the waveguide combiner 502 and is outcoupled at a respective outcoupling region, here shown comprising a first set of outcouplers 516, 518, 520. Similarly, light generated by a second of the image sources 506 traverses the waveguide combiner 502 and is outcoupled at a respective outcoupling region, here shown comprising a second set of outcouplers 522, 524, 526.

[0097] Each outcoupler of the first set of outcouplers 516, 518, 520 generates a replicated beam corresponding to light generated by the first image source 504, thereby forming a first set of replicated beams 536. Each replicated beam of the first set of replicated beams 536 is of a different replicated processed image in a first set of replicated processed images. Similarly, each outcoupler of the second set of outcouplers 522, 524, 526 generates a replicated beam corresponding to light generated by the second image source 506, thereby forming a second set of replicated beams 538. Each replicated beam of the second set of replicated beams 538 is of a different replicated processed image in a second set of replicated processed images.

[0098] According to the discussion above, the light generated by each image source 504, 506 is sparsely replicated in a direction substantially towards a viewer’s pupil. The sparseness of the outcouplers 516, 518, 520, 522, 524, 526 of each of the outcoupling regions is such that only a single replication of each input beam corresponding to each virtual image point of an input processed image is received at a viewing area 530 located near the pupil, with other spurious copies of the virtual image point missing the viewing area 530 and hence not being seen.

[0099] In Figure 5, this can be seen for an example virtual image point, where beams from a single example virtual image point 528 are received at the viewing area 530, while beams from other spurious copies of the virtual image point 532, 534 miss the viewing area 530. The other copies of the virtual image point 532, 534 may be referred to as spurious replicated virtual image points because they are not intended to reach the viewer’s pupil; they would appear as spurious artefacts if they did. The display system 500 is therefore arranged such that the spurious replicated virtual image points do not reach the viewing area 530, and so are not visible to the viewer. The result is a light field image displayed at the eyebox that is substantially free of artefacts.

[0100] The content that is displayed by each image source 504, 506 can be controlled such that the desired image content reaches the viewing area 530, as will be discussed in further detail with respect to the method 1500 shown in Figure 15. Then, if the pupil moves, the input processed images generated by the image sources 504, 506 can be updated to ensure that the desired target light field is produced at an updated viewing area 530 that approximately aligns with the updated pupil position.

[0101] Figures 4 and 5 show two input processed images multiplexed together for explanatory purposes, but it is understood that more than two such inputs may be multiplexed in a similar fashion. As one example, the arrangement 1200 shown in Figure 12, discussed in more detail below, has five inputs.

[0102] Figures 6 to 8 illustrate how the input processed images may be generated such that a light field image is formed at a viewing area. Figure 6 shows an optical arrangement 600 wherein beams corresponding to a first input processed image 604 are input to a first incoupling region 606 (i.e. incoupler) of a waveguide combiner 602. As described above, light corresponding to the first input processed image 604 traverses the waveguide combiner 602 and interacts with a first outcoupling region 608 comprising a first set of outcouplers 610. A first set of replicated processed images 612 are generated by the first outcoupling region 608 in a direction substantially towards a viewer.

[0103] For each virtual image point 614, 616, 618, 620 of a desired image (here illustrated as points behind the waveguide combiner 602 with respect to the viewer), a first set of construction rays 624, 626, 628, 630 may be determined (or drawn) such that each construction ray 624-630 has a first end at one of the virtual image points 614, 616, 618, 620 and a second end at an intersection with a viewing area 622. This can be achieved by, starting from each virtual image point 614-620, drawing a construction ray from that virtual image point 614-620 that passes through a centre of one of the outcouplers 610 and reaches the viewing area 622. If no such construction ray exists (i.e. no rays exist that start from a virtual image point 614-620, pass through the centre of one of the outcouplers 610, and are incident on the viewing area 622), then this virtual point cannot be displayed for the first input processed image 604, which is one reason that multiplexing input processed images is advantageous.

[0104] The content of the first input processed image 604 may be calculated, or determined, based on the first set of construction rays 624-630, for example by summing up all of the contributions corresponding to the first set of construction rays 624-630.

[0105] In some examples, the first set of construction rays 624-630 may be determined by determining a first set of cones 632, 634, 636, 638 representing an allowable space of construction rays that can be incident on the viewing area 622 for the given first set of outcouplers 610. Each cone of the first set of cones 632-638 may then be considered as an independent space of allowable virtual image points so that the desired virtual points 614-620 can be determined within the cones 632-638. This reduces compute resource utilisation because virtual image points that do not have associated valid construction rays (satisfying the above criteria) are not considered. Similarly, each cone of the first set of cones 632-638 comprises only a limited range of angles of construction rays, and hence may be associated with an independent patch of the field of view. Hence each patch of the field of view may be rendered to form render data independently according to its 3D (x, y, z) information to create the appropriate perspective for the corresponding cone. This render data can be combined to form the desired input processed image. Alternatively, each patch of the field of view may be rendered to form a first render data including depth information (such as a depth map, collection of independent layers, or otherwise) and then distorted independently of other patches, and depending upon its depth information, to form second render data with the appropriate perspective for the corresponding cone. This second render data can be combined to form the desired input processed image.

[0106] An optical system (not shown) may then generate the first input processed image 604 such that the desired virtual image points 614-620 are generated. The first set of replicated processed images 612 forms a first component of a light field image at the viewing area.

[0107] Figure 7 shows a similar optical arrangement 700 to that shown in Figure 6, but for beams corresponding to a second input processed image 704 that are input to a second incoupling region 706 (i.e. incoupler) of the waveguide combiner 602. The second input processed image 704 traverses the waveguide combiner 602 and interacts with a second outcoupling region 708 comprising a second set of outcouplers 710. A second set of replicated processed images 712 are generated by the second outcoupling region 708 in a direction substantially towards a viewer. The first and second sets of outcouplers 610, 710 are as described above in that their corresponding outcoupling regions 608, 610 at least partially overlap spatially, though each outcoupler may occupy a distinct non-overlapping region of the waveguide combiner 602.

[0108] Again, for each virtual image point 614, 616, 618, 620 of the desired image, a second set of construction rays 724, 726, 728 may be determined such that each construction ray 724- 728 has a first end at one of the virtual image points 614, 616, 618, 620 and a second end at an intersection with the viewing area 622. This can be achieved in a similar manner as has been described above for Figure 6. Note that in the arrangement 700 shown in Figure 7, it is not possible to determine a construction ray that has an end at the virtual image point 620, that passes through a centre of an outcoupler 710 of the second set of outcouplers 708, and reaches the viewing area 622. Hence, the virtual image point 620 cannot be addressed by the second input processed image 704.

[0109] The content of the second input processed image 704 may be calculated, or determined, based on the second set of construction rays 724-728, for example by summing up all of the contributions corresponding to the second set of construction rays 724-728. An optical system may then generate the second input processed image 704 such that the desired virtual image points 614-618 are generated. The second set of replicated processed images 712 forms a second component of the light field image at the viewing area 622.

[0110] Figure 8 illustrates an arrangement 800 that represents a combination of the arrangements 600 and 700 wherein the first and second input processed images 604, 704 (determined as described above) are received at the first and second incoupling regions 606, 706. The first and second set of replicated processed images 612, 712 are generated by the first and second outcoupling regions 608, 708 in a direction substantially towards a viewer. Light from the first and second sets of replicated processed images 612, 712 combines at the viewing area 622 to form the first and second components of the light field image respectively, and the first and second components of the light field image combine at the viewing area 622 to form the light field image. As can be seen from Figure 8, the first and second input processed images 604, 704 are such that rays corresponding to virtual image points 614, 616, 618 appear to diverge from the virtual image points 614, 616, 618, creating the desired light field.

[0111] The process described for the virtual image points 614-620 can be repeated and continued for any number of desired virtual image points, and any number of input processed images.

[0112] The waveguide combiners 100, 300, 400, 502, 602 shown in Figures 1, 3, 4, 5, 6, 7 and 8 show incoupling regions 102, 404, 408, 512, 514, 606, 706 and outcoupling regions 104, 304, 410, 412, 608, 708 with an assumption that some form of exit pupil expansion is used to get light from an incoupling region to an outcoupling region in the required way. As described above, an EPE region itself may not be necessary provided the display has a sufficiently large etendue. However, where this is not the case, an EPE region at the waveguide combiner may be used to perform exit pupil expansion. Figure 9 shows an example waveguide k-space diagram 900 showing: an extent 902 in k-space of light incident on an incoupling region; a k- vector 904 corresponding to the incoupling region; an extent 906 in k-space of light after interaction with the incoupling region; a k-vector 908 corresponding to an EPE region; an extent 910 in k-space of light after interaction with the EPE region; a k-vector 912 corresponding to an outcoupling region; and an extent 914 in k-space of light after interaction with the outcoupling region.

[0113] In this example 900, the k vectors 904, 908, 912 form a closed triangle, so that the extent 902, 914 in k-space of the incoupled and outcoupled light are congruent. That the k- vectors form a closed triangle may be desirable for a broadband image source to avoid chromatic spread, but may not be necessary for a monochromatic image source, such as a laser. Also, in this k-space diagram 900, the incoupling and outcoupling region k-vectors 904, 912 are shown as being at 90 degrees to one another. It is understood that this is just an example, and not necessary. In particular, other means of transporting light from the incoupling region to the outcoupling region are known, such as cascaded EPE region as used in a WaveOptics® waveguide.

[0114] Now that a suitable k-space diagram 900 has been described for a single input, an example of how a plurality of input processed images can be multiplexed will now be described using the k-space diagram 900 as a foundation. It may be desirable for each outcoupling region to only interact with light coming from its corresponding image source. For n multiplexed outcoupling regions, this can be achieved by using n corresponding image sources, each with its own corresponding incoupling region and EPE, such that the light from the n different image sources is separated in k-space when interacting with the outcoupling regions.

[0115] Figure 10 shows an example k-space diagram 1000 where n=5. As will be described in further detail later, it will be seen that n=5 may be an appropriate number of image sources and EPEs to attempt to fit into a glasses or spectacles form factor. Figure 10 shows regions of k- space 1002-1010 occupied by light corresponding to five separate input processed images following interaction with their respective incoupling regions and EPEs. The light corresponding to each input processed image has undergone a process similar to that shown in Figure 9 for a single image field. In particular, light has been incoupled into the waveguide combiner, transported via an incoupling region k-vector and an EPE region k-vector to the regions 1002-1010. It can be seen that the extent in k-space of light from each incoupling region has been successively rotated by an additional 60 degrees in k-space. In particular, an extent in k-space 1002 of light from a first incoupling region incident on a corresponding first outcoupling region is rotated by 60 degrees with respect to an extent in k-space 1004 of light from a second incoupling region incident on a corresponding second outcoupling region, which itself rotated by 60 degrees with respect to an extent in k-space 1006 of light from a third incoupling region incident on a corresponding third outcoupling region, and so on. The k-space diagram 1000 shows five k-vectors 1012-1020 describing five sparse, overlapping outcoupling regions configured to out-couple their respective sets of replicated processed images to form a composite light field image 1022. The outcoupling region k-vectors 1012-1020 are of equal magnitude and arranged to transport light corresponding to a respective image source to an extent of k-space occupied by the composite light field image 1022. Each of the k-vectors 1012-1020 form an edge of a triangular tiling. Note that, in the example shown in Figure 10, the orientation of the input processed images (which could mean for example, the orientation of the optical engine, or of a display in the optical engine, that is generating the input processed image) may be such that the composite light field image 1022 has five overlapping fields at the centre, and fewer at the edges, to enlarge a field of view without reducing the number of light fields covering a central portion.

[0116] The k-space diagram 1000 illustrates an arrangement that may ensure that spurious out- couplings of an input image field, whilst having inconsistent focus due to being coupled out of an incorrect outcoupling region, still have the correct output k-vectors. This means that the spurious output image is as closely matched to the desired image as possible, rather than having multiple significantly shifted spurious image copies. This is illustrated in Figure 11 A which shows a k-space diagram 1100 illustrating an example spurious out-coupling (light outcoupled by an outcoupling region associated with a non-corresponding incoupling region).

[0117] Light having interacted with a corresponding EPE region occupies an extent 1102 of k- space as described above with respect to Figures 9 and 10. The outcoupling region corresponding to this light has a k-vector 1110 arranged to outcouple the light in the desired region of k-space. However, the light occupying the extent 1102 of k-space after interacting with the EPE region may interact with one or more outcoupling regions that correspond to different incoupling and EPE regions.

[0118] As shown in Figure 11 A, light occupying a region or extent 1102 of k-space after having interacted with its corresponding EPE region may interact with an outcoupling region k-vector 1104 corresponding to a different outcoupling region. The k-vector 1104 transports the light to a region 1108 of k-space, but without outcoupling the light since this outcoupling region defined by k-vector 1104 does not correspond to the incoupling region from which this light is received. The light is then out-coupled by outcoupling region k-vector 1106, which again is associated with an outcoupling region that does not correspond to the incoupling region from which the light is received.

[0119] Comparing with the k-space diagram 1000 shown in Figure 10, it can be seen that the light has interacted with outcoupling region k-vectors 1016 and 1020. The resultant k-vector 1110 is equal to the vector sum of the outcoupling region k-vectors 1016 and 1020 and produces an out-coupled image 1112 which is congruent in k-space with the desired out-coupled image. The outcoupling location of this image will be spatially offset as the light has been out-coupled from an outcoupling region with k-vector 1106 rather than the outcoupling region with k-vector 1110. There will be other interactions between light that has interacted with a given EPE region and other non-corresponding outcoupling regions, but the light will typically be outcoupled at a different region in k-space that is away from the region occupied by the composite light field image 1022, meaning the light will be outcoupled in a direction away from the viewing area.

[0120] Overall then, the type of triangular relationship between outcoupling region k-vectors as shown in Figure 10 ensures that spurious outcouplings in the region of k-space occupied by the desired composite light field image are congruent in k-space to the desired outcoupling.

[0121] Generally, it may be desirable, irrespective of configuration, to try to minimise spurious outcouplings between light from different image sources, for example, by having a blazed or slanted SRG (Surface Relief Grating), a volume holographic grating, or any other angularly selective outcoupler. However, given some level of spurious interaction, it can be seen that such an arrangement of k-vectors shown in Figure 10 may be preferred, as it minimises the perceived impact of the error resulting from a given level of spurious out-coupling. Other arrangements of k-vectors are possible that may have a similar advantageous effect on spurious outcouplings, such as k-vectors forming an edge of a square tiling etc.

[0122] Figure 11B shows a k-space diagram 1150 illustrating an example spurious outcoupling in a square tiling arrangement, similar to the triangular tiling arrangement shown in Figure 11 A. As in that example, light having interacted with a corresponding EPE region occupies an extent 1152 of k-space as described above with respect to Figures 9 and 10. The outcoupling region corresponding to this light has a k-vector 1162 arranged to outcouple the light in the desired region of k-space. However, the light occupying the extent 1152 of k-space after interacting with the EPE region may interact with one or more outcoupling regions that correspond to different incoupling and EPE regions.

[0123] As shown in Figure 1 IB, light occupying a region or extent 1152 of k-space after having interacted with its corresponding EPE region may interact with an outcoupling region k-vector 1154 corresponding to a different outcoupling region. The k-vector 1154 transports the light to a region 1156 of k-space, without outcoupling the light since this outcoupling region defined by k-vector 1154 does not correspond to the incoupling region from which this light is received. In the square tiling arrangement, this region 1156 includes an additional section labelled X that falls outside the range of k-vectors supported by the waveguide, which means that the coupling of k-vector 1154 is weaker than the coupling of k-vector 1104 in the triangular tiling arrangement shown in Figure 11 A.

[0124] The light then interacts with an additional outcoupling region k-vector 1158, which further transports the light before final outcoupling by outcoupling region k-vector 1160. This additional interaction with k-vector 1158, which is not present in the triangular tiling arrangement of Figure 11 A, further weakens the relative strength of the spurious outcoupling compared to the desired outcoupling of light.

[0125] The resultant k-vector 1162 produces an out-coupled image 1164 which, as in the triangular tiling case, is congruent in k-space with the desired out-coupled image, but the outcoupling location will be spatially offset. Due to the two effects described above (the weaker coupling of k-vector 1154 due to section X falling outside the supported k-vector region, and the additional interaction with k-vector 1158) the square tiling arrangement results in reduced relative strength of crosstalk compared to the triangular tiling arrangement shown in Figure 11 A.

[0126] Figure 12 illustrates an example waveguide combiner arrangement 1200. The arrangement 1200 uses a waveguide combiner 1202 and comprising five incoupling regions 1204-1212. Each of the incoupling regions 1204-1212 has a respective EPE region 1214-1222 and receives light from a respective image source, or optical engine (not shown). The light is then redirected by its respective EPE region 1214-1222 analogously to the way shown for the light in Figure 9. Following redirection from their respective EPE region 1214-1222, light occupies the regions of k-space 1002-1010 respectively, as shown in Figure 10.

[0127] Five sets of outcouplers 1224-1232 occupy a spatially overlapping region 1234 of the waveguide combiner 1202. The outcoupling region k-vectors of each sparse outcoupling region defined by each of the sets of outcouplers 1224-1232 may be the outcoupling region k-vectors 1012-1020 shown in Figure 10. The arrangement of sparse outcoupling regions is such that: gaps between outcouplers from the same set of outcouplers are kept as large as possible (given the overall spot density) and; the overall spot pattern is approximately uniformly distributed, in order to keep the largest gaps in the pattern as small as possible.

[0128] In the example arrangement 1200 illustrated in Figure 12, with five image sources, there is sufficient space for all five EPE regions 1214-1222, and an appropriately large area 1234 for the five sets of outcouplers 1224-1232. While there may be a spatial limit on the number of image sources that can be multiplexed based on the display form factor, additional image sources can be added by, for example, including a plurality of stacked waveguide combiners. Each of the stacked waveguide combiners may have a similar arrangement to that of Figure 12, but with differently distributed outcoupling region spots.

[0129] Figure 12A shows a close up of the overlapping region 1234 of the waveguide combiner 1202, illustrating the outcoupling region 1236 for the set of outcouplers 1224 and the outcoupling region 1238 for the set of outcouplers 1232. The outcoupling regions 1236, 1238 are defined as the convex hull of the respective set of outcouplers 1224, 1232.

[0130] Note that the arrangement 1200 illustrated in Figure 12, whereby a plurality of image sources are located around a perimeter of a waveguide combiner in this manner, may be suitable for other display types, such as CGH or standard 2D displays.

[0131] All the k-vector diagrams shown so far have assumed monochrome light for simplicity. However, it is understood that the techniques described herein work for a colour display and / or with broadband LED sources. These techniques are applicable where the waveguide combiner system comprises one waveguide combiner or more than one waveguide combiner (e.g. stacked combiners).

[0132] In some examples, a plurality of waveguide combiners (also referred to as layers, or plates) may be used to give sufficient spectral bandwidth in the same ways that are used for conventional waveguides. Each waveguide combiner (or layer) may have one or more incoupling features that are specific to light of a certain range of wavelengths, and so only incouple light at those certain ranges of wavelengths. This allows for a spatial multiplexing between waveguide combiners (or layers). In some other examples, the incoupling regions themselves may be distributed appropriately to only incouple light from a desired one or more image sources.

[0133] The number of outputs can be increased without increasing the number of image sources in some examples. For example, an image source with a sufficiently fast refresh rate (e.g. some multiple of 60Hz) can be time-multiplexed, such that light is incoupled to a different waveguide combiner (or different layer) as a function of time. This could be achieved by controlling the output polarisation of the image source, and using incoupling regions which have significantly better coupling efficiency for a certain polarisation, or by using polarisation volume gratings for the incoupling regions, or by other means.

[0134] Figure 13A illustrates an AR glasses configuration 1300 utilising two copies 1302, 1304 of the waveguide combiner arrangement 1200 shown in Figure 12, one for each eye, wherein the two waveguide combiners are mirror-images of one another. Other configurations of the two waveguide combiners relative to one another are possible, for example where the two waveguide combiners are not mirror-images of one another. The waveguide combiner arrangements 1302, 1304 are mounted on or partially within a frame, 1306, such that the images sources may be significantly concealed and / or embedded within the frame.

[0135] It may be seen that for each copy 1302, 1304, at least some of the image sources, or optical engines, are at a relatively large distance from one another. In some examples, at least two optical engines are at least 25mm apart, at least 30mm apart, at least 40mm apart, at least 50mm apart.

[0136] The waveguide combiner arrangements 1302, 1304 are mounted on or partially within frame 1306 such that at least part of the waveguide combiner system, such as the incoupling regions of the combiner system, may also be significantly concealed, embedded, or partially embedded within the frame.

[0137] Figure 13B illustrates an alternative AR glasses configuration 1350 utilising two copies 1352, 1354 of a waveguide combiner arrangement similar to that shown in Figure 12, but implementing a square tiling in k-space as described with reference to Figure 1 IB. As described above, the effect of crosstalk caused by spurious outcouplings can be reduced in a square tiling compared with a triangular tiling.

[0138] As with Figure 13 A, the two waveguide combiners 1352, 1354 may be mirror-images of one another, though other configurations are possible. The waveguide combiner arrangements 1352, 1354 are mounted on or partially within a frame 1356. In this configuration, the positioning of the optical engines may be ergonomically improved compared with the arrangement shown in Figure 13 A. The frame areas near the temples of the user and the arms of the glasses, as well as the area near the user’s cheekbones, provide more available space and are therefore preferable locations for optical engine placement in practical implementations. Accordingly, most of the optical engines are positioned toward the “outside” portions of the frame 1356, away from the central nose area, which provides improved ergonomics and packaging efficiency.

[0139] In some examples, the configuration may utilise four optical engines per eye, with the optical engine positioned nearest to the nose area having lower specifications compared to the other engines, such as smaller field of view, lower brightness, lower resolution, or monochrome operation. This may be due to the more constrained packaging space in that region and / or the reduced weight of such an optical engine positioned near the user’s nose. Alternatively, the configuration may utilise only three optical engines per eye, in which case the optical engine nearest the nose may be omitted entirely. Note that future developments in optical engine miniaturisation may provide greater freedom over optical engine numbers and placement within the glasses frame. It is understood that the AR glasses configurations 1300 and 1350 illustrated in Figures 13 A and 13B may be more generally utilised, for example, where the waveguide combiners are not necessarily those described above with sparse outcouplers, and the display system does not need to be capable of generating light field imagery, which is possible due to the use of appropriately small image sources, or optical engines, such as micro LEDs, as discussed above.

[0140] Figure 14 shows a schematic diagram of an example display system 1400 that may be implemented in any of the above described systems to generate and display light field images to a viewer. The display system 1400 comprises a processing system 1410, an optical system 1420, waveguide combiner system 1430, and a pupil tracking system 1440. The processing system 1410, which may also be referred to as a controller, may comprise one or more processors, memory, and software components. The one or more processors are configured to process data, and the memory can comprise a computer-readable medium (e.g., a tangible, non- transitory computer-readable medium, data storage loaded with one or more of the software components) configured to store instructions for performing various operations and / or functions. The processors are configured to execute the instructions stored on the memory to perform one or more of the operations.

[0141] The processing system 1410 is coupled to the optical system 1420 which is configured to generate input processed images according to instructions received from the processing system 1410. The optical system 1420 is arranged to display a plurality of input processed images near respective entrance pupils of the waveguide combiner system 1430, such that the waveguide combiner system 1430 produces a target (composite) light field for display to a viewer. The processing system 1410 may be part of, or remote from, the optical system 1420.

[0142] The pupil tracking system 1440 is coupled to the processing system 1410 and is configured to monitor a location of a viewer’s pupil. The pupil tracking system 1440 provides the processing system 1410 data indicating the location of the viewer’s pupil. The pupil tracking system 1440 may provide periodically, aperiodically or substantially continuously updated data indicating the location of the viewer’s pupil as a function of time. The processing system 1410 uses the provided data to determine a location at which to display the target light field. The processing system 1410 also calculates the target light field image to be displayed at the determined location. Further, the processing system 1410 may be configured to calculate the input processed images by first determining respective sets of construction rays, as described with reference to Figures 6, 7, 8 and as will be discussed in more detail with reference to Figure 15. The processing system 1410 is further configured to instruct the optical system 1420 to display the calculated input processed images. In some examples, the pupil tracking system 1440 is configured to generate data indicating the location of the viewer’s pupil as part of an initial calibration procedure. After that initial calibration procedure, the pupil tracking system 1440 may then not further monitor the location of a viewer’s pupil, so that the location remains the same until another calibration is executed. In other examples, the pupil tracking system may not be needed, and the viewing area is determined by others means, as discussed above.

[0143] Figure 15 shows an example method 1500 of displaying a light field image. The method utilises a waveguide combiner system comprising a plurality of sets of sparse outcouplers, such as in any of waveguide combiner arrangements shown in Figures 4, 5, 6, 7, 8, 12, 13 and 14. The method 1500 may be similar to what has been described with regards to Figures 6 to 8, and may be performed by a processing system, such as the processing system 1410 of the display system 1400.

[0144] At block 1502, the method 1500 comprises determining a viewing area. This may be a predetermined location where a viewer’s pupil is expected to be located when the waveguide combiner is in use. The predetermined location may be determined as part of a calibration procedure. In some other examples, the viewing area may be determined by a pupil tracking system that detects a current position of a viewer’s pupil.

[0145] At block 1504, the method 1500 comprises determining sets of construction rays. Each construction ray of the sets of construction rays has a first end at one of a set of virtual image points and a second end at an intersection with the viewing area. Each construction ray of a first set of construction rays passes through a centre of an outcoupler of a first set of outcouplers of the waveguide combiner, and each construction ray of a second set of construction rays passes through a centre of an outcoupler of a second set of outcouplers of the waveguide combiner. As described above, the sets of construction rays can be determined in any suitable way. For example, the sets of construction rays may be determined for each desired virtual image point by drawing the construction rays and determining which construction rays are valid. In another example, the sets of construction rays may be determined by first determining a set of cones representing the space in which valid construction rays can be constructed, and then determining which desired virtual image points fall within the set of cones.

[0146] At block 1506, the method 1500 comprises displaying a plurality of input processed images. Each of the input processed images comprises a set of beams having angles relating to a respective set of construction rays. The input processed images may be displayed by a suitable optical system. At block 1508, the method 1500 comprises replicating the plurality of input processed images to produce sets of replicated processed images. Light corresponding to the input processed images is replicated by the waveguide combiner. Light from the respective sets of replicated processed images combines at the viewing area to form respective components of a light field image, and the components of the light field image combine at the viewing area to form the desired light field image.

[0147] The techniques described so far relate to display of light field images. Returning to Figure 5, which shows a light field image displayed at a viewing area 530, the viewing area 530 may be considered a valid eyebox as it is the desired location that the target light field is to be viewed. There are other eyeboxes present in this arrangement 500 corresponding to the light outcoupled from out-couplers 516, 520, 522, 526, but which miss the viewer’s pupil, and are a consequence of the replicating nature of the waveguide combiner 502. The arrangement 500 in Figure 5 is such that only one copy of a virtual image point reaches the pupil, and no spurious virtual image points reach the pupil, resulting in a good quality light field image without artefacts.

[0148] This is due to the viewing area 530 (i.e. valid eyebox) being located at the viewer’s pupil, and so this is a typical choice for locating the viewing area. However, there are advantages where, instead, the viewing area is located at the rotational centre of the eye. For instance, this positioning potentially avoids the need to have a sufficiently large viewing area to handle gaze-dependent eye pupil location. Alternately, it avoids the need to track eye location with changing gaze, and to move the viewing area accordingly which may impose high computational requirements.

[0149] To see this, Figures 16A and 16B illustrate an example arrangement where a viewing area 1605 is positioned at a rotational centre 1604 of a viewer’s eye 1606. In Figure 16A, the eye 1606 is looking horizontally at point 1608, while in Figure 16B, the eye 1606 has rotated around a rotational centre point and is looking at point 1616.

[0150] In this arrangement, there is a viewing area 1605 and spurious replicated points 1610,1612 and 1614,1618. Even though the pupil sweeps out a larger area than the viewing area , by positioning the viewing area 1605 at the rotational centre of the eye 1606, spurious copies of the point being looked at are not seen, and only spurious copies of the point not being looked at are seen. In other words, the cone of spurious rays 1620 from point 1614 are seen when the eye is looking at the point 1608, and the cone of spurious rays 1622 from point 1612 when the eye is looking at 1616. Figure 17 shows an example method 1700 of displaying a target light field using a pupil replicating optical system. The method may be used in conjunction with the method 1500 of displaying a light field image shown in Figure 15, and so the method 1700 is compatible with any of the examples discussed herein. For instance, the pupil replicating optical system may comprise any of the waveguide combiners shown in Figures 1, 3, 4, 5, 6, 7, 8, 12, 13 and 14. However, the method 1700 may be also applied to any display that has a notion of viewing area, including holographic field displays.

[0151] At block 1702, the method 1700 involves determining a target light field to be displayed at a rotational centre of a viewer’s eye. The target light field may be a holographic field or a 4- dimensional light field. The location of the rotational centre of the viewer’s eye may be determined prior to determining the target light field. This could involve performing an initial calibration procedure that locates the centre of the viewer’s eye, and which is fixed until a later calibration occurs. This could also be achieved using pupil tracking.

[0152] At block 1704, the method 1700 involves determining an input light field to be displayed at an input location such that the target light field is produced at the rotational centre of the viewer’s eye. Various methods of determining such an input light field have been discussed herein (such as in the method 1500), though it is understood that any suitable technique could be used.

[0153] At block 1706, the method 1700 comprises displaying the input light field, for example, by any optical engine or image source described herein.

[0154] The above examples are to be understood as illustrative examples of the invention further examples are envisaged. For example, although the description describes light field control of focus in order to display virtual image points at different depths, the same technique may be additionally used for controlling higher order aberrations, for example astigmatism and coma. This could be used to correct for the prescription of a viewer, or for aberrations of the optical system. In the latter case, this may allow for increased flatness and stiffness tolerance of the waveguide combiner(s), allowing for a lighter waveguide combiner system.

[0155] In addition, one or more of the sets of outcouplers in the waveguide combiner system may also outcouple one or more sets of invisible light beams, for example infra-red beams. The sparseness of the sets of outcouplers may result in a structured light pattern projected on or near the eyes of a viewer, which may assist with eye-tracking.

[0156] Eye glow is a phenomenon in AR glasses where part of the image intended for the wearer leaks out through the front of the waveguide, making the display content faintly visible to onlookers. It may appear as a glowing image on the lens surface, and may be especially noticeable in low light environments. The glow may look unusual and can make AR glasses feel awkward or conspicuous, reducing consumer willingness to wear them, and limiting their adoption more widely.

[0157] To eliminate eye glow in any of the embodiments described herein (and indeed in any waveguide combiner system comprising sparse outcouplers), the waveguide combiner system may comprise masked-off outcouplers. The masking may be applied by blocking areas corresponding to the outcouplers on the outer surface of the display. Such masking is feasible in situations where the outcoupling is sparse, as described herein. While masking off outcoupled areas may affect the see-through experience by reducing transparency in those regions, this approach can solve the eye glow problem.

[0158] Alternative design methods may also be used, such as methods using similar design constraints and assumptions.

[0159] It is to be understood that any feature described in relation to any one embodiment or example may be used alone, or in combination with other features described, and may also be used in combination with one or more features of any other of the embodiments or examples, or any combination of any other of the embodiments or examples. Furthermore, equivalents and modifications not described above may also be employed without departing from the scope of the invention, which is defined in the accompanying claims.

Claims

CLAIMS1. A waveguide combiner system comprising: a plurality of incoupling regions, each incoupling region arranged to incouple light corresponding to a respective one of a plurality of input processed images relating to a light field; a plurality of sets of outcouplers, each of the sets of outcouplers coupled to a respective incoupling region and each set of outcouplers defining a respective outcoupling region, the plurality of outcoupling regions at least partially overlapping spatially; and one or more waveguide combiners, at least one of the one or more waveguide combiners comprising at least two of the incoupling regions and corresponding sets of outcouplers, wherein: each set of outcouplers is configured to replicate light corresponding to a respective input processed image to generate a respective set of replicated processed images, each set of replicated processed images combining at a viewing area to form a component of a light field image, such that the components of the light field image combine at a viewing area to form the light field image, and an area covered by a set of outcouplers is less than half an area covered by the respective outcoupling region.

2. The waveguide combiner system according to claim 1, wherein the coupling between each incoupling region and corresponding outcoupling region is such that light corresponding to each incoupling region received at an outcoupler occupies a region of k-space different from others of the incoupling regions.

3. The waveguide combiner system according to claim 1 or claim 2, wherein the outcouplers from any set do not overlap any outcouplers from any other set.

4. The waveguide combiner system according to any preceding claim, comprising at least two waveguide combiners each of the at least two waveguide combiners comprising at least one incoupling region and corresponding set of outcouplers.

5. The waveguide combiner system according to any preceding claim, comprising a plurality of exit pupil expander regions arranged to:receive light from a respective one of the incoupling regions, replicate the received light, and transmit the replicated received light to a respective one of the outcoupling regions.

6. The waveguide combiner system according to any preceding claim, wherein centres of nearest outcouplers in a set of outcouplers are separated by at least 3mm at the outcoupling region.

7. The waveguide combiner system according to any preceding claim, wherein a width of the outcouplers in a set of outcouplers is less than 1mm.

8. The waveguide combiner system according to any preceding claim, wherein each set of outcouplers has a respective k-vector such that a magnitude of a difference between any two k-vectors is greater than a threshold amount.

9. The waveguide combiner system according to any preceding claim, wherein each set of outcouplers has a respective k-vector, and each of the k-vectors corresponding to the plurality of sets of outcouplers forms an edge of a triangular tiling.

10. The waveguide combiner system according to any of claims 1 to 8, wherein each set of outcouplers has a respective k-vector, and each of the k-vectors corresponding to the plurality of sets of outcouplers forms an edge of a square tiling.

11. The waveguide combiner system according to any preceding claim, wherein a thickness of each of the one or more waveguide combiners is less than 2mm.

12. The waveguide combiner system according to any preceding claim, wherein the viewing area is positioned at a rotational centre of a viewer’s eye.

13. A light field display system comprising: an optical system configured to generate a plurality of input processed images relating to a light field image; andthe waveguide combiner system according to any preceding claim arranged in an optical path between the optical system and a viewing area.

14. The light field display system according to claim 13, wherein the optical system comprises a plurality of optical engines, wherein each of the optical engines is configured to generate light corresponding to a at least one input processed image of the plurality of input processed images.

15. The light field display system according to claim 14, wherein the plurality of optical engines comprises at least three optical engines.

16. The light field display system according to any of claims 13 to 15, wherein the optical system is configured to generate light at a plurality of wavelengths.

17. The light field display system according to any of claims 13 to 16, wherein the optical system comprises an optical engine configured to: at a first time, generate light having a first characteristic and corresponding to a first input processed image of the plurality of input processed images, wherein the first characteristic causes the light corresponding to the first input processed image to be coupled to a first incoupling region; and at a second time, generate light having a second characteristic and corresponding to a second input processed image of the plurality of input processed images, wherein the second characteristic causes the light corresponding to the second input processed image to be coupled to a second incoupling region.

18. The light field display system according to any of claims 13 to 17, comprising: a controller configured to determine the plurality of input processed images based on a determined viewing area, and to cause the optical system to generate the light corresponding to the plurality of input processed images.

19. The light field display system according to claim 18, wherein the determined viewing area is based on a determined pupil position.

20. The light field display system according to claim 19, comprising:a pupil tracking system configured to determine the determined pupil position.

21. The light field display system according to claim 20, wherein the pupil tracking system is further configured to determine a width of a pupil, and wherein the controller is further configured to determine the plurality of input processed images such that each input processed image is the same input processed image when the determined width of the pupil is greater than a threshold width.

22. The light field display system according to any of claims 13 to 21, comprising: a push-pull lens arranged to reduce a difference in focal power between the components of a light field image and a virtual object represented in the light field image.

23. A head-mounted display comprising the light field display system according to any of claims 13 to 22.

24. A pair of glasses comprising: a first light field display system according to any of claims 13 to 22 corresponding to a viewer’s left eye; and a second light field display system according to any of claims 13 to 22 corresponding to a viewer’s right eye.

25. A display system comprising: a waveguide combiner system having a perimeter; and a plurality of optical engines positioned along the perimeter, wherein: the waveguide combiner system comprises: a plurality of incoupling regions, each incoupling region arranged to receive light from a respective one of the optical engines; a plurality of outcoupling regions, each outcoupling region coupled to a respective incoupling region and configured to output light received from the respective incoupling region towards a viewing area; and at least one waveguide combiner comprising at least two of the incoupling regions and corresponding outcoupling regions.

26. A pair of glasses comprising: a frame delimiting at least two eye apertures; a first at least partially transparent element within a first of the two eye apertures; a second at least partially transparent element within a second of the two eye apertures; a first plurality of optical engines embedded in the frame around a first of the at least two eye apertures; and a second plurality of optical engines embedded in the frame around a second of the at least two eye apertures, wherein at least two optical engines of each of the first and second pluralities of optical engines are at least 25mm apart, and wherein the first and second at least partially transparent elements comprise respective waveguides arranged in optical paths between the respective plurality of optical engines and a respective viewing area.

27. A method of displaying a light field image comprising: determining a viewing area; determining a first and second set of construction rays, each construction ray of the first and second set of construction rays having a first end at one of a set of virtual image points to be displayed and a second end at an intersection with the viewing area, wherein: each construction ray of the first set of construction rays passes through a centre of an outcoupler of a first set of outcouplers of the waveguide combiner system according to any of claims 1 to 11, and each construction ray of the second set of construction rays passes through a centre of an outcoupler of a second set of outcouplers of the waveguide combiner system; displaying, by an optical system, first and second input processed images, wherein: the displayed first input processed image comprises a first set of beams having angles relating to the first set of construction rays, and the displayed second input processed image comprises a second set of beams having angles relating to the second set of construction rays; and replicating, by the waveguide combiner system arranged in an optical path between the optical system and the viewing area, the first and second input processed images to produce first and second sets of replicated processed images, wherein:light from the first set of replicated processed images combines at the viewing area to form a first component of a light field image, light from the second set of replicated processed images combines at the viewing area to form a second component of a light field image, and the first and second components of the light field image combine at the viewing area to form the light field image.

28. The method according to claim 27, wherein the viewing area is determined based on a detected position of a viewer’s pupil received from a pupil tracking system.

29. The method according to claim 27 or claim 28, comprising: determining a width of the viewer’s pupil is greater than a threshold width, and wherein the first and second input processed images are determined to be the same input processed image.

30. A method of displaying a target light field using a pupil replicating optical system, the method comprising: determining a target light field to be displayed at a rotational centre of a viewer’s eye; determining an input light field to be displayed at an input location such that the target light field is produced at the rotational centre of the viewer’s eye; and displaying the input light field at the input location.

31. The method according to claim 30, comprising: prior to determining the target light field, determining the rotational centre of the viewer’s eye.

32. The method according to claim 28 or claim 31, wherein the target light field is a holographic field or a 4-dimensional light field.

33. The method according to claim 32, wherein the target light field comprises a 4- dimensional light field comprising a plurality of replicated processed images corresponding to a light field image, and wherein displaying the input light field at the input location comprises, displaying, via respective light sources, light corresponding to a plurality of input processed images.

Images