Holographic virtual reality display

Abstract

The present disclosure relates to holographic virtual reality displays. A virtual reality (VR) display is a computer display that presents images or video in a way that simulates a real experience for a viewer. In many cases, VR displays are implemented as head-mounted displays (HMDs) that provide a display within the user's field of view. Since current HMDs consist of a display panel and a magnifying lens with a gap between the two, the normal functioning of the HMD limits its design to a box-like form factor, negatively impacting comfort and aesthetics. The present disclosure provides a different configuration for a VR display that allows for improved comfort and aesthetics, specifically including at least one coherent light source, at least one pupil replication waveguide coupled to the at least one coherent light source to receive light therefrom, and at least one spatial light modulator coupled to the at least one pupil replication waveguide to modulate the light.

Description

[0001] Cross-references to related applications

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 176,108, filed April 16, 2021, entitled “Virtual Reality Display with Holographic Optics,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to virtual reality displays. Background Technology

[0004] A virtual reality (VR) display is a computer monitor that presents images or videos in a way that simulates a realistic experience for the viewer. For example, a VR display can present a three-dimensional (3D) environment, which may or may not be interactive. VR displays are very useful for a variety of applications that use VR, such as entertainment (e.g., video games), education (e.g., training), and business (e.g., conferences).

[0005] In many cases, VR displays are implemented as head-mounted displays (HMDs). By definition, an HMD is worn on a user's head to provide a display within the user's field of vision. By viewing the display, the user is able to experience VR. To encourage wider adoption of HMDs, it is important to focus HMD design on a more comfortable form factor, higher performance, and better aesthetics.

[0006] However, the typical configuration of HMDs to date has limited their comfort and aesthetics. In particular, HMDs currently consist of a display panel and a magnifying lens (i.e., an eyepiece). To provide the user with a perceptible image, the distance between the display panel and the lens should be slightly less than the lens's focal length. Since it is not feasible to manufacture lenses with very short focal lengths for a given aperture size, current HMDs have a boxy shape, thus failing to replicate traditional forms of eyeglasses, negatively impacting both their comfort and aesthetics.

[0007] These issues and / or other problems related to existing technologies need to be addressed. Summary of the Invention

[0008] An apparatus and related method for a holographic virtual reality (VR) display are disclosed. The VR display includes at least one coherent light source, at least one pupil replica waveguide coupled to the at least one coherent light source to receive light from it, and at least one spatial light modulator coupled to the at least one pupil replica waveguide to modulate the light. Attached Figure Description

[0009] Figure 1 A VR display with a coherent light source, a pupil replication waveguide, and a spatial light modulator according to one embodiment is shown.

[0010] Figure 2 An implementation of a VR display having a coherent light source, a pupil replication waveguide, a spatial light modulator, and a linear polarizer according to one embodiment is shown.

[0011] Figure 3 An implementation of a VR display having a coherent light source, a pupil replication waveguide, a spatial light modulator, and a geometric phase lens according to one embodiment is shown.

[0012] Figure 4 An implementation of a VR display having a coherent light source, a pupil replication waveguide, a spatial light modulator, a geometric phase lens, and a quarter-wave plate according to one embodiment is shown.

[0013] Figure 5 An implementation of a VR display having a coherent light source, a pupil replication waveguide, a spatial light modulator, a linear polarizer, a geometric phase lens, and a quarter-wave plate according to one embodiment is shown.

[0014] Figure 6A An implementation of a VR display with a coherent light source, a pupil replica waveguide, and a spatial light modulator according to one embodiment is shown, wherein the VR display exhibits a high diffraction order.

[0015] Figure 6B An implementation of a VR display having a coherent light source, a pupil replication waveguide, and a spatial light modulator according to one embodiment is shown, wherein the incident angle of light received by at least one pupil replication waveguide is adjustable.

[0016] Figure 7 An implementation of the pupil replication waveguide according to an embodiment is shown.

[0017] Figure 8 An operating method of the HMD according to an embodiment is shown.

[0018] Figure 9 An exemplary computing system according to an embodiment is shown. Detailed Implementation

[0019] Figure 1A VR display 100 having a coherent light source, a pupil replication waveguide, and a spatial light modulator according to an embodiment is shown. In the context of this specification, the VR display 100 is a device configured to display VR images and / or VR videos for a user to view. In one embodiment, the VR display 100 may be an HMD capable of being worn on a user's head to provide a display within the user's line of sight, or the VR display 100 may be a sub-component of such an HMD.

[0020] As shown in the figure, the VR display 100 includes at least one coherent light source 102, at least one pupil replication waveguide 104, and at least one spatial light modulator 106. When these elements of the VR display 100 are described below as being at least partially coupled to each other (and shown as directly coupled), it should be noted that in the context of this specification, the term "coupled" can refer to any direct coupling (i.e., without any elements between them), any indirect coupling (i.e., with one or more elements located in the space between them), partial coupling, complete coupling, and / or any other coupling capable of connecting different elements. Any gaps or spaces between elements may be unfilled (e.g., composed of air) or may be filled with a substance such as an anti-reflective coating.

[0021] Within the same context of this specification, coherent light source 102 refers to any light source capable of outputting any type of light (e.g., ordinary light, data-encoded light, etc.) that is at least partially coherent (e.g., only partially coherent, fully coherent, etc.). Coherent light can refer to a beam of photons having at least partially the same frequency, such as a laser beam output from a laser source. In one embodiment, VR display 100 may include a single coherent light source 102, optionally having the ability to output light in multiple different colors. In another embodiment, VR display 100 may include multiple coherent light sources 102, each capable of outputting light of a different color. In the case of multiple coherent light sources 102, the coherent light sources 102 may be time-division multiplexed, such that light is output from the coherent light sources 102 in a time-division multiplexed manner.

[0022] In one embodiment, the coherent light source 102 may include a point light source that emits light, a concave mirror that reflects the light emitted by the point light source, and a beam splitter that guides the light reflected by the concave mirror.

[0023] A coherent light source 102 is coupled to at least one pupil replication waveguide 104 such that the at least one pupil replication waveguide 104 receives light from the coherent light source 102. For example, the aforementioned beam splitter can guide light reflected by a concave mirror to the pupil replication waveguide 104. In the context of this specification, pupil replication waveguide 104 refers to any waveguide (which may include an optical guide) that includes at least one optical replication element or function. For example, pupil replication waveguide 104 can use at least one optical replication element or function to replicate light received from the coherent light source 102 (e.g., as shown in the image). Figure 1 As indicated by the arrow, the light received from the coherent light source 102 is copied at the output of the pupil replication waveguide 104.

[0024] In one embodiment, at least one pupil replication waveguide 104 may include a diffraction surface grating (or DOE) waveguide, a holographic volume grating (or HOE) waveguide, a partially mirror-based waveguide, and other possible waveguides having at least one optical replication element or function. In another embodiment, at least one pupil replication waveguide 104 may include a pupil replication waveguide with at least one waveguide coupler. For example, at least one pupil replication waveguide 104 may include a pupil replication waveguide with a waveguide in-coupler and a waveguide out-coupler. The waveguide in-coupler may refract light received from the coherent light source 102 so that it propagates through the pupil replication waveguide 104 to the waveguide out-coupler, which may then direct the light to at least one spatial light modulator 106, as described again in more detail later.

[0025] In any case, the pupil replication waveguide 104 can be configured such that, when traveling through the pupil replication waveguide 104, the coherence of the light output by the coherent light source 102 is at least partially maintained. Additionally, the pupil replication waveguide 104 can be configured such that, when traveling through the pupil replication waveguide 104, the polarization of the light output by the coherent light source 102 is at least partially maintained. Furthermore, the pupil replication waveguide 104 can be configured such that the direction of the light input to the pupil replication waveguide 104 (which may be perpendicular to the pupil replication waveguide 104 or at any other angle relative to the pupil replication waveguide 104) is at least partially maintained when output from the pupil replication waveguide 104.

[0026] At least one pupil-replicating waveguide 104 is coupled to at least one spatial light modulator 106 to modulate light. In the context of this specification, spatial light modulator 106 refers to any device or component that at least partially alters the modulation of light in space (i.e., changes the properties of light having a spatial pattern). Thus, spatial light modulator 106 can at least partially apply spatial variation modulation to light transmitted (e.g., output) by pupil-replicating waveguide 104. For example, spatial light modulator 106 can be a phase-only spatial light modulator or a spatial light modulator operating with only a phase pattern. In one embodiment, at least one spatial light modulator 106 can be directly coupled to at least one pupil-replicating waveguide 104 without any space (i.e., gap) between them. In one embodiment, at least one spatial light modulator 106 can be indirectly coupled to at least one pupil-replicating waveguide 104 with space (and / or some other material or component) between them.

[0027] Spatial light modulator 106 may be the display plane of VR display 100. In one embodiment, spatial light modulator 106 may create VR images or videos (from the user's eye perspective) behind spatial light modulator 106. In another embodiment, spatial light modulator 106 may be a reflective spatial light modulator 106. In one embodiment, spatial light modulator 106 is driven using pixel data received from an image source. Alternatively, the receiver of VR display 100 may receive pixel data from a remote source. Of course, in another embodiment, the pixel data may be generated locally relative to VR display 100.

[0028] This configuration of the VR display 100 reduces or eliminates any gap between at least one pupil replication waveguide 104 and at least one spatial light modulator 106. As a result, the cross-sectional thickness of the VR display 100, or particularly the combined cross-sectional thickness of at least one pupil replication waveguide 104 and at least one spatial light modulator 106, can be 2.5 mm. In one embodiment, the cross-sectional thickness of the VR display 100, or particularly the combined cross-sectional thickness of at least one pupil replication waveguide 104 and at least one spatial light modulator 106, can be less than 5 mm. In another embodiment, the cross-sectional thickness of the VR display 100, or particularly the combined cross-sectional thickness of at least one pupil replication waveguide 104 and at least one spatial light modulator 106, can be less than 4 mm. In still other embodiments, such a combined cross-sectional thickness can be less than 3 mm, and so on.

[0029] Furthermore, even when the aforementioned gaps are reduced or eliminated, the quality of the VR images and / or videos displayed by the VR display 100 can still be improved relative to conventional VR displays. In one embodiment, this can be achieved by using a coherent light source 102 with a coherence length greater than that of the spatial light modulator 106, thereby ensuring light interference. Moreover, the above-described configuration of the VR display 100 can support three-dimensional (3D) VR images and / or videos. For example, the spatial light modulator 106 may be able to display 3D images and / or videos behind the plane (virtual) of the spatial light modulator 106.

[0030] Alternatively, the VR display 100 can be configured such that the light is unpolarized. Alternatively, the VR display 100 can be configured such that the light is polarized. Yet another option, the VR display 100 may not need to include a beam splitter. Yet another option, the VR display 100 can be filterless and, for example, can rely on using simulation to determine the propagation path (algorithm) of the phase and amplitude to be used by the spatial light modulator 106, or, for example, can rely on a phase generalization algorithm that takes into account the pupil diameter effect, both of which are described in more detail below.

[0031] It should be noted that although the VR display 100 is described above as including a coherent light source 102, a pupil replication waveguide 104, and a spatial light modulator 106, other embodiments are contemplated in which the VR display includes additional elements. The following description... Figure 2-7 Other possible embodiments (implementations) of the VR display are provided. By way of example only, in one embodiment, the VR display may include at least one polarizer, which may be a linear polarizer (e.g., see at least [link to documentation]). Figure 2 In another embodiment, the VR display may include at least one lens, such as a geometric phase lens (see at least [reference needed]). Figure 3 In yet another embodiment, the VR display may include at least one polarizer or quarter-wave plate used in conjunction with a lens (e.g., see at least [reference needed]). Figure 4 In yet another embodiment, the VR display may include a combination of a linear polarizer, a geometric phase lens, and a quarter-wave plate (e.g., see at least [reference needed]). Figure 5 In a further embodiment, the VR display may include dynamic eye frames (e.g., see at least [reference needed]). Figure 6B ).also, Figure 8 The operation method of an HMD configured according to one or more embodiments described herein is described.

[0032] Now, based on the user's needs, further illustrative information will be provided regarding the various optional architectures and features that can be utilized to implement the aforementioned framework. It should be strongly noted that the following information is for illustrative purposes only and should not be construed as limiting in any way. Any of the following features may be optionally combined, excluding or not excluding the other features described. For example, in one embodiment, VR display 100 may include a receiver that receives pixel data (e.g., representing VR images or VR video) from a remote source over a network for display by VR display 100. The remote source can be any computer system capable of transmitting pixel data to VR display 100 over a network. For example, the remote source can be a server, a video game console, a mobile device (e.g., a user's), or any other computer system, such as those referenced below. Figure 9 As described, the VR display 100 can be connected to a network via wired or wireless means to receive pixel data from a remote source.

[0033] Figure 2 An implementation 200 of a VR display having a coherent light source, a pupil replication waveguide, a spatial light modulator, and a linear polarizer according to one embodiment is shown. It should be noted that the foregoing definitions and / or descriptions can also apply to the following description.

[0034] As shown, at least one pupil replication waveguide 104 is coupled to at least one coherent light source 102 to receive light from it. Furthermore, at least one linear polarizer 108 is coupled between the pupil replication waveguide 104 and at least one spatial light modulator 106. The at least one linear polarizer 108 polarizes the light output from the pupil replication waveguide 104. In one embodiment, the at least one linear polarizer 108 polarizes the light according to the polarization of the at least one spatial light modulator 106. Of course, while a linear polarizer 108 is shown, it should be noted that other types of polarizers (referring to components or devices that polarize light according to some predefined polarization) can be used. In other embodiments described herein, the polarizer may not necessarily be used in conjunction with the spatial light modulator 106, particularly where the light output from the pupil replication waveguide 104 is polarized with the spatial light modulator 106, or where the spatial light modulator 106 can function as intended without any specific light polarization.

[0035] In this embodiment, light is transmitted from at least one coherent light source 102 through at least one pupil replication waveguide 104. The light output from the at least one pupil replication waveguide 104 is then transmitted through a linear polarizer 108 to a spatial light modulator 106 for modulation. The modulated light output from the spatial light modulator 106 is then transmitted for output to the user's eye.

[0036] Figure 3 An implementation 300 of a VR display having a coherent light source, a pupil replication waveguide, a spatial light modulator, and a geometric phase lens according to one embodiment is shown. It should be noted that the foregoing definitions and / or descriptions can also apply to the following description.

[0037] As shown, at least one pupil replication waveguide 104 is coupled to at least one coherent light source 102 to receive light from it. At least one pupil replication waveguide 104 is coupled to at least one spatial light modulator 106 to modulate the light. At least one spatial light modulator 106 is also coupled to at least one geometric phase lens 110. In the illustrated embodiment, at least one spatial light modulator 106 is coupled to a first surface of at least one pupil replication waveguide 104 to modulate the light. Furthermore, regarding this embodiment, at least one geometric phase lens 110 is coupled to a second surface of at least one pupil replication waveguide 104 opposite to the first surface of at least one pupil replication waveguide 104.

[0038] At least one geometrical phase lens 110 can be an element of the VR display viewed by the eyes of a user of the VR display (i.e., it can be an eyepiece of the VR display). Specifically, in this embodiment, modulated light can be transmitted from at least one spatial light modulator 106 to at least one geometrical phase lens 110 for output to the eyes of the user of the VR display. In one embodiment, at least one geometrical phase lens 110 can polarize the light. In another embodiment, at least one geometrical phase lens 110 can create a virtual image that appears farther away from the user.

[0039] In one embodiment, the geometric phase lens 110 may be a Pancharatnam-Berry phase lens. In another embodiment, the geometric phase lens 110 may be a polarization-dependent liquid crystal lens, which serves as a positive lens for polarizing a specific input beam. Of course, while the geometric phase lens 110 is shown, it should be noted that other types of lenses may be used. In other embodiments described herein, the lens may not be used in conjunction with the spatial light modulator 106.

[0040] In this embodiment, light is transmitted by at least one coherent light source 102 through at least one pupil replication waveguide 104. The light output from the at least one pupil replication waveguide 104 is then transmitted to a spatial light modulator 106 for modulation. The modulated light output from the spatial light modulator 106 is then transmitted through a geometric phase lens 110 for output to the user's eye.

[0041] Figure 4 An implementation 400 of a VR display having a coherent light source, a pupil replication waveguide, a spatial light modulator, a geometric phase lens, and a quarter-wave plate according to one embodiment is shown. It should be noted that the foregoing definitions and / or descriptions can also apply to the following description.

[0042] and Figure 3 Similar to implementation 300, at least one pupil replica waveguide 104 is coupled to at least one coherent light source 102 to receive light from it. At least one pupil replica waveguide 104 is coupled to at least one spatial light modulator 106 to modulate the light. At least one spatial light modulator 106 is further (indirectly) coupled to at least one geometrical phase lens 110. In this embodiment, additionally, at least one quarter-wave plate 112 is located (coupled) between the spatial light modulator 106 and the geometrical phase lens 110. The quarter-wave plate 112 is transmitted to the geometrical phase lens 110 before the light is received from the spatial light modulator 106 to shift the phase of the light.

[0043] In one embodiment, a quarter-wave plate 112 is coupled between the spatial light modulator 106 and the geometrical phase lens 110, such that linearly polarized diffracted light can be effectively converted into right-handed circularly polarized light (as required by the geometrical phase lens 110), and then the light is again left-handedly circularly polarized by the geometrical phase lens 110. It should be noted that although a quarter-wave plate 112 is shown, a polarizer can be used in other embodiments. Of course, in yet another embodiment, it is not necessary to combine a polarizer or the geometrical phase lens 110 with the geometrical phase lens 110.

[0044] In this embodiment, light is transmitted from at least one coherent light source 102 through at least one pupil replication waveguide 104. The light output from the at least one pupil replication waveguide 104 is then transmitted through a spatial light modulator 106 for modulation. The modulated light output from the spatial light modulator 106 is then transmitted through a quarter-wave plate 112 and then through a geometric phase lens 110 to be output to the user's eye.

[0045] Figure 5 An implementation 500 of a VR display having a coherent light source, a pupil replication waveguide, a spatial light modulator, a linear polarizer, a geometric phase lens, and a quarter-wave plate according to one embodiment is shown. It should be noted that the foregoing definitions and / or descriptions can also apply to the following description.

[0046] As shown, at least one pupil replica waveguide 104 is coupled to at least one coherent light source 102 to receive light from it. Furthermore, at least one spatial light modulator 106 is indirectly coupled to a first surface of at least one pupil replica waveguide 104 to modulate the light. A linear polarizer 108 is located (coupled) between at least one spatial light modulator 106 and at least one pupil replica waveguide 104.

[0047] At least one spatial light modulator 106 is further (indirectly) coupled to at least one geometric phase lens 110. In particular, at least one quarter-wave plate 112 is located (coupled) between the spatial light modulator 106 and the geometric phase lens 110. The quarter-wave plate 112 causes a phase shift in the light received from the spatial light modulator 106 and input to the geometric phase lens 110.

[0048] In this embodiment, light is transmitted from at least one coherent light source 102 through at least one pupil replication waveguide 104. The light output from the at least one pupil replication waveguide 104 is then transmitted through a linear polarizer 108 to a spatial light modulator 106 for modulation. The modulated light output from the spatial light modulator 106 is then transmitted through a quarter-wave plate 112 and subsequently through a geometric phase lens 110 to be output to the user's eye 114. As shown, the target plane is positioned behind the spatial light modulator 106 relative to the user's eye 114.

[0049] This embodiment illustrates coherent and collimated light with wavelength λ being input-coupled and passing through a thickness of t. w The pupil replicates waveguide 104, and then it converges at an angle θ. c Output coupling. Afterwards, light travels through a region with a thickness of t. p The light is polarized at linear polarizer 108 and modulated at spatial light modulator 106, where the pixel spacing is p. s The number of pixels is N x ×N y Width is w s And the height is h s Using the diffraction angle θ s The modulated light passes through the pupil again to replicate waveguide 104, and then through a thickness of t. q A quarter-wave plate 112, then a plate of thickness t L And the focal length is f L The geometric phase lens refracts the light at 110°. At this point, the central ray will utilize the horizontal field of view (FOV) θ. v At eye distance d e Create a viewpoint at a distance, and its eye border w e From the diffraction angle θ s Decision. e θ v and w e The equations shown in Table 1 can be used for calculation.

[0050] Table 1

[0051]

[0052] as well as

[0053]

[0054] The thickness t of the linear polarizer 108 p The thickness t of the quarter-wave plate 112 q The thickness t of the geometric phase lens 110 L Ignored. Spatial light modulator 106 generates light at a distance d behind spatial light modulator 106. h A three-dimensional (3D) image. Then, the geometric phase lens 110 magnifies the 3D image and moves it back to a perceptible distance d. i .

[0055] Figure 6AAn implementation 600 of a VR display with a coherent light source, a pupil replica waveguide, and a spatial light modulator according to one embodiment is shown, wherein the VR display exhibits a high diffraction order. It should be noted that the foregoing definitions and / or descriptions can also apply to the following description.

[0056] As shown in the figure, implementing 600 includes... Figure 5 The same configuration is used in implementation 500. However, as mentioned above, implementation 600 demonstrates the high diffraction orders (denoted as "HDOs") exhibited by this configuration. In particular, the periodic structure of the 106 pixels of the spatial light modulator can produce repeating high diffraction orders, each displaying the same holographic image, and they are spaced w apart due to the eyepiece. h They converge along the pupillary plane. If w h Smaller than the pupil diameter w p Then HDOs (e.g., higher-order gradient descent, or HOGD) can be considered during phase calculation, as described in more detail below.

[0057] In one embodiment, the eye socket w e and diffraction order repeat interval w h Mismatch can be corrected by selecting pixel spacing p s and focal length f L To match the eye socket w e and diffraction order repeat interval w h This can be addressed by achieving a continuous eye socket. However, in this embodiment, the intensity of high diffraction orders may be lower compared to the central lobe.

[0058] In another embodiment, the viewpoint can be modified on a plane by changing the incident angle of the light received by the pupil replication waveguide 104. This embodiment relates to... Figure 6B Show and describe.

[0059] Figure 6B An implementation 650 of a VR display having a coherent light source, a pupil replication waveguide, and a spatial light modulator according to one embodiment is shown, wherein the incident angle of light received by at least one pupil replication waveguide is adjustable. It should be noted that the foregoing definitions and / or descriptions can also apply to the following description.

[0060] As shown in the figure, implementation 650 has the same configuration as implementation 600, except that the incident angle of the light received by the pupil replication waveguide 104 is adjustable. In particular, since the pupil replication waveguide 104 is designed to reproduce a light field with a certain range of incident angles (θi), the direction of illumination of the entire spatial light modulator 106 can be controlled by a small input beam direction, resulting in a dynamic eye frame.

[0061] In one embodiment, the angle of incidence of light can be adjusted based on the current gaze direction of a user using the VR display. In one embodiment, the VR display or a device including a VR display (e.g., an HMD) can include a gaze tracking device to determine the user's current gaze direction. Using a gaze tracking device (e.g., a camera, sensor, etc.), the device can follow the user's gaze and move around the center lobe by simply changing the direction of the input beam provided by the coherent light source 102.

[0062] In one embodiment, the VR display may include at least one beam control unit (not shown) coupled to the pupil replication waveguide 104 to adjust the angle of incidence of light input to the pupil replication waveguide 104. The beam control unit may be any device capable of receiving light from the coherent light source 102 and adjusting the angle of incidence of the light before it is input to the pupil replication waveguide 104. For example, the beam control unit may include an adjustable mirror (e.g., a folding mirror) whose position, angle, etc., are adjustable to change the angle of incidence of light received from the coherent light source 102 before it is input to the pupil replication waveguide 104. As another example, the beam control unit may be a lens shifter. In one embodiment, the beam control unit may be coupled to an input coupler of the pupil replication waveguide 104, which will be referenced below. Figure 7 To describe it in more detail.

[0063] Since VR displays are not Maxwellian view displays but rather create specific eyeboxes, it is not necessarily required that the VR display always precisely redirect the viewpoint to the pupil center. Furthermore, the required moving parts (i.e., the lens beam steering unit) have minimal impact on the VR display's shape factor. Additionally, compared to holographic optical element image combiners, the pupil replication waveguide 104 has less angular selectivity and can cover a wider input k-vector. Maximum incident light angle θ i,max The refractive index and thickness t of waveguide 104 are replicated by the pupil. w The decision is made, and it is usually sufficient to cover the user's eye movements.

[0064] In some embodiments, there may be design trade-offs for VR displays. For example, when the spatial light modulator 106 is sized w s When it increases, the field of view θ v It gets larger, while the spatial light modulator has a 106-pixel pitch p s When it shrinks, the eye box w e It increases. Therefore, θ v and w e Both are limited by the characteristics of the spatial light modulator 106. For wearable shape factors, the interpupillary distance d... e A target size of less than 20 mm is possible. A shorter eyepiece focal length (f) is also possible. L and a larger convergence angle θc This is beneficial for wearable shape factors. While short interocular distances may be crucial for wearable shape factors and large fields of view, it can result in smaller eyeboxes and higher-order irises closer to the center. In turn, the pupil can be used as a Fourier filter with a dynamic aperture based on scene brightness.

[0065] If the high diffraction order interval w h If the aperture is smaller than the minimum pupil diameter (e.g., 2 mm), a high diffraction order can always be observed by the user. In this case, an algorithm can be used to calculate the phase map that will be used by the spatial light modulator 106 to produce the desired output. Initially, the complex wavefront at the spatial light modulator 106 is simulated using a phase map and unit amplitude. This is a Fourier transform (FT) to move to the frequency domain (called FFT amplitude and FFT phase). The frequency domain is repeated to produce a higher-order copy (i.e., a copy of the repeated FFT amplitude and FFT phase). Propagation is then performed by multiplying the wavefront by a 2D sinc amplitude and an angular spectral method (ASM) phase delay, thus resulting in propagated FFT amplitude and propagated FFT phase, where the 2D sinc amplitude represents the finite size of the pixels of the spatial light modulator 106. The output of the algorithm is calculated by transforming the propagated FFT amplitude and propagated FFT phase back from the frequency domain to produce the propagated amplitude and propagated phase to be used by the spatial light modulator 106.

[0066] The combined use of repetitive frequency domain and 2D sinc amplitude produces an accurate simulation of higher-order propagation channels. In this way, the spatial light modulator 106 can be optimized using propagation amplitude and propagation phase, enabling it to produce the desired output. Using this algorithm, image quality can be improved when filters are absent in the VR display. Eliminating filters, in turn, allows for a more compact VR display without sacrificing image quality. Furthermore, utilizing higher-order light increases the optical extension of the VR display without adding additional hardware components.

[0067] If w h Within the pupil diameter range, a user can perceive high diffraction levels based on the pupil diameter. On the other hand, if the high diffraction levels are sufficiently separated, the user's eye may not see them (i.e., the user's pupil will act as a "natural" filter and will not see the high diffraction levels), thus the high diffraction levels and pupil diameter can be ignored. Since the spatial light modulator 106 itself may not satisfy this condition, a phase generation algorithm that specifically considers the pupil diameter effect can be used, as described below. In one embodiment, this condition can be relaxed by utilizing a spatial light modulator 106 with smaller pixel pitch or scene brightness control.

[0068] In one embodiment, the processor (as follows) Figure 9The phase map (shown) can be included in a VR display or a device including such a VR display (e.g., an HMD) for synthesizing a phase map of the input image. Specifically, the phase map can be synthesized such that the spacing of high diffraction orders is larger than the pupil diameter of the user using the VR display.

[0069] In an exemplary embodiment, the phase map can be synthesized using the HOGD and HOGD-CITL (Camera-in-the-Loop Holography) algorithms. These algorithms model the propagation of high diffraction orders to achieve good image quality without optically filtering out higher orders. This high diffraction order propagation, detailed in the equations shown in Table 2, can provide good image quality when the pupil collects light from multiple diffraction orders. In this equation, φ is the phase map of the spatial light modulator 106, p s α is the pixel pitch of the spatial light modulator 106, α is the set of orders to be optimized, λ is the wavelength, z is the distance between the spatial light modulator 106 and the target plane, and M... p It is a pupil mask.

[0070] Table 2

[0071]

[0072]

[0073]

[0074]

[0075] In these equations, u is the propagation wavefront, U is the frequency representation of the wavefront with a high-order spatial light modulator 106, H is the ASM kernel in the frequency domain, and A is the ASM kernel attenuated due to the pixel pitch of the SLM and the pupil aperture. The pupil aperture is represented by the masking term M. p To explain, it extends the HOGD algorithm to an algorithm we call Pupil-HOGD. In this algorithm, the pupil mask M p This allows for optimization of the phase diagram while considering wavefront filtering performed on the pupil. Here, pupil filtering is modeled for the case de = fL, where the diameter is w. p The pupil acts as a diameter of w p / (λf L A circular filter in the Fourier domain of ). Therefore, the pupil aperture can be simply determined by the mask M. p The model is constructed such that the frequency within the circular filter is 1, and otherwise 0.

[0076] In addition to modeling the pupil aperture, the HOGD and HOGD-CITL algorithms are extended to generate phase maps of RGBD content using masked multiplane loss, as outlined in the equations shown in Table 3. In these equations, a target D(x,y) is the desired target amplitude, and D(x,y) is the desired target depth map. Using a mask m... (·) The desired scene is decomposed on the J target plane, and these masks are obtained by quantizing the target depth map to the nearest depth z in the multi-plane decomposition. (·) Generated.

[0077] Table 3

[0078]

[0079]

[0080] Minimizing this objective generates a phase map to display the desired 3D content. For the multi-plane HOGD algorithm, this objective can be directly optimized using the Adam optimizer in PyTorch. The HOGD-CITL algorithm enhances this optimization by pairing the gradients propagated in the simulation with the captured VR display output. These algorithms are available on Nvidia. TM It runs on an RTX 3090 graphics card and performs alternating optimization steps for the content on each target plane to limit memory usage.

[0081] It should be noted that gaze tracking devices, such as infrared gaze trackers, can be used to capture and measure a user's pupil diameter. In reality, due to pupillary reflection, any user's pupil diameter will change with scene intensity (e.g., if the target scene is much brighter than the previous frame, pupil size will decrease in the new frame, so pullil-HOGD will not work). However, the user's pupil diameter does not necessarily need to be precisely measured for every image; instead, the user's pupillary reflection can be calibrated based on scene intensity. For example, a model can be calibrated once per user and integrated into pullil-HOGD based on scene brightness for an optimized viewing experience.

[0082] Figure 7 An implementation 700 of a pupil replication waveguide according to one embodiment is shown. The pupil replication waveguide may include the pupil replication waveguide 104 described in the above embodiments. It should be noted that the foregoing definitions and / or descriptions may also apply to the following description.

[0083] As shown in the figure, the input coupler 116 guides light (from the coherent light source 102) into the pupil replication waveguide 104, and the output coupler 118 guides the light out of the pupil replication waveguide 104. As shown in the figure, the pupil replication waveguide 104 replicates the input coupled k-vector (light) along the output coupler.

[0084] Figure 8 An operation method 800 of an HMD according to an embodiment is illustrated. In one embodiment, method 800 can use an implementation 100 of a VR display (in... Figure 1 The HMD is implemented as described in the text, such that the HMD includes at least one coherent light source, at least one pupil replica waveguide coupled to at least one coherent light source, and at least one spatial light modulator coupled to at least one pupil replica waveguide.

[0085] In this embodiment, the HMD can be an augmented reality (AR) display or a mixed reality (MR) display. Therefore, the HMD is not necessarily limited to VR displays, but can be used in conjunction with… Figure 1 Similar to the VR display 100, the HMD may also include a camera for capturing real-time images to create AR or MR images and / or videos. Of course, any of the embodiments described above with respect to the various figures can be used in the context of an HMD performing this method 800.

[0086] In operation 802, light from at least one coherent light source is received by at least one pupil replica waveguide. In operation 804, light is transmitted from at least one pupil replica waveguide to at least one spatial light modulator. In operation 806, light is modulated using at least one spatial light modulator.

[0087] In one embodiment, the HMD may further include at least one polarizer (e.g., a linear polarizer) coupled to at least one pupil replica waveguide. The at least one polarizer can polarize light according to the polarization of the spatial light modulator. In this embodiment, method 800 may include polarizing light output from at least one pupil replica waveguide using at least one polarizer.

[0088] In one embodiment, the HMD may further include at least one lens (e.g., a geometric phase lens) coupled to at least one spatial light modulator. In this embodiment, method 800 may include using the lens to polarize light. As a further option, the HMD may also include at least one quarter-wave plate coupled to at least one lens. In this embodiment, method 800 may include using at least one quarter-wave plate to shift the phase of light input to the lens.

[0089] In one embodiment, the HMD can be configured such that the angle of incidence of light received by at least one pupil replication waveguide is adjustable. In this embodiment, method 800 may then include adjusting the angle of incidence of light based on the current gaze direction of a user using the head-mounted display. In one embodiment, the HMD may include a gaze tracking device for determining the user's current gaze direction. In another embodiment, the HMD may include at least one beam control unit coupled to at least one pupil replication waveguide (e.g., an input coupler coupled to at least one pupil replication waveguide) to adjust the angle of incidence of light using the at least one beam control unit.

[0090] In one embodiment, the HMD may further include a processor for synthesizing a phase map of the input image. In this embodiment, method 800 may include using the processor to synthesize a phase map of the input image. In one embodiment, the phase map may be synthesized such that the spacing of high diffraction orders is larger than the pupil diameter of a user using the head-mounted display.

[0091] In yet another embodiment, the HMD may include a receiver. In this further embodiment, method 800 may include receiving pixel data from a remote source via a network by the receiver for display via the HMD. The HMD may perform method 800 to output the pixel data as a VR image or video for a user to view. The remote source may be as follows: Figure 9 An exemplary computing system is described.

[0092] Figure 9 An exemplary computing system 900 according to an embodiment is shown. Figure 8 Method 800 HMD (not shown) or Figure 1 The VR display 100, or any other embodiment described above (also not shown), can communicate with the system 900 to receive output from and provide input to the system 900. By way of example only, the HMD / VR display can receive virtual images in the form of pixel data from the system 900. The HMD / VR display and the system 900 can be located in the same environment or remotely (e.g., the system 900 can be located in the cloud). It should be noted that the HMD / VR display can communicate with the system 900 via a wired or wireless network connection (e.g., WiFi, cellular network, etc.). Alternatively, one or more components shown in the system 900 can be implemented within the HMD / VR display.

[0093] As shown in the figure, system 900 includes at least one central processing unit 901 connected to a communication bus 902. System 900 also includes main memory 904 (e.g., random access memory (RAM)). System 900 also includes a graphics processor 906 and a display 908.

[0094] System 900 may also include secondary storage 910. Secondary storage 910 includes, for example, hard disk drives and / or removable storage drives, representing floppy disk drives, tape drives, optical disk drives, flash memory drives, or other flash memory, etc. The removable storage drives read from and / or write to the removable storage unit in a well-known manner.

[0095] For this purpose, computer programs or computer control logic algorithms can be stored in main memory 904, secondary storage 910, and / or any other memory. When executed, such a computer program enables system 900 to perform various functions, including, for example, calibration of HMD 102, formation of live video, and shading of pixels on display 104, as described above. The computer program can also, when executed, integrate live video with a virtual environment to provide users with an improved virtual reality, mixed reality, or augmented reality experience. Memory 904, storage 910, and / or any other storage are possible examples of non-transitory computer-readable media.

[0096] System 900 may also include one or more communication modules 912. The communication module 912 can be used to facilitate communication between system 900 and one or more networks and / or with one or more devices (e.g., game consoles, personal computers, servers, etc.) via various possible standard or proprietary wired or wireless communication protocols (e.g., via Bluetooth, Near Field Communication (NFC), cellular communication, etc.).

[0097] As also shown in the figure, system 900 may include one or more input devices 914. Input devices 914 may be wired or wireless input devices. In various embodiments, each input device 914 may include a keyboard, touchpad, touchscreen, game controller, remote control, or any other device that can be used by a user to provide input to system 900.

[0098] While various embodiments have been described above, it should be understood that they are presented by way of example only and not as limitations. Therefore, the breadth and scope of preferred embodiments should not be limited by any of the exemplary embodiments described above, but should be defined solely by the following claims and their equivalents.

[0099] At least one embodiment of this disclosure may be described in accordance with the following terms:

[0100] 1. An apparatus comprising:

[0101] Virtual reality (VR) displays, including:

[0102] At least one coherent light source,

[0103] At least one holographic waveguide coupled to at least one coherent light source to receive light from it, and

[0104] At least one spatial light modulator coupled to at least one holographic waveguide to modulate light.

[0105] 2. The device as described in Clause 1, wherein the VR display further includes at least one magnifying glass.

[0106] 3. The apparatus as described in Clause 2, wherein the at least one magnifying lens comprises a Fresnel lens.

[0107] 4. The apparatus as described in Clause 2, wherein the at least one magnifying lens comprises a holographic lens.

[0108] 5. The apparatus as described in Clause 2, wherein the VR display further includes at least one polarizing element coupled between the at least one magnifying lens and the at least one holographic waveguide.

[0109] 6. The apparatus as described in Clause 2, wherein the at least one magnifying glass and the at least one holographic waveguide are directly coupled without any space between them.

[0110] 7. The apparatus as described in Clause 1, wherein the VR display further comprises at least one quarter-wave element coupled between the at least one holographic waveguide and the at least one spatial light modulator.

[0111] 8. The apparatus as described in Clause 1, wherein the at least one holographic waveguide comprises a backlit holographic waveguide.

[0112] 9. The apparatus as described in Clause 1, wherein the at least one holographic waveguide comprises a holographic waveguide having at least one waveguide coupler.

[0113] 10. The apparatus as described in Clause 1, wherein the at least one holographic waveguide comprises a holographic waveguide including a waveguide input coupler and a waveguide output coupler.

[0114] 11. The apparatus as described in Clause 1, wherein the at least one spatial light modulator is directly coupled to the at least one holographic waveguide, with no space between them.

[0115] 12. The apparatus as described in Clause 1, wherein the apparatus is configured such that the light is not polarized.

[0116] 13. The apparatus as described in Clause 1, wherein the apparatus is configured such that the light is polarized.

[0117] 14. The apparatus as described in Clause 1, wherein the VR display does not include a beam splitter.

[0118] 15. The apparatus as described in Clause 1, wherein the combined cross-sectional thickness of the at least one holographic waveguide and the at least one spatial light modulator is less than 10 mm.

[0119] 16. The apparatus as described in Clause 1, wherein the combined cross-sectional thickness of the at least one holographic waveguide and the at least one spatial light modulator is less than 7 mm.

[0120] 17. The apparatus of Clause 1, wherein the VR display further includes a receiver for receiving pixel data from a remote source via a network for display via the VR display.

[0121] 18. The apparatus of Clause 1, wherein the coherent light source comprises a point source emitting the light, a concave mirror reflecting the light emitted by the point source, and a beam splitter guiding the light reflected by the concave mirror to a holographic waveguide.

[0122] 19. The apparatus as described in Clause 1, wherein the VR display is filterless.

[0123] 20. The apparatus as described in Clause 19, wherein the phase and amplitude to be used by the spatial light modulator are determined based on simulations of multiple higher-order copies of the phase and amplitude used in the frequency domain of the spatial light modulator.

[0124] 21. The apparatus as described in Clause 19, wherein the phase and amplitude to be used by the spatial light modulator are determined based on a simulation using two-dimensional (2D) sinc amplitudes that describe the finite pixel pitch of the spatial light modulator.

[0125] 22. The apparatus as described in Clause 19, wherein the phase and amplitude to be used by the spatial light modulator are determined based on a simulation in which:

[0126] The complex wavefront at the spatial light modulator is simulated using input phase and input amplitude.

[0127] The input phase and input amplitude are transformed to be moved to the frequency domain of the spatial light modulator.

[0128] Repeat the frequency domain to generate multiple higher-order copies of the input phase and input amplitude in the frequency domain.

[0129] The plurality of higher-order copies of the input phase and input amplitude are multiplied by an angular spectral method (ASM) phase delay and a two-dimensional (2D) sinc amplitude to result in the propagation amplitude and propagation phase in the frequency domain, wherein the two-dimensional (2D) sinc amplitude describes the finite pixel pitch of the spatial light modulator, and

[0130] The propagation amplitude and propagation phase in the frequency domain are converted back from the frequency domain to generate the phase and amplitude that the spatial light modulator will use.

[0131] 23. The apparatus as described in Clause 1, further comprising:

[0132] A geometric phase lens coupled to the first side of the holographic waveguide, and

[0133] A quarter-wave plate is coupled to a second side of the holographic waveguide together with the spatial light modulator, wherein the quarter-wave plate is coupled between the holographic waveguide and the spatial light modulator.

[0134] 24. A method comprising:

[0135] In a head-mounted display, there is at least one coherent light source, at least one holographic waveguide coupled to the at least one coherent light source, and at least one spatial light modulator coupled to the at least one holographic waveguide, such that the combined cross-sectional thickness of the at least one holographic waveguide and the at least one spatial light modulator is less than 10 mm.

[0136] Light from the at least one coherent light source is received by at least one holographic waveguide;

[0137] Transmitting light from the at least one holographic waveguide to at least one spatial light modulator.

[0138] Instrument; and

[0139] The light is modulated using the at least one spatial light modulator.

[0140] 25. The method of claim 24, wherein the head-mounted display further comprises at least one magnifying lens, and the method further comprises:

[0141] The modulated light is transmitted through the at least one magnifying glass.

[0142] 26. The method of claim 25, wherein the at least one magnifying glass comprises a Fresnel lens.

[0143] 27. The method of claim 25, wherein the at least one magnifying glass comprises a holographic lens.

[0144] 28. The method of claim 25, wherein the head-mounted display further comprises at least one polarizing element coupled between the at least one magnifying lens and the at least one holographic waveguide, and the method further comprises:

[0145] The modulated light is polarized using the at least one polarizing element.

[0146] The modulated light of this polarization is transmitted through the at least one magnifying glass.

[0147] 29. The method as described in Clause 25, wherein the at least one magnifying glass and the at least one holographic waveguide are directly coupled without any space between them.

[0148] 30. The method of claim 24, wherein the head-mounted display further comprises at least one quarter-wave element coupled between the at least one holographic waveguide and the at least one spatial light modulator, and the method further comprises:

[0149] The light from the at least one holographic waveguide is modified using the at least one quarter-wave element.

[0150] The modified light is received by the at least one spatial light modulator.

[0151] 31. The method as described in Clause 24, wherein the at least one holographic waveguide comprises a backlit holographic waveguide.

[0152] 32. The method as described in Clause 24, wherein the at least one holographic waveguide comprises a holographic waveguide having at least one waveguide coupler.

[0153] 33. The method of Clause 24, wherein the at least one holographic waveguide comprises a holographic waveguide including a waveguide input coupler and a waveguide output coupler.

[0154] 34. The method as described in Clause 24, wherein the at least one spatial light modulator is directly coupled to the at least one holographic waveguide, with no space between them.

[0155] 35. The method as described in Clause 24, wherein the light is not polarized.

[0156] 36. The method described in Clause 24 further includes:

[0157] The light is polarized.

[0158] 37. The method of Clause 24, wherein the head-mounted display does not include a beam splitter.

[0159] 38. The method as described in Clause 24, wherein the combined cross-sectional thickness of the at least one holographic waveguide and the at least one spatial light modulator is less than 7 mm.

[0160] 39. The method of claim 24, wherein the head-mounted display further includes a receiver, and the method further includes:

[0161] The receiver receives pixel data from a remote source via a network for display on the head-mounted display.

[0162] 40. The method of Clause 24, wherein the coherent light source comprises a point source emitting the light, a concave mirror reflecting the light emitted by the point source, and a beam splitter guiding the light reflected by the concave mirror to the holographic waveguide.

[0163] 41. The method as described in Clause 24, wherein the phase and amplitude to be used by the spatial light modulator are determined based on simulations of multiple higher-order copies of the phase and amplitude in the frequency domain using the spatial light modulator.

[0164] 42. The method as described in Clause 24, wherein the phase and amplitude to be used by the spatial light modulator are determined based on a simulation using two-dimensional (2D) sinc amplitudes that describe the finite pixel pitch of the spatial light modulator.

[0165] 43. The method as described in Clause 24, wherein the phase and amplitude to be used by the spatial light modulator are determined based on a simulation in which:

[0166] The complex wavefront at the spatial light modulator is simulated using input phase and input amplitude.

[0167] The input phase and input amplitude are transformed to be moved to the frequency domain of the spatial light modulator.

[0168] Repeat the frequency domain to generate multiple higher-order copies of the input phase and input amplitude in that frequency domain.

[0169] Multiple higher-order copies of the input phase and input amplitude are multiplied by an angular spectral method (ASM) phase delay and a two-dimensional (2D) sinc amplitude to result in the propagation amplitude and propagation phase in the frequency domain, wherein the two-dimensional (2D) sinc amplitude describes the finite pixel pitch of the spatial light modulator, and

[0170] The propagation amplitude and propagation phase in the frequency domain are converted back from the frequency domain to produce the phase and amplitude that the spatial light modulator will use.

Claims

1. An apparatus comprising: Virtual reality (VR) displays, including: At least one coherent light source, At least one pupil replica waveguide coupled to the at least one coherent light source to receive light from it, and At least one spatial light modulator coupled to the at least one pupil replica waveguide to modulate light; and A processor for synthesizing a phase map of an input image, wherein the phase map is synthesized such that the spacing of high diffraction orders is greater than the pupil diameter of a user using the VR display.

2. The apparatus of claim 1, wherein the VR display further comprises: At least one polarizer coupled to the at least one pupil replica waveguide to polarize the light output from the at least one pupil replica waveguide.

3. The apparatus of claim 2, wherein the at least one polarizer is a linear polarizer.

4. The apparatus of claim 2, wherein the at least one polarizer polarizes the light according to the polarization of the spatial light modulator.

5. The apparatus of claim 1, wherein the VR display further comprises: At least one lens coupled to the at least one spatial light modulator.

6. The apparatus of claim 5, wherein the at least one lens is a geometric phase lens.

7. The apparatus of claim 5, wherein the VR display further comprises: At least one polarizer coupled to the at least one lens to polarize light input to the lens.

8. The apparatus of claim 5, wherein the VR display further comprises: At least one quarter-wave plate coupled to the at least one lens.

9. The apparatus of claim 1, wherein the incident angle of the light received by the at least one pupil replica waveguide is adjustable.

10. The apparatus of claim 9, wherein the angle of incidence of the light is adjusted according to the current gaze direction of the user using the VR display.

11. The apparatus of claim 10, further comprising: A gaze tracking device used to determine the user's current gaze direction.

12. The apparatus of claim 9, wherein the VR display further comprises: At least one beam control unit coupled to the at least one pupil replication waveguide, wherein the at least one beam control unit is configured to adjust the incident angle of the light.

13. The apparatus of claim 12, wherein the at least one beam control unit is coupled to the input coupler of the at least one pupil replication waveguide.

14. A method comprising: The head-mounted display includes at least one coherent light source, at least one pupil replica waveguide coupled to the at least one coherent light source to receive light therefrom, at least one spatial light modulator coupled to the at least one pupil replica waveguide to modulate light, and a processor for synthesizing a phase map of the input image. The at least one pupil-replicating waveguide receives light from the at least one coherent light source; Light from the at least one pupil replication waveguide is transmitted to the at least one spatial light modulator; The processor is used to synthesize a phase map of the input image, wherein the phase map is synthesized such that the interval of the high diffraction order is greater than the pupil diameter of the user using the head-mounted display. The light is modulated using the at least one spatial light modulator.

15. The method of claim 14, wherein the head-mounted display further comprises at least one polarizer coupled to the at least one pupil replica waveguide, and the method further comprises: The light output from the at least one pupil replica waveguide is polarized using the at least one polarizer.

16. The method of claim 15, wherein the at least one polarizer is a linear polarizer.

17. The method of claim 15, wherein the at least one polarizer polarizes the light according to the polarization of the spatial light modulator.

18. The method of claim 14, wherein the head-mounted display further comprises at least one lens coupled to the at least one spatial light modulator.

19. The method of claim 18, wherein the at least one lens is a geometric phase lens, and the method further comprises: The light is polarized using the geometric phase lens.

20. The method of claim 18, wherein the head-mounted display further comprises at least one polarizer coupled to the at least one lens, and the method further comprises: The light input to the lens is polarized using the at least one polarizer.

21. The method of claim 18, wherein the head-mounted display further comprises at least one quarter-wave plate coupled to the at least one lens, and the method further comprises: The phase of the light is shifted using the at least one quarter-wave plate.

22. The method of claim 14, wherein the head-mounted display is configured such that the angle of incidence of the light received by the at least one pupil replica waveguide is adjustable.

23. The method of claim 22, wherein the method further comprises: The angle of incidence of the light is adjusted according to the current gaze direction of the user using the head-mounted display.

24. The method of claim 23, wherein the head-mounted display further includes a gaze tracking device for determining the user's current gaze direction.

25. The method of claim 23, wherein the head-mounted display further comprises at least one beam control unit coupled to the at least one pupil replication waveguide, wherein the at least one beam control unit is used to adjust the angle of incidence of the light.

26. The method of claim 25, wherein the at least one beam control unit is coupled to the input coupler of the at least one pupil replica waveguide.

Images