Eye-matched occlusion-compatible optical see-through head-mounted display

Abstract

A compact occlusion-compatible optical see-through head-mounted display (OCOST-HMD) is described that has a dual winding path and is capable of rendering pixel-by-pixel mutual occlusion and correct see-through viewing projection or pupil-aligned viewing between virtual and real views. The example device includes a polarizer, a polarizing beam splitter, an objective lens, a spatial light modulator (SLM), an eyepiece, a quarter-wave plate, and a reflective optical element configured to reflect light incident thereon in a first direction and transition light received from a microdisplay incident thereon from a second direction.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims priority to provisional application bearing Docket No. 62 / 900,204, entitled "Pupil Matched Occlusion-Capable Optical See-Through Head-Mounted Display," filed September 13, 2019. The entire contents of the above provisional application are incorporated by reference as part of the disclosure herein.

[0002] The technology herein relates generally to head mounted displays, and more particularly to head mounted displays with a wide field of view. [Background technology]

[0003] See-through head-mounted displays (HMDs) are one of the key viable technologies for merging digital information and physical scenes in augmented reality systems. While video see-through and optical see-through displays both have their unique advantages, optical see-through HMDs (OST-HMDs) tend to be preferred in terms of real-world resolution, parallax, field of view (FOV), and image latency. Conventional optical see-through head-mounted displays typically rely on optical combiners, such as beam splitters or diffraction gratings, to uniformly combine real-world light with that of virtual objects. Aside from the added cost, weight, alignment considerations, and increased footprint of previous systems, the FOV of conventional systems is limited by the see-through objective lens optics, which is typically narrow. Summary of the Invention [Means for solving the problem]

[0004] The techniques disclosed herein can be implemented in various embodiments to achieve an optical architecture for a compact, occlusion-aware OCOST-HMD design, including, among other features and advantages, a wide FOV that is not limited by objective lens optics. The disclosed methods, systems, and devices utilize a dual winding path to result in an OCOST-HMD that is capable of rendering per-pixel mutual occlusion, correct see-through viewing projection or pupil-aligned viewing between virtual and real views, and a very wide see-through FOV.

[0005] One aspect of the disclosed embodiments relates to an occlusion-enabled optical see-through head-mounted display (OCOST-HMD) including a polarizing element configured to receive light from a real scene and generate polarized light at its output, a polarizing beam splitter (PBS), an objective lens, a spatial light modulator (SLM), an eyepiece lens, a quarter-wave plate (QWP), and a reflective optical element configured to reflect substantially all or a portion of light incident thereon in a first direction and transition substantially all or a portion of light received from a microdisplay incident thereon from a second direction. The SLM and objective lens form a first double-pass configuration that allows at least a portion of the light passing through the objective lens to be reflected from the SLM and propagate again through the objective lens. The eyepiece lens and the reflective optical element form a second double-pass configuration that allows at least a portion of the light passing through the eyepiece lens to be reflected from the reflective optical element and propagate again through the eyepiece lens. The present invention provides, for example, the following. (Item 1) An occlusion-compatible optical see-through head-mounted display (OCOST-HMD), a polarizing element configured to receive light from a real scene and to produce polarized light at its output; a polarizing beam splitter (PBS); An objective lens, a spatial light modulator (SLM); An eyepiece and A quarter-wave plate (QWP) and a reflective optical element configured to reflect substantially all or a portion of light incident thereon in a first direction and to transition substantially all or a portion of light received from the microdisplay incident thereon from a second direction; Equipped with the SLM and the objective lens form a first double-pass configuration that allows at least a portion of light passing through the objective lens to be reflected from the SLM and propagate again through the objective lens; the eyepiece and the reflective optical element form a second double-pass configuration that allows at least a portion of the light passing through the eyepiece to be reflected from the reflective optical element and propagate again through the eyepiece. OCOST-HMD. (Item 2) the PBS is positioned to receive the polarized light and reflect the polarized light toward the objective lens; the PBS is positioned to receive and transmit light output from the first double-pass configuration toward the eyepiece and to reflect light it receives from the second double-pass configuration, including light from the microdisplay, toward a position of a human eye; OCOST-HMD described in item 1. (Item 3) further comprising a first reflective surface; The PBS is receiving the polarized light and transmitting the polarized light therethrough toward the objective lens; receiving light output from the first double pass configuration and reflecting it towards the eyepiece; reflecting light received by the PBS from the second double-pass configuration, including light from the microdisplay, toward the first reflective surface; 2. The OCOST-HMD of item 1, wherein the first reflective surface is positioned to reflect light incident thereon toward the position of a human eye. (Item 4) Item 2. The OCOST-HMD of item 1, wherein the SLM is configured to modulate light incident thereon. (Item 5) 5. The OCOST-HMD of item 4, wherein the SLM is configured to operate in an on-off modulation mode. (Item 6) 2. The OCOST-HMD of item 1, further comprising an occlusion mask corresponding to a virtual image presented on the microdisplay, the occlusion mask being used to effect modulation of one or more regions of the SLM. (Item 7) Item 1, an OCOST-HMD further comprising the microdisplay. (Item 8) 8. The OCOST-HMD of item 7, wherein the reflective optical element is positioned on the surface of the microdisplay. (Item 9) 8. The OCOST-HMD of item 7, wherein the microdisplay includes an organic light-emitting diode (OLED) device. (Item 10) Item 2. The OCOST-HMD of item 1, wherein the QWP is positioned between the eyepiece and the reflective optical element. (Item 11) Item 2. The OCOST-HMD of item 1, wherein the QWP is positioned between the eyepiece and the PBS. (Item 12) Item 2. The OCOST-HMD of item 1, wherein the SLM comprises a liquid crystal on silicon (LCoS) device. (Item 13) Item 2. The OCOST-HMD of item 1, wherein the OCOST-HMD is configured to generate an orthogonal image without using a roof prism. (Item 14) Item 1. The OCOST-HMD provides a pupil matching optical configuration that maps the user's pupil, i.e., the relay pupil, back to the position of the user's eye, allowing correct parallax to be maintained. (Item 15) Item 2. The OCOST-HMD of item 1, wherein the OCOST-HMD is configured to generate a field of view (FOV) that is not limited in at least one direction by the eyepiece. (Item 16) Item 2. The OCOST-HMD of item 1, wherein the OCOST-HMD has a field of view (FOV) of more than 40 degrees diagonally and optical performance of more than 20% modulation contrast across the full FOV. (Item 17) Item 2. The OCOST-HMD of item 1, wherein the OCOST-HMD has a see-through field of view (FOV) of 90 degrees by 40 degrees with an angular resolution of 1.0 arc minutes. (Item 18) Item 2. The OCOST-HMD of item 1, wherein at least a portion of the OCOST-HMD corresponds to a set of two afocal 4f relays that image an entrance pupil to a conjugate intermediate pupil location. (Item 19) 2. The OCOST-HMD according to item 1, wherein the OCOST-HMD forms a single-layer, double-pass, pupil-aligned OCOST-HMD. (Item 20) (a) an objective lens group including the objective lens; or (b) an eyepiece group including the eyepiece; Item 1. The OCOST-HMD according to item 1, comprising one or both of the following: (Item 21) An occlusion-compatible optical see-through head-mounted display (OCOST-HMD), a polarizer for generating polarized light associated with the real scene; Beam splitter (PBS), An objective lens, a spatial light modulator (SLM); An eyepiece and A retarder, a half-mirror configured to reflect substantially all of the light associated with the occlusion mask incident thereon in a first direction and to transition substantially all of the light associated with the virtual scene incident thereon from a second direction; Equipped with The PBS is receiving and directing said polarized light towards said SLM; receiving and directing light associated with the virtual scene toward a location for viewing by an eye of a user; receiving the light associated with the occlusion mask and directing it towards the half mirror; It is positioned to the SLM is configured to modulate light incident thereon according to a two-dimensional shape of the occlusion mask; The OCOST-HMD is configured to generate orthotopic images, and the positions of the user's pupils, i.e., relay pupils, are mapped to the positions of the user's eyes, allowing correct parallax to be maintained. OCOST-HMD. [Brief explanation of the drawings]

[0006] [Figure 1A] FIG. 1A illustrates a schematic diagram of a previously proposed two-layer folded OCOST-HMD optical design.

[0007] [Figure 1B]FIG. 1B illustrates a schematic diagram of a single-layer OCOST-HMD optical system, according to an exemplary embodiment.

[0008] [Figure 1C] FIG. 1C illustrates another configuration of a single-layer OCOST-HMD optical system, according to an exemplary embodiment.

[0009] [Figure 2] FIG. 2 illustrates an unfolded layout of some of the optical elements of the exemplary OCOST-HMD system of FIG. 1B.

[0010] [Figure 3] FIG. 3 illustrates the layout of optical components of a prototype OCOST-HMD, according to an exemplary embodiment.

[0011] [Figure 4A] FIG. 4A illustrates a modulation transfer function (MTF) plot for the virtual display path of an exemplary OCOST-HMD system, in accordance with disclosed embodiments.

[0012] [Figure 4B] FIG. 4B illustrates an MTF plot for the spatial light modulator (SLM) path of an exemplary OCOST-HMD system, in accordance with disclosed embodiments.

[0013] [Figure 4C] FIG. 4C illustrates an MTF plot for the see-through path of an exemplary OCOST-HMD system, in accordance with a disclosed embodiment.

[0014] [Figure 5] FIG. 5 illustrates a distorted grid of see-through optical paths across a full 40-degree diagonal field of view (FOV) for an exemplary OCOST-HMD system, according to disclosed embodiments.

[0015] [Figure 6]FIG. 6 illustrates an example OCOST-HMD layout and prototype, according to disclosed embodiments.

[0016] [Figure 7A] FIG. 7A illustrates an example augmented view of the real world and virtual scene without occlusion capabilities enabled.

[0017] [Figure 7B] FIG. 7B illustrates an example real-world scene when an occlusion mask is displayed on the SLM but no virtual content is shown on the OLED display.

[0018] [Figure 7C] FIG. 7C illustrates an example view captured with a mask on the SLM and a virtual scene displayed on an OLED display, where virtual content is inserted between two real objects.

[0019] [Figure 8] FIG. 8 shows measured on-axis MTF plots for the SLM, OLED, and optical see-through path, and camera, according to an example embodiment.

[0020] [Figure 9A] FIG. 9A shows an example plot illustrating image contrast degradation of a virtual object for different background scene brightness values ​​with occlusion disabled.

[0021] [Figure 9B] FIG. 9B shows an example plot illustrating image contrast degradation of a virtual object for different background scene lightness values ​​with occlusion enabled.

[0022] [Figure 10] FIG. 10 illustrates a block diagram of a device that can be used to implement certain aspects of the disclosed technology. DETAILED DESCRIPTION OF THE INVENTION

[0023] Detailed Description Disclosed embodiments relate to a novel optical architecture for a compact, occlusion-compatible optical see-through head-mounted display (OCOST-HMD), which uses an optical architecture with a dual winding path to provide an OCOST-HMD solution capable of rendering, among other features and advantages, per-pixel mutual occlusion, correct see-through viewing projection or pupil-aligned viewing between virtual and real views, and a very wide see-through field of view (FOV). A detailed example implementation and an example prototype system are described for illustrative purposes. The prototype provides a virtual display FOV of over 40 degrees diagonally and a resolution of 1,920 x 1,200 pixels with optical performance of over 20% modulation contrast across the full FOV. Furthermore, a wide see-through FOV of 90° x 40° can be achieved with an angular resolution of 1.0 arcminute, comparable to 20 / 20 vision.

[0024] The development of OST-HMDs presents many technical challenges, one of which is the challenge of correctly rendering the light-blocking behavior between digital and physical objects in space, otherwise known as mutual occlusion. Occlusion in augmented reality displays is a powerful depth cue, without which serious misjudgments, such as incorrect color registration, degraded image contrast, and object placement parallax, can occur. To have a correct mutual occlusion relationship, opaque virtual objects in the foreground should appear solid and occlude real objects located in the background, and vice versa. When blending virtual and real objects together, opaque virtual objects should appear completely opaque and occlude real objects located behind them, which should necessarily occlude the view of virtual objects located behind them. Thus, there are two types of occlusion: one in which a real scene object occludes a virtual one, and one in which a virtual object occludes a real scene. Occlusion of a virtual object by a real object can often be achieved in a simple manner, for example, by simply not rendering the virtual object on which the occluding real object sits when the real object's location relative to the virtual scene is known. Thus, in this case, occlusion of a virtual object by a real object can be performed simply by not rendering the known digitally overlapped area of ​​the virtual object that is occluded by the virtual object. However, occlusion of a real object by a virtual one presents a much more complex problem because it requires blocking light in the real scene.

[0025] Over the past decade, several OCOST-HMD concepts have been proposed, and even fewer designs have been prototyped. Existing methods for implementing OCOST-HMDs fall into two types: direct ray blocking and per-pixel modulation. Direct ray blocking methods selectively block light rays from a see-through scene without focusing them. This can be implemented by selectively modifying the reflective properties of physical objects or by passing light from the real scene through one or more layers of spatial light modulators (SLMs) placed directly near the eyes. For example, in some previous systems, creating natural occlusion of virtual objects by physical ones via head-mounted projection display (HMPD) devices has been investigated using retroreflective screens on non-occluding physical objects, which can only be used in limited settings. In some previous systems, occlusion functionality has been investigated through transmissive SLMs placed directly near the eyes without imaging optics. The direct ray blocking method via an SLM would be a simple and reasonable solution if the eye were a pinhole aperture, allowing a single ray from each real-world point to reach the retina. Instead, the eye has an area aperture, which makes it practically impossible to use a single-layer SLM to block all rays seen by the eye from an object without blocking rays from other surrounding objects.

[0026] Some previous systems include lensless computed multilayer OST-HMD designs, which consist of a pair of stacked transmissive SLMs, a thin, transparent backlight, and a high-speed optical shutter. Multiple occlusion patterns can be created using a multilayer computed bright-field method so that the occluded bright-field of the see-through view can be properly rendered. While the multilayer bright-field rendering method can theoretically overcome some of the limitations of the single-layer ray-blocking method, it suffers from several major limitations, such as a significantly degraded see-through view, limited accuracy of the occlusion mask, and low light efficiency. The unfavorable results can be attributed to the lack of imaging optics, the low light efficiency of the SLM, and, most importantly, severe diffraction artifacts caused by the SLM's fine pixels located close to the pupil.

[0027] The disclosed embodiments can be classified as per-pixel occlusion methods, such as those shown in FIG. 2 (the embodiment of FIG. 2 falls into this category), in which a focused image of the see-through view is formed at a modulation plane into which an SLM is inserted to render an occlusion mask and selectively occlude the real-world scene point-by-point. One previous per-pixel modulation method includes the ELMO series prototypes developed in the early 2000s. The ELMO-4 prototype consists of four lenses, two prisms, and three optical mirrors, forming two afocal 4f relays arranged in a ring configuration to present a pupil-matched, upright, occlusion-compatible view of the real scene. Pupil-matched occlusion, otherwise known as projection correction, maps the user's pupil, i.e., the relay pupil, back to the user's eye position so that correct parallax can be maintained. Limited by microdisplay and SLM technology at the time, the ELMO prototype has very low resolution for both the see-through and virtual display paths, both of which used 1.5-inch QVGA (320 × 240) transmissive LCD modules. The ring structure has been further modified using a 3D-printed housing with a modern transmissive LCD and microdisplay. However, the ring-like folding architecture presents a very bulky package and blocks a large portion of the user's face. Additionally, utilizing a transmissive liquid crystal display (LCD) as an SLM results in extremely low light efficiency (<20%) when coupled with a beamsplitter.

[0028] Several systems have attempted to overcome these limitations by proposing an x-cube prism that utilizes polarization-based optics combined with a reflective SLM to combine the two optical paths to achieve a more compact form factor and high optical efficiency. While this design appears to be a promising solution to the OST-HMD occlusion problem, this solution failed to orient the see-through horizontal view, resulting in incorrect projection. Another system proposed a complex two-layer folded optical system using a prism and a reflective SLM to create a compact, high-resolution OCOST-HMD. By utilizing a reflective liquid crystal-on-silicon (LCoS) device as the SLM, this system enabled high luminance throughput and high optical resolution for both the virtual and see-through paths. Optical design and preliminary experiments demonstrated some potential for desirable form factor and high optical performance. Based on this same two-layer folded optical system architecture, several systems have been designed and built using off-the-shelf optical components to provide working prototypes of high-resolution OCOST-HMDs. A prototype capable of rendering pixel-by-pixel mutual occlusion utilizes an OLED microdisplay for the virtual display path combined with a reflective LCoS as the SLM for the see-through path to achieve an occlusion-aware OST-HMD, resulting in a 30-degree diagonal FOV and 1,920 × 1,080 pixel resolution. However, the two-layer folded optics architecture fails to provide aligned pupils or correct viewing projection as the pupils are relayed directly in front of the user's eyes. In addition, like the original ring-shaped architecture, the see-through FOV of the two-layer folded architecture is limited to the FOV available through the see-through objective optics and is therefore typically narrow. This poses a significant limitation for many applications where access to a much wider peripheral field of view is important.

[0029] Thus, state-of-the-art OST-HMDs lack the ability to selectively block real-world light from reaching the eyes. As a result, digitally rendered virtual objects viewed through an OST-HMD typically appear "afterglow-like" (or translucent) and constantly floating "in front of" the real world. Figure 7A provides one illustrative example of such an afterglow condition.

[0030] The disclosed embodiments provide, among other features and advantages, an optical system based on per-pixel modulation that helps overcome the above-mentioned limitations of current state-of-the-art OCOST-HMD systems. The disclosed embodiments achieve this goal, at least in part, by utilizing a double-pass optical architecture for a compact, high-resolution OCOST-HMD system.

[0031] An implementation of the disclosed embodiments includes an OCOST-HMD system that includes three overlaid optical paths, referred to as the display path, the SLM path, and the see-through path, respectively. The disclosed optical architecture results in an OCOST-HMD solution capable of per-pixel mutual occlusion, correct see-through viewing projection or pupil-aligned viewing between virtual and real views, and very wide see-through field of view rendering.

[0032] Note that in different embodiments, the central portion of the see-through FOV can be occlusion-enabled to create wide peripheral vision. Based on the disclosed optical design architecture and custom optimization of the optical system, we experimentally demonstrated a wearable stereoscopic prototype system utilizing an OLED microdisplay for the virtual display path combined with a reflective LCoS as the SLM for the see-through path optical mask. Furthermore, note that OLED is one example of a microdisplay used in the prototype's construction; other display technologies may also be implemented based on the disclosed embodiments. The prototype provides a virtual display FOV of over 40 degrees diagonally and a resolution of 1,920 x 1,200 pixels with optical performance of over 20% modulation contrast across the full FOV. Furthermore, the system is capable of a wide see-through FOV of 90° x 40° with an angular resolution of 1.0 arcminute, comparable to 20 / 20 vision. The prototype has a total weight of approximately 300 grams, stereo dimensions of 140 mm (H) x 130 mm (W) x 25 mm (D), and is capable of achieving a dynamic range of over 100:1 in well-lit ambient environments. The optical performance of the OST-HMD is further compared with and without occlusion to demonstrate the side effect of virtual content contrast for non-occluding devices in bright environments. Note that the disclosed prototype is provided for purposes of illustrating an example implementation of an OCOST-HMD system according to disclosed embodiments.

[0033] 1A illustrates a schematic diagram of a previously proposed two-layer folded OCOST-HMD optical design, and FIG. 1B illustrates an OCOST-HMD optical system according to an exemplary embodiment of the disclosed embodiments. The system in FIG. 1B is part of a double-pass single-layer architecture that provides a pupil-aligned OCOST-HMD design.

[0034] In the system of FIG. 1A, two folding mirrors 108, 124, a roof prism 104, and a PBS 112 are used to fold the optical path into a two-layer optical architecture. The system of FIG. 1A can be more simply separated into two components (objective lens 106 and eyepiece lens 122) to form the basis of a Keplerian telescope. Furthermore, the light through this system can be categorized into two distinct components: a virtual display and a real-world view. The light path for the real-world view is shown by an oval enclosing an arrow, while the light path for the virtual display does not include an arrow and consists of a path from the OLED display 120 to the first 45-degree surface mirror 108 and the eyepiece lens 122. The simplified architecture shown in FIG. 1A presents a series of design constraints and optical considerations.

[0035] First, utilizing all glass frit lenses 106, 110 while placing the folded objective lens path as a second layer in front of the eyepiece 122 increases the optical form factor by a factor of two in thickness, leading to a relatively bulky and heavy system design. Second, a polarizing beam splitter (PBS) 112 combines the optical paths of the modulated actual view 102 and the virtual view associated with the virtual content 118 together so that the same eyepiece 122 module is shared for viewing the virtual display and the modulated real-world view (the combined image is viewed at eye position 126). Because the system is constructed like a simple telescope, this forces the virtual display and real-world view paths to combine as a 1:1 conjugate to ensure unit magnification of the real scene, forcing the optical mask 116 and display 120 to share identical optical requirements. However, this poses a problem because the SLM 114 and the microdisplay 120 are often implemented using different technologies and do not share the same technical specifications, resulting in mismatches in optical performance. Another limitation of this arrangement is that the see-through FOV is limited to the FOV imaged by the telescope system, resulting in tunnel vision for the see-through path, which is undesirable for augmented reality (AR) systems. Additionally, the configuration of FIG. 1A can be considered a single afocal 4f system. Similar to a telescope, this single afocal 4f design shifts the conjugate pupil to the other side of the optical relay, resulting in an incorrect viewpoint offset. This incorrect viewing projection results in objects appearing closer than they actually are, thus resulting in a depth difference between the object, the hand, and the eye. Furthermore, the inclusion of a roof prism 104 to generate an upright image adds to the cost, size, and weight of the system.

[0036] FIG. 1B illustrates a schematic diagram of a single-layer OCOST-HMD optical system according to an exemplary embodiment. The system uses a PBS 136, an objective lens group 138 (which may include one or more lenses), and an eyepiece lens group 144 (which may include one or more lenses), which also form two relay lens groups due to a second pass through the objective lens 138 and the eyepiece lens 144, as further illustrated in FIG. 2. As is evident from FIG. 1B, this configuration provides a compact form factor and enables per-pixel occlusion capabilities due to the small number of components and associated folded path. The double-pass system enables a single optical layer and eliminates the use of heavy optics such as first surface mirrors and roof prisms compared to the configuration of FIG. 1A, creating a significantly reduced form factor along with a lightweight wearable HMD design. The term "single layer" can be further understood by considering that when a viewer of an OCOST-HMD looks through eyepiece 144, the viewer receives light only from a single layer or set of optical systems in the system of FIG. 1B, whereas in FIG. 1A the viewer will receive light from two sets of optical components, one each from the real-world view and the virtual content.

[0037] The illustration of FIG. 1B is facilitated by splitting the optical path into three optical paths: the display path (square), the SLM path (oval), and the see-through path (circular). The single PBS 136 acts as both a foldable element and a combiner, merging together the optical paths of the modulated real view 132 (see-through and SLM paths) and the virtual view from virtual content 152 (display path), so that the same eyepiece 144 and relay module are shared for viewing the virtual display and the modulated real-world view. Unlike the configuration of FIG. 1A, the double-pass architecture of FIG. 1B enables the decoupling of the SLM and display paths by utilizing a relay, allowing the objective lens 138 and eyepiece 144 to have different focal length parameters, no longer needing to be 1:1 conjugate, to achieve unit magnification of the real scene while maintaining unit magnification for the overall system. This decoupling of the different optical paths allows the element paths to be specifically designed for different display technologies.

[0038] Each of the three paths in FIG. 1B—the display path, the SLM path, and the see-through path—offers its own unique optical performance. Light from the see-through path, identified by a circle, passes through a retarder (e.g., polarizer 134). In this exemplary configuration, the retarder polarizes the light in the S orientation. The S-polarized light is reflected from PBS 136. An objective lens 138 (or lens group) then collects light from the physical environment and forms an intermediate image at its focal plane, where an amplitude-based reflective SLM (e.g., LCoS) 140 is located. SLM 140 can manipulate the polarization orientation and render an occlusion mask 142 to control the opacity of the actual view. For example, SLM 140 can generate an on-off modulation scheme in which light incident on SLM 140 is selectively reflected (the SLM is off) or allowed to be transmitted through it on a pixel-by-pixel basis. Other exemplary modulation schemes can also be implemented.

[0039] In the above example, light reflected from the SLM / LCoS 140 is flipped to the P orientation within the SLM path (circular) and then passed back through the same objective lens 138, through the PBS 136, and through the eyepiece 144, forming an optical relay. The light is then transmitted (or otherwise delayed) through a quarter-wave plate (QWP) 146 to produce light having a right-hand circular (RHC) polarization direction that is incident on the half mirror 148. The half mirror 148 is one non-limiting example of a reflective surface that allows substantially all or a portion of the reflection of light incident thereon from a first direction (e.g., in the example configuration of FIG. 1B, light traveling upward) to be retroreflected, while allowing substantially all or a portion of the light incident thereon from a second direction (e.g., in the configuration of FIG. 1B, light traveling downward from the microdisplay 150, e.g., an OLED) to pass therethrough. Such elements can be designed, for example, as multi-layer thin film stacks, as part of a microdisplay, or as stand-alone components.

[0040] Upon reflection from the "half mirror" 148, the RHC polarization is inverted to left-handed circular (LHC) polarization, and the LHC light then passes back through the QWP 146 and eyepiece 144, where the light is converted back to S-polarized light, which is then reflected by the PBS 136 into the user's eye at eye position 154. In the display path (square), unpolarized light from the OLED display 150 is imaged by the shared eyepiece 144, and S-polarized light is combined by the PBS 136 into the user's eye, giving three high-resolution overlaid images: the optical mask 142, the real scene 132, and the virtual scene 152. Instead of using a roof prism as implemented in FIG. 1A, an even number in the parity switch ensures that the see-through image in the configuration of FIG. 1B is upright and orientated, which is important for an OCOST-HMD system because it cannot be digitally altered.

[0041] FIG. 1C illustrates another configuration of a single-layer OCOST-HMD optical system according to an exemplary embodiment. FIG. 1C has similar components to the configuration in FIG. 1B for combining real view 162 and virtual content 182, but polarizer 164 modifies light from real scene 162 based on the desired modulation to generate P-polarized light that passes through PBS 166 and is incident on SLM 170, which implements mask 188. Thus, the positions of some components are rearranged compared to FIG. 1B to provide a single-layer configuration. The configuration of FIG. 1C provides similar features and advantages to the configuration of FIG. 1B in providing a combined view to eye position 188 after reflection from reflective surface 186.

[0042] 1B and 1C illustrate that the QWPs 146, 176 are positioned between the eyepieces 144, 174 and the microdisplays 150, 180 (or between the eyepieces 144, 174 and the half-mirrors 148, 178). However, the QWPs 146, 176 can also be positioned on either side of the eyepieces 144, 174. Thus, in some embodiments, the QWPs 146, 176 are positioned between the PBSs 136, 166 and the eyepieces 144, 174.

[0043] In contrast to two-layer designs, the disclosed double-pass single-layer architecture allows the optical paths to be properly pupil-aligned. To properly determine the pupil location, we can view the optical system as a set of two afocal 4f relays that image the entrance pupil to a conjugate intermediate pupil location. FIG. 2 shows the unfolded layout of the optical elements in the exemplary OCOST-HMD system of FIG. 1B (only the unfolded paths are illustrated in FIG. 2). The first afocal relay (left) includes an eyepiece (labeled as primary eyepiece 154) and the first system relay lens (labeled as secondary eyepiece 154, the left lens in relay group 208), which is the eyepiece itself. (As shown at the top of FIG. 1B, it can be seen that light passing through the eyepiece is reflected off the OLED (microdisplay 150) and passes a second time through the eyepiece (secondary eyepiece 154).) The second afocal relay in Figure 2 includes a second system relay lens (the right lens in the relay group, secondary objective lens 138) and an objective lens (primary 138 objective lens), which is identical to the objective lens (as shown at the bottom of Figure 1B, light passing through the objective lens is reflected from LCoS 140 and passes through the objective lens a second time). Figure 2 further illustrates the locations of entrance pupil 202, intermediate pupil 210, and exit pupil 216.

[0044] The improvement in achievable FOV can be observed by comparing the configuration of FIG. 1A, in which the FOV is limited by the objective FOV, with the configuration of FIG. 1B, in which the FOV can be significantly increased. In particular, the see-through FOV can become essentially unlimited in the horizontal direction (i.e., into the paper), limited only by the extent of the eyepiece and objective (or the size of the beamsplitter). The increase in FOV in the horizontal direction can be beneficially used to provide improved peripheral visibility for OCOST-HMD systems.

[0045] Due to the reflective LCoSSLM option, the exemplary configuration of FIG. 1B requires image-space telecentricity for the SLM path to achieve high contrast. Therefore, the distance from the intermediate pupil x1 to the objective lens is determined by the focal length (f) of the objective lens. obj ) must be equal to f . Understanding that we have a double-pass system, we can see that f , b and the second relay lens focal length (f r2 ) can be assumed to be equal, allowing the system exit pupil to be relayed back onto itself. We can then calculate the position of the intermediate pupil relative to the system focal length and entrance pupil to be equation (1). [ka] In the formula, f eye and f r1 are the individual focal lengths of the eyepiece and the first relay lens, and E relief is the pupil distance, [ka] is the distance to the intermediate pupil after the first relay lens. Again, since the system utilizes double optical passes, we eye f r1 and obtain the greatly reduced equation (2). [ka]

[0046] For the system to be properly pupil aligned, the distance after the first lens group is equal to the pupil distance (ER) of the system, i.e., l=E relief This relationship then becomes E relief =f eye or in other words, the display path must also be telecentric. We then calculate the half FOV (θhalf Based on the appropriate eye clearance (E clearance ) to find the required principal plane (P p ) location can be determined to be equation (3). [ka] In the formula, D ep is the diameter of the entrance pupil.

[0047] Exemplary Prototype Design 1B and the analytical relationships described in equations (1)-(3) above, an exemplary custom prototype system was designed. Key specifications of the prototype system are summarized in Table 1 below. [Table 1]

[0048] One of the key parameters driving the exemplary design is the choice of display technology. We selected a 0.85-inch Emagin OLED microdisplay for the virtual display path. With an active area of ​​18.4 mm–11.5 mm and an aspect ratio of 8:5, the eMagin OLED offers a pixel size of 9.6 μm at a native resolution of 1,920 × 1,200 pixels. Based on this microdisplay, we aimed to achieve an OCOST-HMD prototype with a diagonal FOV of >40°, i.e., 34° horizontally and 22° vertically, and an angular resolution of 1.06 arcmin per pixel, corresponding to a Nyquist frequency of 53 cycles / mm in microdisplay space or 28.6 cycles / degree in visual space. Separately, for the SLM path, we used a reflective LCoS 0.7-inch display from the projector. A reflective SLM was chosen due to its substantial advantages in light efficiency, contrast, and low diffraction artifacts typically found in the optically transmissive SLMs used in previous studies.

[0049] The selected LCoS offers a native resolution of 1,400 × 1,050 pixels, a pixel pitch of 10.7 μm, and an aspect ratio of 4:3. Based on the SLM's different display specifications, we aimed to achieve an optical mask diagonal FOV of >42.5°, i.e., 34° horizontally and 25.5° vertically, and an angular resolution of 1.45 arcmin per pixel, corresponding to a Nyquist frequency of 47 cycles / mm in SLM space or 19.66 cycles / degree in visual space. Furthermore, our system requires an objective lens focal length of 24.4 mm and an eyepiece focal length of 29.8 mm, giving a relay magnification of 1:1.22. To allow for approximately ±25° of eye rotation within the eye socket without causing vignetting, we set an exit pupil diameter (EPD) of 10 mm. An eye-clearance distance of 20 mm was used to allow for compatibility with most head shapes.

[0050] To achieve high optical performance across the three optical paths, we optimized our system using three zoom configurations, each corresponding to a different optical path and design specification. Figure 3 illustrates the lens layout of the final OCOST-HMD prototype design. The optical path for the virtual display (eyepiece) is indicated by the rays designated by the dashed rectangle, while the optical path for the SLM (relay + eyepiece) is indicated by the rays designated by the dashed oval. The see-through path (objective + relay + eyepiece) is indicated by the rays designated by the dashed circle. Note that the see-through path overlaps with the microdisplay and SLM paths after the PBS; therefore, only the virtual display rays are traced to the pupil.

[0051] Overall, the final lens design for the prototype example in Figure 3 includes 10 lenses, including five original crown glass lenses, three acrylic aspheric lenses, and two custom flint glass lenses. Lenses labeled 2-6 form the eyepiece group. Lenses labeled 8-12 form the objective group. A half mirror is shown as the gray element between the quarter-wave plate and OLED 7. A wire grid polarizer and QWP film, along with a single custom PBS from Moxtek, were used to manipulate polarization. The system was optimized for three wavelengths, namely, 430, 555, and 625 nm, with weights of 1, 2, and 1, respectively, according to the dominant wavelengths of the OLED microdisplay. To ensure that the system was properly pupil-aligned for correct viewing projection according to equations (1) and (2), the objective and eyepiece lenses were optimized to have a chief ray deviation of less than ±0.5°, demonstrating image-space telecentricity. The eyepieces were cropped to achieve an eye clearance of 20 mm to 10 mm EPD.

[0052] Tables 2-9 provide the optical system specifications for the virtual display and see-through paths, respectively, for the prototype system described above. Both optical paths were ray traced from the system's exit pupil, which coincides with the eye's entrance pupil. The term "aspheric" in the tables refers to an aspheric surface, which can be represented by the following equation: [ka] where z is the surface deflection measured along the z-axis of the local x, y, z coordinate system, c is the vertex curvature, r is the radial distance, k is the conic constant, and AE are the fourth, sixth, eighth, tenth, and twelfth order deformation coefficients, respectively. Tables 4-9 provide the aspheric coefficients for aspheric surfaces 11, 12, 15, 16, 23, and 24, respectively. [Table 2] [Table 3-1] [Table 3-2] [Table 4] [Table 5]

[0053] The simulated optical performance of a double-pass OCOST-HMD prototype system is assessed across the full FOV in display space, and the spatial frequency is characterized in terms of cycles per millimeter. In the example provided, the optical performance of the see-through path is limited to a 40° diagonal according to the field that passes through the system and is optically overlaid on the virtual and masked images. Light from the real scene outside the field passes only through a single PBS and is optically unaffected, as it would otherwise be seen at the natural resolution of the human eye.

[0054] Figures 4A-4C show polychromatic modulation transfer function (MTF) curves evaluated using a 4-mm pupil, similar to the human eye, for several weighted fields of the virtual display, SLM, and see-through paths. In these figures, modulation transfer functions of 0, 6, 10, 15, and 20 degrees are evaluated for transverse (Tan) and radial (Rad) diagonal half-FOVs with a 4-mm pupil diameter, and a cutoff spatial frequency of 53 cycles / mm is plotted for the OCOST-HMD OLED optical path (Figure 4A), the OCOST-HMD SLM optical path (Figure 4B), and 110 cycles / degree for the OCOST-HMD see-through optical path (Figure 4C). Starting with Figure 4A, the OLED display optical performance preserves approximately 40% modulation on average across the full field at the designed Nyquist frequency of 52 cycles / mm, corresponding to a 9.6 μm pixel size. The optical performance of the LCoS, shown in Figure 4B, demonstrates an average 50% modulation over the full field at the designed Nyquist frequency of 47 cycles / mm, corresponding to a 10.7 μm pixel size. Finally, Figure 4C shows the see-through optical performance, maintaining an average modulation of 15% at the human eye cutoff frequency of 110 cycles / mm, corresponding to 1.0 arc minute resolution or 20 / 20 human visual acuity.

[0055] Figure 5 shows the distortion grid of the see-through optical path over the full 40-degree diagonal FOV, overlapped with the virtual image. In the figure, the real and paraxial FOVs nearly coincide, illustrating the high agreement between the two. According to our optical specifications seen in Table 1 above, the see-through path has <1% distortion over the full field. This is important because, unlike display distortion, which can be digitally corrected, the see-through path cannot be corrected. Distortion for the microdisplay (virtual image) was kept below 5%, while distortion for the SLM (optical mask) was kept up to 20% due to digital correction.

[0056] Along with MTF and distortion, several other metrics, such as wavefront error and spot diagram, were also used to characterize the optical performance of the virtual display path. Both the SLM and microdisplay paths suffer primarily from lateral chromatic aberration and coma. This is due to the telecentric design of the non-pupil-forming eyepieces utilized in both the SLM and microdisplay paths, which does not allow the stop position to be moved to balance off-axis aberrations. Overall, the wavefront aberrations within each of the three paths are sufficiently low, below one wave. The average root-mean-square (RMS) spot diameter across the field is 9 μm for both the see-through and display paths, however, due to such large allowable distortion, it jumps to 16.4 μm for the SLM path. Although this appears to be larger than the 10.7 μm pixel size, this difference is primarily due to lateral chromatic aberration and can be corrected. Exemplary System Prototype and Experimental Demonstration

[0057] FIG. 6 illustrates an exemplary OCOST-HMD layout and prototype according to disclosed embodiments. Panel (a) shows a front view of the fully assembled OCOST-HMD Solidwords CAD design referenced to an average human head. Panel (b) shows a side view of the fully assembled OCOST-HMD CAD design, demonstrating a significantly reduced form factor due to the double-pass architecture. Due to multiple passes of light through the same optical path, optical performance is sensitive to optical and mechanical tolerances. Regarding the mechanical design, individual lenses were held in place by locking screws to achieve more compensation, reduce cumulative tolerances within the mechanical design, and meet minimum MTF requirements. Panels (c) and (d) show front and angled views, respectively, of a binocular prototype of an OCOST-HMD system built on the optical design in FIG. 3, further illustrating a quarter (coin) and providing size metrics. The overall height and width of the prototyped OCOST-HMD system was 130 mm × 140 mm, with a depth of 25 mm and an adjustable intraocular distance of 50–70 mm.

[0058] Figure 7 shows a qualitative demonstration of the occlusion capabilities of the prototype OCOST-HMD of Figures 3 and 6. A real-world background scene with generic objects (text, cans, and boxes with different colors, shapes, and print fonts) was used to provide several different spatial frequencies and object depths. These items were placed against a well-illuminated white background wall (approximately 300-500 cd / m²). The virtual 3D scene used in this example was a simple image of a wild cat. Panels (a)-(c) in Figure 7 show a set of images captured with a digital camera placed at the eyepiece exit pupil. The same 16 mm camera lens and 1 / 3-inch Point Grey image sensor with a 3.75 μm pixel pitch were used, with an increased 4 mm aperture to better match the F / # of the human eye under typical lighting conditions.

[0059] By simply turning on the OLED microdisplay and not applying a modulated mask to the SLM, panel (a) in Figure 7 shows an augmented view of the real-world and virtual scenes without occlusion capabilities enabled. Without a mask occluding the see-through path, the "wildcat" figure with a cowboy hat (see panel (b) for a better outline of the wildcat) appears washed out, transparent, and unrealistic due to the brightness of the background scene shared with the virtual display. This spatially obscures the wildcat's depth.

[0060] Panel (b) depicts the view of the real-world scene in the reverse situation, i.e., where an occlusion mask is displayed on the SLM but no virtual content is shown on the OLED display. This verifies that the mask can effectively block the superimposed portion of the see-through view.

[0061] Panel (c) shows the view captured with a mask on the SLM and the virtual scene displayed on the OLED display, where a virtual wildcat is inserted between two real objects to demonstrate the mutual occlusion capabilities of our system. In this case, full capability and correct depth perception are rendered with improved contrast. By knowing the relative location of a WD-40 can intended to occlude part of the wildcat figure, we removed pixels from the wildcat rendering that corresponded to the occluding can's projection on the virtual display. Again, the significance of the result is that correct occlusion relationships can be created and used to impart a non-parallel perception of depth to virtual images in the OST-HMD.

[0062] The disclosed double-pass OCOST-HMD system can achieve high optical performance and dynamic range for real and virtual content with a form factor, viewpoint projection, and technical specifications that are significantly improved over our previous OCOST-HMD designs.

[0063] Example optical performance test results The vertical and horizontal FOV of the exemplary system was measured for each optical path. The see-through FOV was determined to be approximately 90° horizontally and 40° vertically, with an occlusion-enabled see-through FOV of approximately 34° horizontally and 26° vertically, while the virtual display had an FOV of approximately 33.5° horizontally and 23° vertically, giving a measured diagonal full FOV of 41.6°. Due to our improved double-pass architecture and added optical repeaters, the LCoS is able to fully occlude the virtually displayed scene.

[0064] The optical performance of the prototype system was further quantified by characterizing the MTF performance of the three optical paths through the prototype. A high-performance camera, consisting of an approximately diffraction-limited 25 mm camera lens from Edmund Optic and a 1 / 2.3 in. Point Gray image sensor with a 1.55 μm pixel pitch, was placed at the system's exit pupil. This resulted in an angular resolution of approximately 0.5 arc minutes per pixel, significantly higher than the prototype's expected performance. Therefore, it is assumed that no performance loss in MTF was caused by the camera. The camera then captured images of a tilted edge target displayed by either a microdisplay at an angle or a printed target placed within the see-through view. To provide separable quantification of performance for the virtual and see-through paths, virtual images of the tilted edge were taken while the see-through scene was completely blocked by the SLM. Similarly, see-through images of the target were taken with the microdisplay turned off. The captured tilted edge images were analyzed using Imatest® software to obtain the MTF of the corresponding optical paths.

[0065] Figure 8 shows the measured on-axis MTF performance of the SLM, OLED, and optical see-through paths, as well as the camera, along with each individual tilted edge. Due to the magnification difference between the pixel pitch of the camera sensor, microdisplay, and SLM, the horizontal axis of the Imatest® MTF measurement was scaled by the pixel magnification difference between the camera and display and then converted to define spatial frequency in visual display space in terms of cycles / mm. The prototyped design was able to achieve a contrast of over 50% at the Nyquist frequency of the virtual display, 53 cycles / mm, with similar performance for the SLM path. Meanwhile, the modulation contrast for the see-through path was approximately 15% at a cutoff frequency of 110 cycles / mm, corresponding to one arc minute. The curves shown in Figure 8 are very similar to the on-axis curves in Figure 3, demonstrating that the resolution of the three optical paths through the occlusion module is nearly intact using the originally established design specifications and a human viewer.

[0066] We measured the image contrast between the virtual display and the real-world scene as a function of real-world scene luminance for different spatial frequencies. Grayscale solid images, ranging from black to white in 10 linear steps, were displayed on an LCD monitor at luminances ranging from 0 to 350 cd / m. 2 A controlled background scene was created with varying luminance of 1000 Hz. The monitor was placed approximately 10 cm in front of the OCOST-HMD system to simulate a range of real-scene luminances. Sinusoidal grating patterns with spatial frequencies ranging from 0.88 to 28.2 cycles / degree were displayed on the OLED microdisplay (virtual path) to evaluate the effect of scene luminance on the image contrast of the virtual scene at different spatial frequencies. The percentage reduction in contrast for the virtual scene was then plotted and compared with occlusion-enabled (SLM blocks see-through light) and no occlusion (SLM allows see-through light to pass).

[0067] Figures 9A and 9B illustrate plots of virtual object contrast using the see-through path with and without occlusion, respectively. We found that the contrast of virtual objects without occlusion (Figure 9A) was 300 cd / m 2 It can be observed that the contrast of the virtual target with occlusion in the real scene (Figure 9B) remained nearly constant over increased luminance, while it dropped sharply to zero for well-illuminated environment luminances above 100:1. We further measured the achievable contrast ratio by measuring a collimated, depolarized light source through our system with full occlusion enabled and disabled. The dynamic range of the occlusion system was determined to be greater than 100:1.

[0068] One aspect of the disclosed embodiments relates to an occlusion-enabled optical see-through head-mounted display (OCOST-HMD) including a polarizing element configured to receive light from a real scene and generate polarized light at its output, a polarizing beam splitter (PBS), an objective lens, a spatial light modulator (SLM), an eyepiece lens, a quarter-wave plate (QWP), and a reflective optical element configured to reflect substantially all or a portion of light incident thereon in a first direction and transition substantially all or a portion of light received from a microdisplay incident thereon from a second direction. The SLM and objective lens form a first double-pass configuration that allows at least a portion of the light passing through the objective lens to be reflected from the SLM and propagate again through the objective lens. The eyepiece lens and the reflective optical element form a second double-pass configuration that allows at least a portion of the light passing through the eyepiece lens to be reflected from the reflective optical element and propagate again through the eyepiece lens.

[0069] In one exemplary embodiment, the PBS is positioned to receive polarized light and reflect it toward the objective lens, and the PBS is also positioned to receive light output from the first double-pass configuration, transmit it therethrough toward the eyepiece, and reflect light the PBS receives from the second double-pass configuration, including light from the microdisplay, toward the position of the human eye. In another exemplary embodiment, the OCOST-HMD further includes a first reflective surface, and the PBS is positioned to (a) receive polarized light and transmit it therethrough toward the objective lens, (b) receive light output from the first double-pass configuration, reflect it therethrough toward the eyepiece, and (c) reflect light the PBS receives from the second double-pass configuration, including light from the microdisplay, toward the first reflective surface. In this exemplary embodiment, the first reflective surface is positioned to reflect light incident thereon toward the position of the human eye.

[0070] According to one exemplary embodiment, the SLM is configured to modulate light incident thereon. For example, the SLM is configured to operate in an on-off modulation mode. In another exemplary embodiment, the OCOST-HMD further includes an occlusion mask corresponding to a virtual image presented on the microdisplay, the occlusion mask being used to effect modulation of one or more regions of the SLM. In yet another exemplary embodiment, the OCOST-HMD further includes a microdisplay. In yet another exemplary embodiment, the reflective optical element is positioned on a surface of the microdisplay. According to another exemplary embodiment, the microdisplay includes an organic light-emitting diode (OLED) device.

[0071] In another exemplary embodiment, the QWP is positioned between the eyepiece and the reflective optical element. In one exemplary embodiment, the QWP is positioned between the eyepiece and the PBS. In another exemplary embodiment, the SLM includes a liquid crystal on silicon (LCoS) device. In yet another exemplary embodiment, the OCOST-HMD is configured to generate an orthogonal image without using a roof prism. In another exemplary embodiment, the OCOST-HMD provides a pupil-matching optical configuration that maps the user's pupil, i.e., the relay pupil, back to the user's eye position, allowing correct parallax to be maintained. According to yet another exemplary embodiment, the OCOST-HMD is configured to generate a field of view (FOV) that is not limited in at least one direction by the eyepiece.

[0072] In another exemplary embodiment, the OCOST-HMD has a field of view (FOV) greater than 40 degrees diagonally and optical performance greater than 20% modulation contrast across the full FOV. In one exemplary embodiment, the OCOST-HMD has a 90 degree by 40 degree see-through field of view (FOV) with an angular resolution of 1.0 arcminute. In yet another exemplary embodiment, at least a portion of the OCOST-HMD corresponds to a set of two afocal 4f relays that image the entrance pupil to a conjugate intermediate pupil location. In another exemplary embodiment, the OCOST-HMD forms a single-layer, double-pass, and pupil-aligned OCOST-HMD. In one of several exemplary embodiments, the OCOST-HMD includes one or both of the following: (a) an objective lens group including an objective lens, or (b) an eyepiece lens group including an eyepiece lens.

[0073] Another aspect of the disclosed embodiments relates to an occlusion-enabled optical see-through head-mounted display (OCOST-HMD) including a polarizer for generating polarized light associated with a real scene, a beam splitter (PBS), an objective lens, a spatial light modulator (SLM), an eyepiece, a retarder, and a half-mirror configured to reflect substantially all of the light associated with an occlusion mask incident thereon in a first direction and to transition substantially all of the light associated with a virtual scene incident thereon from a second direction. In this configuration, the PBS is positioned to (a) receive polarized light and direct it toward the SLM, (b) receive light associated with the virtual scene and direct it toward a position for viewing by a user's eyes, and (c) receive light associated with the occlusion mask and direct it toward the half-mirror. The SLM is configured to modulate the light incident thereon according to the two-dimensional shape of the occlusion mask. The OCOST-HMD is configured to generate orthotopic images, and the position of the user's pupil, i.e., relay pupil, is mapped to the position of the user's eye, allowing correct parallax to be maintained.

[0074] 10 illustrates a block diagram of a device 1000 that can be used to implement certain aspects of the disclosed technology. For example, the device of FIG. 10 can be used to capture and process images, receive, process, store, provide for display, and / or transmit various data and signals associated with the disclosed image sensors and / or microdisplays and SLMs, and enable control, display, storage, and processing of virtual content and occlusion masks, as well as brightness control, light modulation, or other operations associated with the electronic and optoelectronic components disclosed herein. The device 1000 comprises at least one processor 1004 and / or controller, at least one memory 1002 unit in communication with the processor 1004, and at least one communication unit 1006 that enables the exchange of data and information, directly or indirectly, with other entities, devices, databases, and networks via a communication link 1008. The communications unit 1006 may provide wired and / or wireless communications capabilities according to one or more communications protocols and may therefore include appropriate transmitters / receivers, antennas, circuitry and ports, and encoding / decoding capabilities that may be necessary for proper transmission and / or reception of data and other information. The example device 1000 of Figure 10 may be integrated as part of a larger component (e.g., a server, computer, tablet, smartphone, etc.) that may be used to implement various calculations, methods, or algorithms disclosed herein.

[0075] Processor 1004 may include a central processing unit (CPU) and, for example, control the overall operation of the host computer. In one embodiment, processor 1004 accomplishes this by executing software or firmware stored in memory 1002. Processor 1004 may be or include one or more programmable general-purpose or special-purpose microprocessors, digital signal processors (DSPs), programmable controllers, application-specific integrated circuits (ASICs), programmable logic devices (PLDs), graphics processing units (GPUs), or the like, or a combination of such devices.

[0076] The memory 1002 can be or include the main memory of a computer system. The memory 1002 represents any suitable form of random access memory (RAM), read-only memory (ROM), flash memory, or the like, or a combination of such devices. In use, the memory 1002 may contain, among other things, a set of machine instructions that, when executed by the processor 1004, cause the processor 1004 to perform operations to implement certain aspects of the techniques of this disclosure.

[0077] While this patent document contains many details, these should not be construed as limitations on the scope of any invention or claimed subject matter, but rather as descriptions of features that may be specific to particular embodiments of a particular invention. Certain features described in this patent document in the context of separate embodiments can also be implemented in combination within a single embodiment. Conversely, various features described in the context of a single embodiment can also be implemented within multiple embodiments, either separately or in any suitable subcombination. Furthermore, while features may be described above as operative in a combination and even initially claimed as such, one or more features from a claimed combination can, in some cases, be excluded from the combination, and the claimed combination may be directed to subcombinations or variations of subcombinations.

[0078] Similarly, although operations are depicted in the figures in a particular order, this should not be understood as requiring such operations to be performed in the particular order or sequential order shown, or that all illustrated operations be performed to achieve a desirable result. Further, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[0079] It should be understood that the various disclosed embodiments may be implemented, individually or collectively, within devices comprised of various optical components, electronic hardware, and / or software modules and components. These devices may comprise, for example, processors, memory units, and interfaces communicatively coupled to one another and may range from desktop and / or laptop computers to mobile devices and the like. The processors and / or controllers may perform the various disclosed operations based on the execution of program code stored on a storage medium. The processors and / or controllers may, for example, communicate with at least one memory and at least one communication unit, directly or indirectly through a communication link, that enables the exchange of data and information with other entities, devices, and networks. The communication unit may provide wired and / or wireless communication capabilities according to one or more communication protocols and, therefore, may comprise appropriate transmitter / receiver antennas, circuitry and ports, and encoding / decoding capabilities that may be necessary for the proper transmission and / or reception of data and other information. For example, the processor may be configured to receive electrical signals or information from a disclosed sensor (e.g., a CMOS sensor), process the received information, and generate an image or other information of interest.

[0080] Various information and data processing operations described herein may, in one embodiment, be implemented by a computer program product embodied in a computer-readable medium, including computer-executable instructions, such as program code, executed by computers in a networked environment. Computer-readable media may include removable and non-removable storage devices, including, but not limited to, read-only memory (ROM), random access memory (RAM), compact discs (CDs), digital versatile discs (DVDs), and the like. Accordingly, computer-readable media, as described herein, comprise non-transitory storage media. Generally, program modules may include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Computer-executable instructions, associated data structures, and program modules represent examples of program code for executing steps of the methods disclosed herein. The particular sequence of such executable instructions or associated data structures represents examples of corresponding acts for implementing the functions described in such steps or processes.

[0081] Only some implementations and examples are described; other implementations, improvements, and modifications can be made based on what is described and illustrated in this patent document.

Claims

1. An occlusion-enabled optical see-through head-mounted display (OCOST-HMD), a polarizing element configured to receive light from a real scene and to generate polarized light at its output; A polarizing beam splitter (PBS); An objective lens; A spatial light modulator (SLM); An eyepiece and A quarter wave plate (QWP); A micro display, a reflective optical element configured to reflect substantially all or a portion of light incident thereon in a first direction and to pass substantially all or a portion of light received from the microdisplay that is incident thereon from a second direction; Equipped with the SLM and the objective lens form a first double-pass configuration that allows at least a portion of light passing through the objective lens to be reflected from the SLM based on an occlusion mask corresponding to a virtual image presented on the microdisplay and to propagate again through the objective lens; The eyepiece and the reflective optical element form a second double pass configuration that allows at least a portion of the light passing through the eyepiece to be reflected from the reflective optical element and propagate through the eyepiece again, OCOST-HMD.

2. the PBS is positioned to receive the polarized light and reflect the polarized light towards the objective lens; The OCOST-HMD of claim 1, wherein the PBS is positioned to receive light output from the first double-pass configuration, transmit it through the PBS toward the eyepiece, and reflect light that the PBS receives from the second double-pass configuration, including light from the microdisplay, toward the position of a human eye.

3. a first reflective surface; The PBS is receiving the polarized light and transmitting the polarized light toward the objective lens; receiving light output from the first double pass arrangement and reflecting it towards the eyepiece; reflecting light received by the PBS from the second double pass configuration, including light from the microdisplay, toward the first reflective surface; 2. The OCOST-HMD of claim 1, wherein the first reflective surface is positioned to reflect light incident thereon toward a position of a human eye.

4. The OCOST-HMD of claim 1, 2 or 3, wherein the SLM is configured to modulate light incident thereon.

5. The OCOST-HMD of claim 1, further comprising an occlusion mask corresponding to a virtual image presented on the microdisplay, the occlusion mask being used to effect modulation of one or more regions of the SLM.

6. The OCOST-HMD of claim 1 , wherein the reflective optical elements are positioned on a surface of the microdisplay.

7. The OCOST-HMD of claim 1 , wherein the QWP is positioned between the eyepiece lens and the reflective optical element.

8. The OCOST-HMD of claim 1 , wherein the QWP is positioned between the eyepiece and the PBS.

9. The OCOST-HMD of claim 1 , wherein the OCOST-HMD is configured to generate an orthogonal image without the use of a roof prism.

10. The OCOST-HMD of claim 1, wherein the OCOST-HMD provides a pupil matching optical configuration that maps the user's pupil, i.e., the relay pupil, back to the user's eye position, allowing correct parallax to be maintained.

11. The OCOST-HMD of claim 1 , wherein the OCOST-HMD is configured to generate a field of view (FOV) that is not limited in at least one direction by the eyepiece.

12. The OCOST-HMD of claim 1, wherein the OCOST-HMD has a field of view (FOV) of greater than 40 degrees diagonally and optical performance of better than 20% modulation contrast across the full FOV, or (b) a see-through field of view (FOV) of 90 degrees by 40 degrees with an angular resolution of 1.0 arc minute.

13. The OCOST-HMD of claim 1 , wherein at least a portion of the OCOST-HMD corresponds to a set of two afocal 4f relays that image an entrance pupil to a conjugate intermediate pupil location.

14. The OCOST-HMD of claim 1 , wherein the OCOST-HMD forms a single-layer, double-pass, pupil-aligned OCOST-HMD.

15. (a) an objective lens group including the objective lens; or (b) an eyepiece group including the eyepiece The OCOST-HMD of claim 1 , comprising one or both of the following:

Images