Pupil-matched optical see-through head-mounted display with occlusion capability

Abstract

A compact occlusion-capable optical see-through head-mounted display (OCOST‑HMD) having dual surround paths capable of rendering pixel-by-pixel mutual occlusion between virtual and real views and correct perspective viewing angle or pupil-matched viewing is described. An example device includes a polarizer, a polarizing beam splitter, an objective lens, a spatial light modulator (SLM), an eyepiece, a quarter wave plate, and a reflective optical element configured to reflect light incident thereon in a first direction and transmit light received from a microdisplay incident thereon from a second direction. These components form a first dual pass configuration that allows light that passes through the objective lens to reflect from the SLM and propagate through the objective lens again, and a second dual pass configuration that allows light that passes through the eyepiece to reflect from the reflective optical element and propagate through the eyepiece again.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATION(S)

[0002] This application claims priority to provisional application serial number 62 / 900,204, entitled “Pupil Matched Occlusion-Capable Optical See-Through Head-Mounted Display,” filed on September 13, 2019. The entire contents of the above provisional application are incorporated by reference as part of the disclosure of this document. Technical Field

[0003] The technology in this document relates generally to head-mounted displays, and particularly to head-mounted displays with a wide field of view. Background Art

[0004] See-through head-mounted displays (HMDs) are one of the key enabling technologies for merging digital information with physical scenes in augmented reality systems. While both video see-through displays and optical see-through displays have their unique advantages, optical see-through HMDs (OST-HMDs) are often preferred when it comes to real-scene resolution, viewpoint differences, field of view (FOV), and image latency. Conventional optical see-through head-mounted displays typically rely on optical combiners such as beam splitters or diffraction gratings to evenly combine real-world light with light from virtual objects. In addition to the added expense, extra weight, additional alignment considerations, and increased footprint of existing systems, the FOV of conventional systems is also limited by the typically narrow see-through objective optics. Summary of the Invention

[0005] The techniques disclosed herein can be implemented in various embodiments to realize an optical architecture for a compact occlusion-capable OCOST-HMD design that includes, among other features and benefits, a wide field of view (FOV) that is not limited by objective lens optics. The disclosed methods, systems, and devices utilize dual surround paths and provide an OCOST-HMD capable of rendering pixel-by-pixel mutual occlusion between virtual and real views, correct perspective viewing angles or pupil-matched viewing, and a very wide perspective FOV.

[0006] One aspect of the disclosed embodiments relates to an optical see-through head-mounted display (OCOST-HMD) with occlusion capability, comprising a polarizing element configured to receive light from a real scene and produce polarized light at its output, a polarizing beam splitter (PBS), an objective lens, a spatial light modulator (SLM), an eyepiece, a quarter-wave plate (QWP), and a reflective optical element configured to reflect substantially all or a portion of light incident thereon in a first direction and transmit substantially all or a portion of light received from a microdisplay incident thereon in a second direction. The SLM and the objective lens form a first double-pass configuration that allows at least a portion of light that passes through the objective lens to reflect from the SLM and propagate again through the objective lens. The eyepiece and the reflective optical element form a second double-pass configuration that allows at least a portion of light that passes through the eyepiece to reflect from the reflective optical element and propagate again through the eyepiece. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1A Figure 2. Schematic diagram of the previously proposed two-layer folded OCOST-HMD optical design.

[0008] Figure 1B Illustrated is a schematic diagram of a single-layer OCOST-HMD optical system according to example embodiments.

[0009] Figure 1C Another configuration of a single-layer OCOST-HMD optical system according to example embodiments is illustrated.

[0010] Figure 2 Pictured Figure 1B Example of the deployed layout of the optical elements of a portion of the OCOST-HMD system.

[0011] Figure 3 Illustrated is the layout of the optical components of a prototype OCOST-HMD according to example embodiments.

[0012] Figure 4A Illustrated is a modulation transfer function (MTF) graph of a virtual display path of an example OCOST-HMD system in accordance with the disclosed embodiments.

[0013] Figure 4B Illustrated is an MTF plot of the spatial light modulator (SLM) path of an example OCOST-HMD system in accordance with the disclosed embodiments.

[0014] Figure 4C Illustrated is an MTF plot of the see-through path of an example OCOST-HMD system in accordance with the disclosed embodiments.

[0015] Figure 5Illustrated is a distortion grid of the see-through optical path of an example OCOST-HMD system over a full 40 degree diagonal field of view (FOV) in accordance with the disclosed embodiments.

[0016] Figure 6 Illustrated are example OCOST-HMD layouts and prototypes according to the disclosed embodiments.

[0017] Figure 7 A illustrates an example augmented view of the real world and a virtual scene without occlusion capabilities enabled.

[0018] Figure 7 B illustrates an example real-world scene when an occlusion mask is displayed on the SLM but no virtual content is shown on the OLED display.

[0019] Figure 7 C illustrates an example view captured with a mask on the SLM and a virtual scene displayed on the OLED display, where the virtual content is inserted between two real objects.

[0020] Figure 8 Measured on-axis MTF plots of an SLM, OLED, and optical see-through path and camera are shown according to example embodiments.

[0021] Figure 9A Example graphs illustrating image contrast degradation of a virtual object for different background scene luminance values, with occlusion disabled, are shown.

[0022] Figure 9B Shown are example graphs illustrating image contrast degradation of a virtual object for different background scene luminance values ​​with occlusion enabled.

[0023] Figure 10 Illustrated is a block diagram of a device that can be used to implement certain aspects of the disclosed technology. DETAILED DESCRIPTION

[0024] The disclosed embodiments relate to a novel optical architecture for a compact occlusion-capable optical see-through head mounted display (OCOST-HMD). Among other features and benefits, the OCOST-HMD utilizes an optical architecture with a double-wrapped path and provides an OCOST-HMD solution capable of rendering pixel-by-pixel mutual occlusion between virtual and real views, correct perspective or pupil-matched viewing, and a very wide perspective field of view (FOV). For illustrative purposes, a detailed example implant and an example prototype system are described. The prototype provides a virtual display FOV greater than 40 degrees diagonally and a resolution of 1920x1200 pixels, with optical performance of greater than 20% modulation contrast across the full FOV. Furthermore, a wide perspective FOV of 90° by 40° can be achieved with an angular resolution of 1.0 arc minute, equivalent to 20 / 20 vision.

[0025] Developing an OST-HMD presents many technical challenges, one of which lies in correctly rendering the light blocking behavior between digital and physical objects in space, also known as mutual occlusion. Occlusion in augmented reality displays is a powerful depth cue; without it, significant misinterpretations can occur, such as incorrect color registration, reduced image contrast, and discrepancies in object placement. To achieve correct mutual occlusion, opaque virtual objects in the foreground should appear solid and occlude real objects in the background, and vice versa. When blending virtual and real objects, opaque virtual objects should appear fully opaque and occlude real objects behind them, while real objects should naturally obscure the view of virtual objects behind them. Therefore, there are two types of occlusion: occlusion where real scene objects occlude virtual objects, and occlusion where virtual objects occlude the real scene. Occlusion of virtual objects by real objects can often be achieved in a straightforward manner, for example, by simply not rendering the virtual object where the real object is located when the position of the real object relative to the virtual scene is known. Therefore, in this case, occlusion of virtual objects by real objects can be achieved simply by not rendering the known digital overlap region of the virtual object that is occluded by the virtual object. However, occlusion of real objects by virtual objects poses a much more complex problem, as it requires blocking light in the real scene.

[0026] Over the past decade, a handful of OCOST-HMD concepts have been proposed, and even fewer designs have been prototyped. Existing approaches for implementing OCOST-HMDs fall into two categories: direct light blocking and pixel-by-pixel modulation. Direct light blocking selectively blocks light rays from a see-through scene without focusing them. This can be achieved by selectively modifying the reflective properties of physical objects or by passing light from the real scene through a single- or multi-layer spatial light modulator (SLM) placed directly near the eye. For example, some existing systems have explored creating natural occlusion of virtual objects by physical objects via head-mounted projection display (HMPD) devices using retroreflective screens on non-occluding physical objects, but this can only be used in limited settings. Other existing systems have explored occlusion capabilities using a transmissive SLM placed directly near the eye without imaging optics. If the eye were a pinhole aperture that allowed a single ray from each real-world point to reach the retina, direct light blocking via the SLM would be a straightforward and sufficient solution. Instead, the eye has a regional aperture, making it nearly impossible to block all light rays from an object from the eye using a single-layer SLM without blocking light from other surrounding objects.

[0027] Some existing systems include a lensless computational multi-layer OST-HMD design, which consists of a pair of stacked transmissive SLMs, a thin and transparent backlight, and a high-speed optical shutter. A multi-layer computational light field approach can be used to generate multiple occlusion patterns so that the occlusion light field of the see-through view can be correctly rendered. Although the multi-layer light field rendering approach can theoretically overcome some limitations of the single-layer light blocking approach, it suffers from several major limitations, such as significantly degraded see-through views, limited accuracy of the occlusion mask, and low light efficiency. The unfavorable results can be attributed to the lack of imaging optics, the low light efficiency of the SLM, and most importantly, the severe diffraction artifacts caused by the fine pixels of the SLM located at a close distance from the eye pupil.

[0028] The disclosed embodiments can be categorized as pixel-by-pixel occlusion methods ( Figure 2 fall into this category); Figure 2As shown, a focused image of the see-through view is formed at the modulation plane, where the SLM interpolates and renders an occlusion mask to selectively block the real-world scene point by point. One existing per-pixel modulation approach includes prototypes from the ELMO series, developed in the early 2000s. The ELMO-4 prototype consists of four lenses, two prisms, and three optical mirrors, forming a dual afocal 4f relay, arranged in a ring configuration to present a pupil-matched, upright, occlusion-capable view of the real scene. Pupil-matched (also known as perspective-corrected) occlusion maps the user's pupil or the relay's pupil back to the user's eye position, thereby maintaining the correct viewpoint disparity. Limited by the microdisplay and SLM technology of the time, the ELMO prototype had fairly low resolution for both the see-through and virtual display paths, both using a 1.5-inch QVGA (320x240) transmissive LCD module. This ring structure has been further recreated using a 3D-printed housing with a modern transmissive LCD and microdisplay. However, this ring-shaped folded architecture results in a very bulky package that blocks most of the user's face. Furthermore, utilizing a transmissive liquid crystal display (LCD) as an SLM when coupled with a beam splitter provides very low light efficiency (<20%).

[0029] Some systems have attempted to overcome these limitations by proposing an x-cube prism that utilizes polarization-based optics combined with a reflective SLM for coupling the two optical paths to achieve a more compact form factor and high optical efficiency. Although this design appears to be a promising solution to the occlusion problem of OST-HMDs, the solution fails to erect the see-through horizontal view, resulting in incorrect perspective. Another system proposed a complex two-layer folded optical system using a prism and a reflective SLM to create a compact, high-resolution OCOST-HMD. By utilizing a reflective liquid crystal on silicon (LCoS) device as the SLM, the system allows for high illumination throughput and high optical resolution for both the virtual path and the see-through path. This optical design and preliminary experiments demonstrated some potential for the desired form factor and high optical performance. Based on the same two-layer folded optical device architecture, several systems have been designed and built to provide high-resolution OCOST-HMD working prototypes using off-the-shelf optical components. Capable of rendering pixel-by-pixel mutual occlusion, this prototype utilizes an OLED microdisplay for the virtual display path coupled with a reflective LCoS SLM as the see-through path to achieve an occlusion-capable OST-HMD with a 30-degree diagonal FOV and a 1920x1080 pixel resolution. However, the two-layer folded optics architecture cannot provide pupil matching or a correct viewing angle because the pupil is relayed in front of the user's eye. Furthermore, like the original ring-shaped architecture, the see-through FOV of the two-layer folded architecture is limited to whatever the see-through objective optics can accommodate, and is therefore typically narrow. This presents a significant limitation for many applications where access to a wider peripheral field of view is crucial.

[0030] Therefore, state-of-the-art OST-HMDs lack the ability to selectively block real-world light from reaching the eye. As a result, digitally rendered virtual objects viewed through an OST-HMD often appear “ghost-like” (or translucent), always floating “in front” of the real world. Figure 7 A provides an illustrative example of this ghost-like condition.

[0031] Among other features and benefits, the disclosed embodiments provide an optical system based on pixel-by-pixel modulation that helps overcome the aforementioned limitations of current state-of-the-art OCOST-HMD systems. The disclosed embodiments achieve this goal at least in part by employing a dual-pass optical architecture for a compact, high-resolution OCOST-HMD system.

[0032] Implementations of the disclosed embodiments include an OCOST-HMD system comprising three superimposed optical paths, namely: a display path, an SLM path, and a see-through path. The disclosed optical architecture provides an OCOST-HMD solution capable of rendering pixel-by-pixel mutual occlusion between virtual and real views, correct see-through viewing angles or pupil-matched viewing, and a very wide see-through field of view.

[0033] It should be noted that, in various embodiments, the central portion of the see-through FOV can be occluded to create wide peripheral vision. Based on the disclosed optical design architecture and custom optimization of the optical system, we experimentally demonstrated a wearable stereoscopic prototype system utilizing an OLED microdisplay for the virtual display path coupled to a reflective LCoS SLM as an optical mask for the see-through path. It should also be noted that OLED is one example of a microdisplay used to construct the prototype, and other display technologies can be implemented based on the disclosed embodiments. The prototype provides a virtual display FOV greater than 40 degrees diagonally and a resolution of 1920x1200 pixels, with optical performance of greater than 20% modulation contrast across the full FOV. Furthermore, the system is capable of achieving a wide see-through FOV of 90° by 40° with an angular resolution of 1.0 arcminute, equivalent to 20 / 20 vision. With a total weight of approximately 300 grams and dimensions of 140 mm (height) x 130 mm (width) x 25 mm (depth), the prototype is capable of achieving a dynamic range greater than 100:1 in well-lit environments. The optical performance of the OST-HMD is further compared with and without occlusion to demonstrate the degradation of virtual content contrast of the non-occluded device in a bright environment. It should be noted that the disclosed prototype is provided for the purpose of illustrating an example implementation of the OCOST-HMD system according to the disclosed embodiments.

[0034] Figure 1A The figure shows a schematic diagram of the previously proposed two-layer folded OCOST-HMD optical design; Figure 1B An OCOST-HMD optical system according to an example embodiment of the disclosed embodiments is illustrated. Figure 1B The system in

[15] is part of a double-pass single-layer architecture designed with OCOST-HMD that provides pupil matching.

[0035] exist Figure 1A In the system, two folding mirrors 108, 124, a roof prism 104 and a PBS 112 are used to fold the optical path into a two-layer optical architecture. Figure 1AThe system can be more simply decomposed into two mechanisms (objective lens 106 and eyepiece 122) to form the basis of the Kepler telescope. Furthermore, the light passing through the system can be categorized into two distinct components: the virtual display and the real-world view. The light path for the real-world view is shown with an ellipse surrounding an arrow, while the light path for the virtual display does not include an arrow and consists of a path from OLED display 120 to first 45-degree surface mirror 108 and to eyepiece 122. Figure 1A The simplified architecture shown in presents a series of design constraints and optical issues.

[0036] First, placing the folded objective lens path in front of the eyepiece 122 as a second layer, while using all glass stock lenses 106, 110, increases the optical form factor by twice the thickness, resulting in a relatively bulky and heavy system design. Next, a polarizing beam splitter (PBS) 112 combines the optical paths of the modulated real view 102 and the virtual view associated with the virtual content 118 so that they share the same eyepiece 122 module to observe both the virtual display and the modulated real-world view (observing the combined image at eye position 126). Since the system is built like a simple telescope, this results in a 1:1 conjugate coupling of the virtual display and real-world view paths to ensure unity magnification of the real scene, forcing the optical mask 116 and the display 120 to share the same optical requirements. However, this creates a problem because the SLM 114 and the microdisplay 120 are often implemented using different technologies and do not share the same technical specifications, resulting in a mismatch in optical performance. Another limitation of this arrangement is that the see-through FOV is limited to the FOV imaged by the telescope system, resulting in tunnel vision of the see-through path, which is undesirable for augmented reality (AR) systems. Figure 1A The configuration can be considered a single afocal 4f system. Much like a telescope, this single afocal 4f design causes the conjugate pupil to shift to the other side of the optical relay, resulting in an incorrect viewpoint offset. This incorrect viewing perspective causes objects to appear closer than they actually are, creating a depth discrepancy between the object, the hand, and the eye. Furthermore, the inclusion of roof prism 104 to produce an upright image increases the cost, size, and weight of the system.

[0037] Figure 1B A schematic diagram of a single-layer OCOST-HMD optical system according to an example embodiment is shown. The system uses a PBS 136, an objective lens group 138 (which may include one or more lenses), and an eyepiece lens group 144 (which may include one or more lenses). Due to the second pass through the objective lens 138 and the eyepiece lens 144, the objective lens group 138 and the eyepiece lens group 144 also form a double relay lens group, as shown in FIG. Figure 2 As further illustrated. Figure 1BIt can be seen that this configuration provides a compact form factor and achieves pixel-by-pixel occlusion capability due to the small number of components and associated folding paths. This dual-pass system allows for a single optical layer and is compatible with Figure 1A Compared to the configuration of OCOST-HMD, the use of heavy optical devices (such as first surface mirrors and roof prisms) is abandoned to create a greatly reduced form factor and light weight wearable HMD design. This can be explained by considering that when the observer of OCOST-HMD looks through the eyepiece 144, the observer only receives the Figure 1B To further understand the term single layer, we can refer to a single layer or a single set of optical devices in a system. Figure 1A In

[15] , the observer will receive light from two sets of optical components, one from each of the real-world view and the virtual content.

[0038] By splitting the light path into three, Figure 1B , the three optical paths are: display path (square), SLM path (ellipse), and see-through path (circle). A single PBS 136 acts as both a folding element and a combiner to merge the optical paths of the modulated real view 132 (see-through and SLM paths) and the virtual view from the virtual content 152 (display path), so that the same eyepiece 144 and relay module are shared for viewing both the virtual display and the modulated real world view. Unlike Figure 1A Configuration, Figure 1B The dual-pass architecture utilizes a repeater to allow decoupling of the SLM path and the display path by allowing the objective lens 138 and the eyepiece 144 to have different focal length parameters (no longer needing to be 1:1 conjugated) to achieve unity magnification of the real scene while maintaining unity magnification of the entire system. This decoupling of the different optical paths allows component paths to be specifically designed for different display technologies.

[0039] Figure 1B Each of the three paths in the display path, the SLM path, and the see-through path provides its own optical performance. Light from the see-through path identified by a circle passes through a retarder (e.g., polarizer 134). In this example configuration, the retarder polarizes the light in an S orientation. The S polarized light is reflected from the PBS 136. The objective lens 138 (or lens group) then collects the light from the physical environment and forms an intermediate image at its focal plane (where an amplitude-based reflective SLM (e.g., LCoS) 140 is placed). The SLM 140 can manipulate the polarization orientation to render an occlusion mask 142 for controlling the opacity of the real view. For example, the SLM 140 can generate an on-off modulation scheme in which light incident on the SLM 140 is selectively reflected (SLM off) or allowed to transmit therethrough on a pixel-by-pixel basis. Other example modulation schemes can also be implemented.

[0040] In the above example, light reflected from the SLM / LCoS 140 is flipped to a P orientation in the SLM path (circles), then passed back through the same objective lens 138, through the PBS 136, and through the eyepiece 144, thereby forming an optical relay. The light is then transmitted (or otherwise delayed) through a quarter wave plate (QWP) 146 to produce light having a right-hand circular (RHC) polarization direction that is incident on a half mirror 148. The half mirror 148 is a non-limiting example of a reflective surface that allows substantially all or a portion of light incident thereon from a first direction (e.g., Figure 1B is reflected back while allowing substantially all or a portion of the light incident thereon from a second direction (e.g., Figure 1B In a configuration of , light traveling downward from the microdisplay 150 (e.g., OLED) is reflected by the .sub.2 element passing through it. Such an element can be designed, for example, as a multilayer thin film stack as part of the microdisplay, or as a separate component.

[0041] Upon reflection from the "half-mirror" 148, the RHC polarization is flipped to left-hand circular (LHC) polarization; the LHC light then passes back through the QWP 146 and the eyepiece 144, where it is converted back to S-polarized light, which is then reflected by the PBS 136 into the user's eye at eye position 154. In the display path (squares), unpolarized light from the OLED display 150 is imaged by the shared eyepiece 144, where the S-polarized light is coupled through the PBS 136 into the user's eye to give three high-resolution superimposed images: the optical mask 142, the real scene 132, and the virtual scene 152. Instead of using a roof prism (such as Figure 1A The even number of odd-even switches ensures that Figure 1B The perspective image in the configuration is upright and upright, which is crucial for the OCOST-HMD system because it cannot be changed digitally.

[0042] Figure 1C Another configuration of a single-layer OCOST-HMD optical system according to example embodiments is illustrated. Figure 1C With Figure 1B Components are similarly configured for combining the real view 162 and virtual content 182, except that the polarizer 164 modifies the light from the real scene 162 to produce P-polarized light that passes through the PBS 166 and is incident on the SLM 170 that implements the mask 188 based on the desired modulation. Figure 1B In contrast, the positions of some components are rearranged to provide a single layer configuration. When providing a combined view at eye position 188 after reflection from reflective surface 186, Figure 1C The configuration provides Figure 1B configurations have similar features and benefits.

[0043] It should be noted that although Figure 1B and Figure 1C The QWP 146, 176 is shown positioned between the eyepiece 144, 174 and the microdisplay 150, 180 (or between the eyepiece 144, 174 and the semi-reflective mirror 148, 178). However, the QWP 146, 176 can be positioned on either side of the eyepiece 144, 174. Thus, in some embodiments, the QWP 146, 176 is positioned between the PBS 136, 166 and the eyepiece 144, 174.

[0044] In contrast to the two-layer design, the disclosed double-pass single-layer architecture allows the optical path to be correctly pupil-matched. To correctly determine the pupil position, we can view the optical system as a set of dual afocal 4f relays that image the entrance pupil to a conjugate intermediate pupil position. Figure 2 Shown Figure 1B Example of the deployed layout of optical elements in the OCOST-HMD system ( Figure 2 Only the unfolding path is shown in FIG. 1 ). It can be seen that the first afocal relay (left side) includes an eyepiece (labeled as primary eyepiece 154) and a first system relay lens (the left lens in the relay group 208, labeled as secondary eyepiece 154), which is the eyepiece itself (the light passing through the eyepiece is reflected from the OLED (microdisplay 150) and passes through the eyepiece (secondary eyepiece 154) for the second time, as shown in FIG. Figure 1B ). Figure 2 The second afocal relay in the embodiment includes a second system relay lens (the right lens in the relay group, the secondary objective lens 138) and an objective lens (the main objective lens 138); the second system relay lens is identical to the objective lens (the light passing through the objective lens is reflected from the LCoS 140 and passes through the objective lens for the second time, as shown in FIG. Figure 1B (as shown at the bottom of the ). Figure 2 The locations of the entrance pupil 202, the intermediate pupil 210, and the exit pupil 216 are further illustrated.

[0045] By comparison Figure 1A configuration (where the FOV is limited by the FOV of the objective lens) with Figure 1B Improvements in achievable FOV can be observed with configurations where the FOV can be significantly increased. In particular, the see-through FOV in the horizontal direction (i.e., the direction into the paper) can become essentially unlimited and limited only by the extent of the eyepiece and objective (or the size of the beam splitter). This enhancement in the horizontal FOV can be beneficially used to provide improved peripheral viewing capabilities for OCOST-HMD systems.

[0046] Due to the choice of reflective LCoS SLM, Figure 1BThe example configuration requires image space telecentricity of the SLM path to achieve high contrast. Therefore, the distance from the intermediate pupil x1 to the objective must be equal to the focal length of the objective (f obj ). Knowing that we have a two-pass system, we can assume that f obj and the second relay lens focal length (f r2 ) are equal, allowing the system's exit pupil to be relayed back to itself. We can then calculate the position of the intermediate pupil with respect to the system's focal length and the entrance pupil as equation (1).

[0047]

[0048] where f eye and f r1 are the corresponding focal lengths of the eyepiece and the first relay lens, E relief is the exit pupil distance, and l is the distance to the intermediate pupil after the first relay lens. Again, since the system utilizes dual optical pathways, we can set f eye Equal to f r1 to obtain the greatly simplified equation (2).

[0049] l=2f eye +E relief (2).

[0050] In order for the system to be properly pupil matched, the distance after the first lens group must be equal to the exit pupil distance (ER) of the system, l = E relief Then, this relationship is proved by E relief =f eye Or in other words, the display path must also be telecentric to achieve proper pupil matching and correct viewing angle. We can then determine the image quality based on the half FOV (θ half ) in order to get the proper eye clearance (E clearance ) required principal plane (P p ) position is equation (3).

[0051]

[0052] Among them D ep is the diameter of the entrance pupil.

[0053] Example Prototyping :based on Figure 1B Based on the optical layout in [1] and the analytical relationships described in equations (1)-(3) above, an example custom prototype system was designed. The key specifications of the prototype system are summarized in Table 1 below.

[0054]

[0055]

[0056] Table 1. Specifications of the example system

[0057] One of the key parameters driving the example design is the choice of display technology. We selected a 0.85" Emagin OLED microdisplay for the virtual display path. With an active area of ​​18.4 mm and 11.5 mm and an aspect ratio of 8:5, the eMagin OLED offers a pixel size of 9.6 microns at a native resolution of 1920x1200 pixels. Based on this microdisplay, we aimed to achieve an OCOST-HMD prototype with a diagonal FOV >40°, or a horizontal 34° and vertical 22° FOV, and an angular resolution of 1.06 arc minutes per pixel, corresponding to a Nyquist frequency of 53 cycles / mm in the microdisplay space or 28.6 cycles / degree in the visual space. Additionally, for the SLM path, we used a 0.7" reflective LCoS from a projector. The reflective SLM was chosen due to its significant advantages in light efficiency, contrast, and low diffraction artifacts, which are common in transmissive SLMs used in previous work.

[0058] The chosen LCoS offers a native resolution of 1400x1050 pixels, a pixel pitch of 10.7 microns, and an aspect ratio of 4:3. Based on the different display specifications of the SLM, we aim to achieve an optical mask diagonal FOV of >42.5°, or a horizontal and vertical FOV of 34° and 25.5°, and an angular resolution of 1.45 arc minutes per pixel, corresponding to a Nyquist frequency of 47 cycles / mm in SLM space or 19.66 cycles / degree in visual space. Furthermore, our system requires an objective lens focal length of 24.4 mm and an eyepiece focal length of 29.8 mm, resulting in a relay magnification ratio of 1:1.22. To allow for approximately ±25° of eye rotation within the eye socket without vignetting, we set an exit pupil diameter (EPD) of 10 mm. An eye relief distance of 20 mm is used to accommodate most head shapes.

[0059] To achieve high optical performance across the three optical paths, we optimized the system using three zoom configurations, each corresponding to a different optical path and design specifications. Figure 3 The lens layout of the final OCOST-HMD prototype is shown. The optical path of the virtual display (eyepiece) is indicated by rays designated by dashed rectangles, while the optical path of the SLM (relay + eyepiece) is shown by rays designated by dashed ellipses, and the see-through path (objective + relay + eyepiece) is indicated by rays designated by dashed circles. It should be noted that the see-through path overlaps with the microdisplay and SLM paths after the PBS, and therefore only the virtual display rays are traced to the eye pupil.

[0060] Overall, Figure 3The final lens design for the prototype example in consists of 10 lenses, including 5 stock crown glass lenses, 3 acrylic aspheric lenses, and 2 custom flint glass lenses. The lenses labeled 2 to 6 form the eyepiece group. The lenses labeled 8 to 12 form the objective group. The half-mirror is shown as the gray element between the quarter wave plate and the OLED 7. Polarization is manipulated using a wire grid polarizer and QWP film together with a single custom PBS from Moxtek. Based on the dominant wavelength of the OLED microdisplay, the system is optimized for 3 wavelengths of 430, 555, and 625 nm with weights of 1, 2, and 1, respectively. To ensure that the system is properly pupil-matched for the correct viewing angle according to equations (1) and (2), the objective and eyepiece are optimized to have a chief ray deviation of less than ±0.5°, demonstrating image space telecentricity. We cropped the eyepiece lenses to achieve a 20 mm eye relief and a 10 mm EPD.

[0061] Table 2-9 provides the optical schemes for the virtual display path and the see-through path of the prototype system described above. Both optical paths are ray traced from the system's exit pupil, which coincides with the eye's entrance pupil. The term "aspheric" in the table refers to an aspheric surface that can be expressed by the following equation:

[0062]

[0063] where z is the surface sag measured along the z-axis of the local x, y, z coordinate system, c is the vertex curvature, r is the radial distance, k is the conic constant, and A through E are the 4th, 6th, 8th, 10th, and 12th order deformation coefficients, respectively. Tables 4 through 9 provide the aspheric coefficients for aspheric surfaces 11, 12, 15, 16, 23, and 24, respectively.

[0064]

[0065] Table 2: System solution for virtual display path

[0066]

[0067]

[0068]

[0069] Table 3: System solutions for perspective paths

[0070] Y radius -18.44 Cone constant (K) -0.356 4th order coefficient (A) -6.23E-07 6th order coefficient (B) -4.35E-08 8th order coefficient (C) 1.02E-10 10th order coefficient (D) -1.67E-13 12th order coefficient (E) -3.43E-15

[0071] Table 4. Surface scheme for surface No. 11 of Table 2.

[0072]

[0073]

[0074] Table 5. Surface scheme for surface No. 12 of Table 2.

[0075] Y radius 8.89 Cone constant (K) 0.042 4th order coefficient (A) 1.69E-05 6th order coefficient (B) 2.84E-07 8th order coefficient (C) -5.89E-09 10th order coefficient (D) 4.47E-11

[0076] Table 6. Surface scheme for surface No. 15 of Table 3.

[0077] Y radius -103.15 Cone constant (K) -90.09 4th order coefficient (A) 0.0001295 6th order coefficient (B) -1.45E-06 8th order coefficient (C) 1.65E-08 10th order coefficient (D) -1.01E-10

[0078] Table 7. Surface solution for surface No. 16 of Table 3.

[0079] Y radius 16.16 Cone constant (K) -19.04 4th order coefficient (A) 0.000273 6th order coefficient (B) -5.03E-06 8th order coefficient (C) 5.25E-08 10th order coefficient (D) -2.78E-10

[0080] Table 8. Surface solution for surface No. 23 of Table 3.

[0081]

[0082]

[0083] Table 9. Surface plan for surface No. 24 of Table 3.

[0084] The simulated optical performance of the dual-pass OCOST-HMD prototype system was evaluated over the full FOV in display space, where the spatial frequency is characterized in cycles per millimeter. In the example provided, the optical performance of the see-through path was limited to a 40° diagonal, based on the field passing through the system and optically superimposed on the virtual and mask images. Light from the real scene outside this field passes only through a single PBS, is not optically affected, and should be seen at the native resolution of the human eye.

[0085] Figures 4A to 4C Multicolor modulation transfer function (MTF) curves for several weighted fields for virtual display, SLM, and see-through paths are shown, evaluated with a 4-mm eye pupil so that it is similar to the human eye. In these figures, the modulation transfer functions of 0, 6, 10, 15, and 20 degrees are evaluated for the lateral (Tan) and radial (Rad) diagonal half FOV with a 4-mm pupil diameter, and for the OCOST-HMD OLED optical path ( Figure 4A ), OCOST-HMD SLM optical path ( Figure 4B ) plotted for a cutoff spatial frequency of 53 cycles / mm and for the OCOST-HMD see-through optical path ( Figure 4C ) plots the cutoff spatial frequency of 110 times / degree. Figure 4A Initially, the OLED display optical performance maintained approximately 40% modulation on average across the entire field at the designed Nyquist frequency of 52 cycles / mm, corresponding to a 9.6 micron pixel size. Figure 4BThe optical performance of the LCoS shown in shows a 50% modulation averaged over the full field at the designed Nyquist frequency of 47 cycles / mm, corresponding to a 10.7 μm pixel size. Figure 4C The see-through optical performance is shown, maintaining an average modulation of 15% at a human eye cutoff frequency of 110 cycles / mm corresponding to 1.0 arc minute resolution or 20 / 20 human visual sensitivity.

[0086] Figure 5 The distortion grid of the see-through optical path over the full 40-degree diagonal FOV is shown, overlaid with a virtual image. In the figure, the actual and paraxial FOVs almost coincide, indicating strong agreement between the two. Based on the optical specifications we saw in Table 1 above, the see-through path has <1% distortion over the full field. This is important because, unlike display distortion, which can be digitally corrected, the see-through path cannot be digitally corrected. The distortion of the microdisplay (virtual image) is kept below 5%, while the distortion of the SLM (optical mask) is kept to 20% for digital correction.

[0087] Along with MTF and distortion, several other metrics are used to characterize the optical performance of the virtual display path, such as wavefront error and spot diagram. Both the SLM and micro-display paths are subject to lateral chromatic aberration and coma to a large extent. This is due to the non-pupilforming telecentric design of the eyepieces utilized in both the SLM and micro-display paths, which does not allow the stop position to be moved to balance off-axis aberrations. Overall, the wavefront aberrations in each of the three paths are sufficiently low, being less than 1 wave. The average root mean square (RMS) spot diameter across the field is 9 microns for both the perspective and display paths, but because the allowed distortion is so large, the average root mean square (RMS) spot diameter jumps to 16.4 microns for the SLM path. Although it appears larger than the 10.7 micron pixel size, this difference is primarily due to lateral chromatic aberration and can be corrected.

[0088] Example system prototype and experimental demonstration

[0089] Figure 6Example OCOST-HMD layouts and prototypes according to the disclosed embodiments are illustrated. Panel (a) shows a front view of a fully assembled OCOST-HMD Solidwords CAD design with reference to an average person's head. Panel (b) shows a side view of the fully assembled OCOST-HMD CAD design, where it demonstrates the greatly reduced form factor due to the double-pass architecture. Since light passes through the same optical path multiple times, the optical performance is sensitive to optical and mechanical tolerances. For the mechanical design, the individual lenses are held by set screws to enable more compensation and reduce the tolerance stack in the mechanical design to meet the minimum MTF requirement. Panels (c) and (d) show the MTF requirements of the OCOST-HMD in the front view, respectively. Figure 3 Front and oblique views of a binocular prototype of the OCOST-HMD system built on the optical design in [1]. A quarter is shown for scale. The overall height and width of the prototype OCOST-HMD system are 130 mm by 140 mm, with a depth of 25 mm and an adjustable intraocular distance of 50-70 mm.

[0090] Figure 7 Shown Figure 3 and Figure 6 Qualitative demonstration of the occlusion capabilities of the prototype OCOST-HMD. A real-world background scene containing common objects (text, cans and boxes with different colors, shapes, and typography) was used to provide several different spatial frequencies and object depths; these objects were contrasted against a well-illuminated white background wall (~300-500 cd / m 2 ) is placed. The virtual 3D scene used in this example is a virtual 3D scene of a simple image of a wild cat. Figure 7 Panels (a) through (c) in the figure show a set of images captured with a digital camera placed at the exit pupil of the eyepiece. The same 16mm camera lens and 1 / 3" Point Grey image sensor with a 3.75µm pixel pitch were used with an increased aperture of 4mm to better match the F / # of the human eye under typical lighting conditions.

[0091] By simply turning on the OLED microdisplay and not applying a modulation mask to the SLM, Figure 7 Panel (a) in Figure 1 shows an augmented view of the real world and virtual scene without occlusion enabled. Without a mask blocking the perspective path, the image of the "wildcat" wearing a cowboy hat (see panel (b) for a better outline of the wildcat) appears washed out, transparent, and unrealistic due to the brightness of the background scene shared with the virtual display. This causes the wildcat's depth to be spatially blurred.

[0092] Panel (b) depicts the opposite situation: a view of the real-world scene when the occlusion mask is displayed on the SLM but no virtual content is shown on the OLED display. This verifies that the mask is able to effectively block the superimposed portion of the see-through view.

[0093] Panel (c) shows a view captured with a mask on the SLM and a virtual scene displayed on the OLED display, where a virtual wildcat is inserted between two real objects, demonstrating the system's mutual occlusion capabilities. In this case, full capabilities and correct depth perception are present along with improved contrast. By knowing the relative position of the WD-40 can (intended to partially occlude the wildcat image), we removed the pixels from the wildcat rendering that corresponded to the projection of the occluding can on the virtual display. Again, the significance of the results is that correct occlusion relationships can be created and used to give an unparalleled sense of depth to virtual images in the OST-HMD.

[0094] The disclosed dual-pass OCOST-HMD system can achieve high optical performance and dynamic range for both real and virtual content, with significantly improved form factor, viewpoint viewing angle, and technical specifications compared to our previous OCOST-HMD design.

[0095] Example Optical Performance Test Results The vertical and horizontal FOVs of the example system were measured for each optical path. The see-through FOV was determined to be approximately 90° horizontally and 40° vertically, while the see-through FOV with occlusion capability was approximately 34° horizontally and 26° vertically. The virtual display had a horizontal FOV of approximately 33.5° and a vertical FOV of approximately 23°, giving a measured diagonal full FOV of 41.6°. Due to our improved dual-pass architecture and the addition of optical relays, LCoS is capable of fully occluding the scene for the virtual display.

[0096] The optical performance of the prototype system was further quantified by characterizing the MTF performance of three optical paths through the prototype. A high performance camera consisting of a nearly diffraction limited 25 mm camera lens from Edmund Optic and a 1 / 2.3" Point Grey image sensor with a 1.55 micron pixel pitch was placed at the exit pupil of the system. It provides an angular resolution of approximately 0.5 arc minutes per pixel, significantly higher than the expected performance of the prototype. Therefore, it was assumed that there was no performance penalty to the MTF caused by the camera. The camera then captured images of a slanted edge target, which was either displayed at a certain angle by the microdisplay or a printed target placed in a see-through view. In order to provide a separable quantification of the performance of the virtual and see-through paths, a virtual image of the slanted edge was taken when the see-through scene was completely blocked by the SLM. Similarly, a see-through image of the target was taken with the microdisplay turned off. Using The software analyzes the captured slanted edge images to obtain the MTF of the corresponding light path.

[0097] Figure 8 The measured on-axis MTF performance of the SLM, OLED, and optical see-through path and camera is shown, along with each individual tilted edge. Due to the difference in magnification between the camera sensor and the pixel pitch of the microdisplay and SLM, The horizontal axis of the MTF measurement is scaled by the difference in pixel magnification between the camera and display and then converted to a spatial frequency defined in cycles / mm in the visual display space. The prototype design was able to achieve a contrast ratio greater than 50% at the virtual display's Nyquist frequency of 53 cycles / mm, and similar performance was achieved for the SLM path. At a cutoff frequency of 110 cycles / mm, corresponding to 1 arc minute, the modulation contrast of the see-through path was approximately 15%. Figure 8 The curve shown in Figure 3 The on-axis curves in are very similar, indicating that the distinguishability of the three optical paths through the occlusion module is almost complete within the originally set design specifications and the case of human observers.

[0098] We measured the image contrast between a virtual display and a real-world scene as a function of real-world scene luminance at different spatial frequencies. A solid grayscale image ranging from black to white in 10 linear steps was displayed on an LCD monitor to create a controlled background scene with varying illuminance from 0 to 350 cd / m². The monitor was placed approximately 10 cm in front of the OCOST-HMD system to simulate an array of real-world scene luminances. Sinusoidal grating patterns with spatial frequencies ranging from 0.88 to 28.2 cycles / degree were displayed on an OLED microdisplay (virtual path) to evaluate the effect of scene luminance on the image contrast of the virtual scene at different spatial frequencies. The degradation of the contrast with the virtual scene was then plotted and compared with both occlusion enabled (the SLM blocked the see-through light) and no occlusion (the SLM passed the see-through light).

[0099] Figure 9A and Figure 9B The contrast of virtual objects with and without occlusion of the perspective path is shown in Figure 2. We can observe that for well-lit ambient illumination above 300cd / m2, there is no occlusion ( Figure 9A ) deteriorates rapidly to zero, while the contrast of virtual objects with real scenes ( Figure 9B The contrast of the occluded virtual target is nearly unchanged at increasing brightness. We further measured the achievable contrast by measuring with a collimated depolarized light source, with and without the system fully occluded. The dynamic range of the occluded system was determined to be greater than 100:1.

[0100] One aspect of the disclosed embodiments relates to an optical see-through head-mounted display (OCOST-HMD) with occlusion capability, comprising a polarizing element configured to receive light from a real scene and produce polarized light at its output, a polarizing beam splitter (PBS), an objective lens, a spatial light modulator (SLM), an eyepiece, a quarter-wave plate (QWP), and a reflective optical element configured to reflect substantially all or a portion of light incident thereon in a first direction and transmit substantially all or a portion of light received from a microdisplay and incident thereon in a second direction. The SLM and the objective lens form a first double-pass configuration that allows at least a portion of light passing through the objective lens to reflect from the SLM and propagate again through the objective lens. The eyepiece and the reflective optical element form a second double-pass configuration that allows at least a portion of light passing through the eyepiece to reflect from the reflective optical element and propagate again through the eyepiece.

[0101] In one exemplary embodiment, the PBS is positioned to receive polarized light and reflect the polarized light toward the objective lens; the PBS is also positioned to receive light output from the first dual-pass configuration and transmit it toward the eyepiece, and reflect light received by the PBS from the second dual-pass configuration, including light from the microdisplay, toward the location of the human eye. In another exemplary embodiment, the OCOST-HMD further comprises a first reflective surface, wherein the PBS is positioned to (a) receive polarized light and transmit it toward the objective lens, (b) receive light output from the first dual-pass configuration and reflect it toward the eyepiece, and (c) reflect light received by the PBS from the second dual-pass configuration (including light from the microdisplay) toward the first reflective surface. In this exemplary embodiment, the first reflective surface is positioned to reflect light incident thereon toward the location of the human eye.

[0102] According to one example embodiment, the SLM is configured to modulate light incident thereon. For example, the SLM is configured to operate in an on-off modulation mode. In another example embodiment, the OCOST-HMD further comprises an occlusion mask corresponding to a virtual image presented on the microdisplay, wherein the occlusion mask is used to achieve modulation of one or more regions of the SLM. In yet another example embodiment, the OCOST-HMD further comprises a microdisplay. In yet another example embodiment, the reflective optical element is positioned on a surface of the microdisplay. According to another example embodiment, the microdisplay comprises an organic light emitting diode (OLED) device.

[0103] In another example embodiment, the QWP is positioned between the eyepiece and the reflective optical element. In one example embodiment, the QWP is positioned between the eyepiece and the PBS. In another example embodiment, the SLM comprises a liquid crystal on silicon (LCoS) device. In yet another example embodiment, the OCOST-HMD is configured to produce an erect image without using a roof prism. In another example embodiment, the OCOST-HMD provides a pupil-matched optical configuration that maps the user's pupil or a relayed pupil back to the user's eye position so that the correct viewpoint disparity can be maintained. According to yet another example embodiment, the OCOST-HMD is configured to produce a field of view (FOV) that is not restricted by the eyepiece in at least one direction.

[0104] In another example embodiment, the OCOST-HMD has a field of view (FOV) greater than 40 degrees diagonally and optical performance of a modulation contrast greater than 20% across the full FOV. In one example embodiment, the OCOST-HMD has a see-through field of view (FOV) of 90 degrees by 40 degrees with an angular resolution of 1.0 arc minute. In yet another example embodiment, at least a portion of the OCOST-HMD corresponds to a set of two afocal 4f relays that image the entrance pupil to a conjugate intermediate pupil position. In another example embodiment, the OCOST-HMD forms a single-layer, double-pass, pupil-matched OCOST-HMD. In some example embodiments, the OCOST-HMD includes one or both of the following: (a) an objective lens set including an objective lens, or (b) an eyepiece set including an eyepiece.

[0105] Another aspect of the disclosed embodiments relates to an optical see-through head-mounted display (OCOST-HMD) with occlusion capabilities, comprising a polarizer that generates polarized light associated with a real scene, a beam splitter (PBS), an objective lens, a spatial light modulator (SLM), an eyepiece, a retarder, and a half-mirror configured to reflect substantially all light associated with an occlusion mask incident thereon in a first direction and transmit substantially all light associated with a virtual scene incident thereon from a second direction. In this configuration, the PBS is positioned to (a) receive the polarized light and direct it toward the SLM, (b) receive light associated with the virtual scene and direct it toward a location for viewing by a user's eyes, and (c) receive light associated with the occlusion mask and direct it toward the half-mirror. The SLM is configured to modulate the light incident thereon according to the two-dimensional shape of the occlusion mask. The OCOST-HMD is configured to generate an erect image, and the position of the user's pupil or a relay pupil is mapped to the position of the user's eyes to enable correct viewpoint disparity to be maintained.

[0106] Figure 10 1 illustrates a block diagram of a device 1000 that can be used to implement certain aspects of the disclosed technology. For example, Figure 10 The device 1000 may be used to receive, process, store, provide for display, and / or transmit various data and signals associated with the disclosed image sensors and / or microdisplays and SLMs for capturing and processing images, to enable control, display, storage, and processing of virtual content and occlusion masks, as well as brightness control, light modulation, or other operations associated with the electronic and optoelectronic components disclosed herein. The device 1000 includes at least one processor 1004 and / or controller, at least one memory 1002 unit in communication with the processor 1004, and at least one communication unit 1006 that enables the exchange of data and information with other entities, devices, databases, and networks, directly or indirectly, via a communication link 1008. The communication unit 1006 may provide wired and / or wireless communication capabilities according to one or more communication protocols, and thus may include appropriate transmitters / receivers, antennas, circuits, and ports, as well as encoding / decoding capabilities that may be necessary to properly transmit and / or receive data and other information. Figure 10 The exemplary device 1000 may be integrated as part of a larger component (eg, a server, a computer, a tablet, a smart phone, etc.) that may be used to perform various calculations, methods, or algorithms disclosed herein.

[0107] The processor(s) 1004 may include a central processing unit (CPU) to control the overall operation of, for example, a host computer. In certain embodiments, the processor(s) 1004 implements this by executing software or firmware stored in the memory 1002. The processor(s) 1004 may be or may include one or more programmable general-purpose or special-purpose microprocessors, digital signal processors (DSPs), programmable controllers, application-specific integrated circuits (ASICs), programmable logic devices (PLDs), graphics processing units (GPUs), etc., or a combination of these devices.

[0108] The memory 1002 may be or include the main memory of the computer system. The memory 1002 represents any suitable form of random access memory (RAM), read-only memory (ROM), flash memory, etc., or a combination of these devices. In use, the memory 1002 may also contain, among other things, a set of machine instructions that, when executed by the processor 1004, causes the processor 1004 to perform operations to implement certain aspects of the presently disclosed technology.

[0109] Although this patent document contains many details, these should not be construed as limitations on the scope of any invention or of what may be claimed, but rather as descriptions of features that may be specific to particular embodiments of particular inventions. Certain features described in this patent document in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as working in certain combinations and even initially claimed as such, in some cases one or more features from the claimed combination may be deleted from the combination, and the claimed combination may be directed to a subcombination or a variation of the subcombination.

[0110] Similarly, while operations may be depicted in a particular order in the drawings, this should not be understood as requiring that these operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, in order to achieve desired results. Furthermore, the separation of various system components in the embodiments described in this patent document should not be understood as requiring such separation in all embodiments.

[0111] It should be understood that the various disclosed embodiments can be implemented individually or collectively in devices including various optical components, electronic hardware and / or software modules and components. For example, these devices may include processors, memory units, interfaces that are communicatively connected to each other, and the range may range from desktop and / or laptop computers to mobile devices, etc. The processor and / or controller may perform various disclosed operations based on the execution of program code stored on a storage medium. For example, the processor and / or controller may communicate with at least one memory and with at least one communication unit that enables data and information to be exchanged directly or indirectly through a communication link with other entities, devices, and networks. The communication unit may provide wired and / or wireless communication capabilities according to one or more communication protocols, and therefore it may include appropriate transmitter / receiver antennas, circuits, and ports, as well as encoding / decoding capabilities that may be needed for correctly transmitting and / or receiving data and other information. For example, the processor may be configured to receive electrical signals or information from a disclosed sensor (e.g., a CMOS sensor) and process the received information to generate an image or other information of interest.

[0112] The various information and data processing operations described herein may be implemented in one embodiment by a computer program product embodied in a computer-readable medium, including computer-executable instructions, such as program code, executed by computers in a networked environment. Computer-readable media may include removable and non-removable storage devices, including but not limited to read-only memory (ROM), random access memory (RAM), compact disks (CDs), digital versatile disks (DVDs), and the like. Therefore, the computer-readable media described in this application include non-transitory storage media. Generally, program modules may include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. Computer-executable instructions, associated data structures, and program modules represent examples of program code for executing the steps of the methods disclosed herein. A specific sequence of such executable instructions or associated data structures represents an example of corresponding actions for implementing the functions described in such steps or processes.

[0113] Only a few implementations and examples are described, and other implementations, enhancements, and variations can be made based on what is described and illustrated in this patent document.

Claims

1. An optical see-through head mounted display OCOST-HMD with occlusion capability, comprising: a polarizing element configured to receive light from a real scene and produce polarized light at its output; Polarization beam splitter PBS; Objective lens; Spatial light modulator SLM; eyepiece; Quarter wave plate QWP; Microdisplay; as well as a reflective optical element configured to reflect substantially all or a portion of light incident thereon in a first direction and to transmit substantially all or a portion of light received from the microdisplay incident thereon from a second direction, wherein The SLM and the objective lens form a first double pass configuration that allows at least a portion of the light that has passed through the objective lens to be reflected from the SLM based on an occlusion mask and to propagate again through the objective lens, the occlusion mask corresponding to a virtual image presented on the microdisplay, and The eyepiece and the reflective optical element form a second double-pass configuration that allows at least a portion of the light that passes through the eyepiece to reflect from the reflective optical element and propagate through the eyepiece again.

2. The OCOST-HMD of claim 1, wherein: The PBS is positioned to receive polarized light and reflect the polarized light toward the objective lens, and The PBS is positioned to receive light output from the first dual pass configuration and transmit therethrough toward the eyepiece, and to reflect light received by the PBS from the second dual pass configuration, including light from the microdisplay, toward the position of a human eye.

3. The OCOST-HMD of claim 1, further comprising a first reflective surface, wherein: The PBS is positioned to: receiving polarized light and transmitting the polarized light therethrough toward said objective lens, receiving light output from the first double pass configuration and reflecting it to the eyepiece, Light received by the PBS from the second dual pass configuration, including light from the microdisplay, is reflected toward the first reflective surface, and wherein the first reflective surface is positioned to reflect light incident thereon toward a location of a human eye.

4. The OCOST-HMD according to claim 1, wherein: The SLM is configured to modulate light incident thereon in a switching modulation mode.

5. The OCOST-HMD according to claim 1, wherein: The occlusion mask is used to achieve modulation of one or more regions of the SLM.

6. The OCOST-HMD according to claim 1, wherein: The reflective optical element is positioned on a surface of the microdisplay.

7. The OCOST-HMD according to claim 1, wherein: The microdisplay includes an organic light emitting diode (OLED) device.

8. The OCOST-HMD according to claim 1, wherein: The QWP is positioned between the eyepiece and the reflective optical element.

9. The OCOST-HMD according to claim 1, wherein: The QWP is positioned between the eyepiece and the PBS.

10. The OCOST-HMD according to claim 1, wherein: The SLM includes a liquid crystal on silicon (LCoS) device.

11. The OCOST-HMD according to claim 1, wherein: The OCOST-HMD is configured to produce an erect image without using a roof prism.

12. The OCOST-HMD according to claim 1, wherein: The OCOST-HMD provides a pupil-matched optical configuration that maps the user's pupil or a relayed pupil back to the user's eye position so that the correct viewpoint disparity can be maintained.

13. The OCOST-HMD according to claim 1, wherein: The OCOST-HMD is configured to produce a field of view FOV that is not limited by the eyepiece in at least one direction.

14. The OCOST-HMD according to claim 1, wherein: The OCOST-HMD has an optical performance of a field of view FOV greater than 40 degrees diagonally and a modulation contrast greater than 20% over the full FOV.

15. The OCOST-HMD according to claim 1, wherein: The OCOST-HMD has a 90 degree by 40 degree see-through field of view FOV with an angular resolution of 1.0 arc minute.

16. The OCOST-HMD according to claim 1, wherein: At least a portion of the OCOST-HMD corresponds to a set of two afocal 4f relays that image the entrance pupil to a conjugate intermediate pupil position.

17. The OCOST-HMD according to claim 1, wherein: The OCOST-HMD forms a single-layer, double-pass, pupil-matched OCOST-HMD.

18. The OCOST-HMD of claim 1, comprising one or both of the following: (a) an objective lens assembly including the objective lens, or (b) An eyepiece set comprising the eyepiece.

19. An optical see-through head mounted display OCOST-HMD with occlusion capability, comprising: a polarizer that produces polarized light associated with the real scene; Beam splitter PBS; Objective lens; Spatial light modulator SLM; eyepiece; Delay device; Microdisplay; as well as a semi-reflective mirror configured to reflect substantially all light associated with the occlusion mask incident thereon in a first direction and to transmit substantially all light associated with the virtual scene incident thereon from a second direction, wherein PBS is positioned as: receiving polarized light and directing it toward the SLM, receiving light associated with a virtual scene and directing it toward a location for viewing by an eye of a user, and receiving light associated with an occlusion mask and directing it toward a semi-reflective mirror, wherein the occlusion mask corresponds to a virtual image presented on the microdisplay, The SLM is configured to modulate light incident thereon according to the two-dimensional shape of the occlusion mask, and The OCOST-HMD is configured to generate an erect image, and the position of the user's pupil or the relay's pupil is mapped to the position of the user's eye so that the correct viewpoint disparity can be maintained.

Images