Polarization Compensation for Wire Grid Polarizers in Head-Mounted Display Systems

Abstract

The present disclosure generally relates to techniques for improving the performance and efficiency of optical systems, such as those used in head-mounted display systems. The optical systems of the present disclosure may include polarized catadioptric systems, or "pancake optics," that utilize a wire grid polarizer as a reflective polarizer. Wire grid polarizers may not perform uniformly across wavelengths or varying angles of incidence. To improve performance, a spatially varying polarizer is provided in the optical system that acts to provide polarization compensation to the wire grid polarizer, causing it to perform more uniformly across wavelengths and / or angles of incidence (e.g., on-axis and off-axis). The spatially varying polarizer may be formed of a liquid crystal material, such as a multi-twist retarder.

Description

Technical Field

[0001] The present disclosure generally relates to optical systems, and more specifically to improving the efficiency and performance of optical systems for head-mounted display systems. [Description of Related Art]

[0002] One modern virtual reality ("VR") experience is generated using a head-mounted display ("HMD"), which can be tethered to a stationary computer (such as a personal computer ("PC"), laptop, or gaming console), combined with and / or integrated with a smartphone and / or its associated display, or made self-contained. Generally, an HMD is a display device worn on a user's head and has a small display device in front of one eye (monocular HMD) or each eye (binocular HMD). The display unit is typically miniaturized and can include, for example, a CRT, LCD, liquid crystal on silicon (LCoS), OLED device, or laser scan beam display. In a binocular HMD, it is possible to display different images for each eye. This function is used to display stereoscopic images.

[0003] The demand for higher-performance displays has been increasing with the development of smartphones, high-resolution televisions, and other electronic devices. Such demand is further increasing due to the growing popularity of virtual reality systems and augmented reality systems, particularly systems using HMDs. A virtual reality system typically completely covers both eyes of the wearer and substitutes a "virtual" reality for the actual visual field or physical visual field (or actual reality) in front of the wearer, while an augmented reality system typically provides a semi-transparent or transparent overlay of one or more screens in front of both eyes of the wearer such that the actual visual field is augmented with additional information, and in a mediated reality system as well, information combining real-world elements and virtual elements can be presented to the viewer.

[0004] However, such head-mounted displays that reduce the distance between the viewer's eyes and the display and often have a field of view that is not fully visible increase the performance requirements of the display in a way that cannot be met by conventional displays (it goes without saying that this is done at a cost-effective level). Microdisplays such as OLED microdisplays are much smaller than conventional displays, but there are further challenges. For example, in a microdisplay, a lens with a very short focal length is required. Furthermore, since the size of the user's eye pupil is fixed, the F-number of the lens of the HMD using the microdisplay decreases, which tends to increase the aberration of a specific lens system. Furthermore, the pixels of the microdisplay are small. The increase in the spatial resolution of the HMD optical system due to this further increases the challenges for the design and manufacture of such lenses for HMDs.

Summary of the Invention

[0005] A head-mounted display system may be summarized as including a display subsystem operable to generate an image including linearly polarized light and an optical system. The optical system includes a lens element that receives light from the display subsystem, the lens element including a partially reflective surface, a wire grid polarizer that receives light from the lens element, a quarter-wave plate disposed between the lens element and the wire grid polarizer, and a spatially variant polarizer disposed between the display subsystem and the lens element, the spatially variant polarizer having a delay characteristic that varies across the entire optical window of the spatially variant polarizer to compensate for off-axis light incident on the wire grid polarizer.

[0006] The spatially-varying polarizer may include a multi-twist retarder. The spatially-varying polarizer may provide a quarter-wavelength phase difference at the center of the optical window, and may gradually decrease the phase difference toward the peripheral portion of the optical window. The spatially-varying polarizer may provide an eighth-wavelength phase difference at the peripheral portion of the window. The spatially-varying polarizer may provide a first phase difference at the center of the optical window, and may provide a second phase difference at the peripheral portion of the optical window, and the second phase difference is smaller than the first phase difference. The phase difference of the spatially-varying polarizer may vary linearly or non-linearly across the entire optical window.

[0007] The head-mounted display system may further include a control circuit operably coupled to the spatially-varying polarizer, and the control circuit is operable to selectively adjust the phase difference provided by the spatially-varying polarizer.

[0008] The head-mounted display system may be summarized as including a display subsystem operable to generate an image including linearly polarized light, and an optical system. The optical system includes a quarter-wave plate that receives light from the display subsystem, a lens element that receives light from the quarter-wave plate and includes a partial reflection surface, a wire grid polarizer that receives light from the lens element, and a spatially-varying polarizer disposed between the lens element and the wire grid polarizer. The spatially-varying polarizer has a delay characteristic that varies across the entire optical window of the spatially-varying polarizer to compensate for off-axis light incident on the wire grid polarizer.

[0009] The spatially-varying polarizer may include a multi-twist retarder. The spatially-varying polarizer may provide a quarter-wavelength phase difference at the center of the optical window and may gradually reduce the phase difference toward the periphery of the optical window. The spatially-varying polarizer may provide an eighth-wavelength phase difference at the periphery of the window. The spatially-varying polarizer may provide a first phase difference at the center of the optical window and may provide a second phase difference at the periphery of the optical window, where the second phase difference is smaller than the first phase difference. The phase difference of the spatially-varying polarizer may vary linearly or non-linearly across the entire optical window.

[0010] The head-mounted display system may further include a control circuit operably coupled to the spatially-varying polarizer, the control circuit being operable to selectively adjust the phase difference provided by the spatially-varying polarizer.

[0011] The head-mounted display system may be summarized as including first and second near-to-eye display systems, each of the first and second near-to-eye display systems including a display subsystem operable to generate an image including linearly polarized light and an optical subsystem including a lens element that receives light from the display subsystem, the lens element including a partially reflective surface, a wire grid polarizer that receives light from the lens element, a quarter-wave plate disposed between the lens element and the wire grid polarizer, and a spatially-varying polarizer disposed between the display subsystem and the lens element, the spatially-varying polarizer having a delay characteristic that varies across the entire optical window of the spatially-varying polarizer to compensate for off-axis light incident on the wire grid polarizer.

[0012] The spatially-varying polarizer may include a multi-twist retarder. The spatially-varying polarizer may provide a quarter-wavelength phase difference at the center of the optical window and may gradually reduce the phase difference toward the periphery of the optical window.

[0013] The spatially-varying polarizer may introduce a quarter-wavelength phase difference at the peripheral portion of the window. The spatially-varying polarizer may introduce a first phase difference at the center of the optical window and a second phase difference at the peripheral portion of the optical window. The phase difference of the spatially-varying polarizer may vary linearly or non-linearly across the entire optical window.

Brief Description of the Drawings

[0014] In this drawing, like elements or operations are identified by like reference numerals. The sizes and relative positions of the elements in the drawing are not necessarily drawn to scale. For example, the shapes and angles of various elements are not necessarily drawn to scale, and some of these elements may be arbitrarily enlarged and arranged to improve the visibility of the drawing. Further, the specific shape of the drawn elements is not necessarily intended to convey any information about the actual shape of the specific elements, and may only be selected to make the drawing easier to recognize.

[0015]

Figure 1

[0016]

Figure 2

[0017]

Figure 3

[0018]

Figure 4

[0019]

Figure 5

[0020]

Figure 6

DETAILED DESCRIPTION OF THE INVENTION

[0021] In the following description, specific details are set forth in order to provide a thorough understanding of the various disclosed implementations. However, one skilled in the art will recognize that these implementations may be practiced without one or more of these specific details, or with other methods, components, materials, etc. In other instances, well-known structures associated with computer systems, server computers, and / or communication networks are not shown or described in detail to avoid unnecessarily obscuring the description of the implementations.

[0022] Unless the context requires otherwise, throughout this specification and the following claims, the word "comprising" is synonymous with "including" and is inclusive or non-limiting (i.e., it does not exclude additional, unrecited elements or method operations).

[0023] Throughout this specification, references to "one implementation" or "an implementation" mean that the particular features, structures, or characteristics described in connection with that implementation are included in at least one implementation. Thus, appearances of the phrases "in one implementation" or "in an implementation" throughout this specification are not necessarily all referring to the same implementation. Further, in one or more implementations, the particular features, structures, or characteristics may be combined in any suitable manner.

[0024] In this specification and the appended claims, the singular forms "a", "an", and "the" include plural referents unless the context clearly dictates otherwise. It should also be noted that the term "or" is generally used in its inclusive sense (i.e., "and / or") unless the context clearly dictates otherwise.

[0025] Each item and summary provided in this specification is for convenience only and does not interpret the scope or meaning of the implementation.

[0026] The present disclosure generally relates to techniques for improving the performance and efficiency of optical systems, such as optical systems using head-mounted display systems. The optical systems of the present disclosure may include a polarization reflection refractive optical system that utilizes a wire grid polarizer as a reflective polarizer, or a "pancake optical system". A wire grid polarizer may not function equally for wavelengths or varying angles of incidence. In at least some implementations of the present disclosure, a spatially variant polarizer is provided in an optical system that acts to provide polarization compensation for the wire grid polarizer, such that the wire grid polarizer functions more equally for wavelengths and / or angles of incidence (e.g., on-axis and off-axis). The spatially variant polarizer may be formed of a multi-twist retarder, as further described below.

[0027] First, with reference to FIGS. 1-4, an exemplary application of the head-mounted display device of the techniques described herein will be described. Next, with reference to FIGS. 5 and 6, an exemplary implementation of a display system including features of the present disclosure will be described. [Exemplary Head-Mounted Display System and Environment]

[0028] FIG. 1 is a schematic diagram of a networked environment 100 that includes a local media rendering (LMR) system 110 (e.g., a gaming console), and the system includes a local computing system 120 and a display device 180 (e.g., an HMD device with two display panels) suitable for performing at least some of the techniques described herein. In the illustrated embodiment of FIG. 1, the local computing system 120 is communicatively coupled to the display device 180 via a transmission link 115 (the display device may be wired or tethered, such as via one or more cables (cable 220) as illustrated in FIG. 2, or alternatively wirelessly connected). In other embodiments, the local computing system 120 may provide encoded display image data via a wired or wireless link to a panel display device (e.g., a TV, console, or monitor), in addition to or instead of the HMD device 180, each display device including one or more addressable pixel arrays. In various embodiments, the local computing system 120 may include a general-purpose computing system, a gaming console, a video stream processing device, a mobile computing device (e.g., a cellular phone, PDA, or other mobile device), a VR processing device or an AR processing device, or other computing systems.

[0029] In the illustrated embodiment, the local computing system 120 includes one or more hardware processors (e.g., a central processing unit or “CPU”) 125, a memory 130, various I / O (“input / output”) hardware components 127 (e.g., a keyboard, a mouse, one or more game controllers, speakers, microphones, IR transmitters and / or receivers, etc.), a video subsystem 140 including one or more dedicated hardware processors (e.g., a graphics processing unit or “GPU”) 144 and video memory (VRAM) 148, a computer-readable storage 150, and a network connection 160. Also, in the illustrated embodiment, one embodiment of the eye tracking subsystem 135 is executed in the memory 130 by using the CPU 125 and / or the GPU 144 to perform an automated process for implementing at least some of the described techniques, and the memory 130 may optionally further execute one or more other programs 133 (e.g., a game program that generates a displayed video or other image). As part of the automated process for implementing at least some of the techniques described herein, the eye tracking subsystem 135 and / or the program 133 executed in the memory 130 may store or retrieve various types of data including those within the data structure of an exemplary database in the storage 150. In this example, the data used may include various types of image data information within a database (“DB”) 154, various types of application data within the DB 152, various types of configuration data within the DB 157, and may also include additional information such as system data or other information.

[0030] In the illustrated embodiment, the LMR system 110 may also be communicatively connected to an exemplary network-accessible media content provider 190 that may further provide content to the LMR system 110 for display via one or more computer networks 101 and network links 102, in addition to, or instead of, the image generation program 133. The media content provider 190 may include one or more computing systems (not shown) having components similar to those of the local computing system 120, including one or more hardware processors, I / O components, local storage devices, and memory. For simplicity, some details of the network-accessible media content provider are not illustrated.

[0031] The display device 180 is illustrated as being different from and separate from the local computing system 120 in the illustrated embodiment of FIG. 1. However, in certain embodiments, it will be understood that some or all of the components of the local media rendering system 110 may be integrated or housed within one device, such as, for example, a portable gaming machine, a portable VR entertainment system, an HMD device, etc. In such embodiments, the transmission link 115 may include, for example, one or more system buses and / or video bus architectures.

[0032] As an example involving the processing performed locally by the local media rendering system 120, assume that the local computing system is a game computing system that includes one or more game applications in which application data 152 is executed by the CPU 125 using the memory 130, and in which various video frame display data is generated and / or processed by an image generation program 133 in cooperation with the GPU 144 of the video subsystem 140. To provide a high-quality game experience, a large amount of video frame data (corresponding to a high image resolution per video frame and a high "frame rate" of about 60 to 180 per second for such video frames) is generated by the local computing system 120 and provided to the display device 180 via a wired or wireless transmission link 115.

[0033] The computing system 120 and the display device 180 are merely examples and are not intended to limit the scope of the present disclosure. The computing system 120 may instead include a plurality of computing systems or devices that interact with each other, and may be connected to other devices not illustrated, such as via one or more networks such as the Internet, via the web, or via a private network (such as a mobile communication network, etc.). More generally, a computing system or other computing node may include any combination of hardware or software that can interact with each other to perform the types of functions described, including, but not limited to, desktop or other computers, gaming machines, database servers, network storage devices and other network devices, PDAs, mobile phones, wireless phones, pagers, electronic notebooks, Internet devices, television-based systems (such as using set-top boxes and / or personal / digital video recorders), and various other consumer products including appropriate communication capabilities. The display device 180 may similarly include one or more devices with one or more display panels of various types and forms, and optionally may include various other hardware components and / or software components.

[0034] Although various items are illustrated as being stored in and used from memory or storage, it will be understood that some or all of these items may be transferred between memory and other storage devices for memory management or data integrity purposes. Thus, in some embodiments, some or all of the described techniques may be performed by one or more processors or other configured hardware circuits or hardware including memory or storage (e.g., by executing software instructions of one or more software programs and / or by storing such software instructions and / or data structures), such as when constituted by one or more software programs and / or data structures. Some or all of the components, systems, and data structures may also be stored on a non-transitory computer-readable storage medium (e.g., as software instructions or structured data), such as a hard disk or flash drive or other non-volatile storage device, volatile memory or non-volatile memory (e.g., RAM), network storage device, or portable media article (e.g., DVD disk, CD disk, optical disk, etc.) readable by an appropriate drive or via an appropriate connection. The systems, components, and data structures may also, in some embodiments, be transmitted in various computer-readable transmission media including wireless-based media and wired / cable-based media as a generated data signal (e.g., as part of a carrier wave or other analog or digital propagated signal) and may take various forms (e.g., as part of one or multiplexed analog signals or as multiple individual digital packets or frames). Such computer program products may take other forms in other embodiments. Accordingly, the present invention may be practiced using other computer system configurations.

[0035] Figure 2 illustrates an exemplary environment 200 that provides a virtual reality display to a human user 206 using at least some of the techniques described, along with an exemplary HMD device 202 coupled to a video rendering computing system 204 via a tether connection 220 (or a wireless connection in other embodiments). The user wears the HMD device 202 and receives display information of a simulated environment different from the actual physical environment from the computing system 204 via the HMD device, and the computing system acts as an image rendering system that supplies an image of the simulated environment (e.g., an image generated by a game program and / or other software program running on the computing system) to the HMD device for display to the user. In this example, the user can further move around within a tracking volume 201 of the actual physical environment 200 and may further have one or more I / O ("input / output") devices (including the handheld controllers 208 and 210 in this example) that enable the user to further interact with the simulated environment.

[0036] In the illustrated example, the environment 200 may include one or more base stations 214 (two are shown labeled as base stations 214a and 214b) that may facilitate tracking of the HMD device 202 or the controllers 208 and 210. When the user moves locations or changes the orientation of the HMD device 202, the position of the HMD device is tracked, such as to enable the corresponding portion of the simulated environment to be displayed to the user wearing the HMD device, and the controllers 208 and 210 may use a similar technique for use in tracking the position of the controllers (and optionally for using information useful in determining or verifying the position of the HMD device). After the tracked position of the HMD device 202 is known, the corresponding information is transmitted to the computing system 204 via a tether 220 or wirelessly. The computing system uses the tracked position information to generate one or more next images of the simulated environment and display them to the user.

[0037] There are a number of different ways of performing position tracking that can be used in various implementations of the present disclosure, including, but not limited to, acoustic tracking, inertial tracking, magnetic tracking, optical tracking, combinations thereof, and the like.

[0038] In at least some implementations, the HMD device 202 may include one or more light receivers or sensors that can be used to implement the tracking functionality or other aspects of the present disclosure. For example, each base station 214 may sweep an optical signal across the entire tracking volume 201. Depending on the requirements of each particular implementation, each base station 214 may generate more than one optical signal. For example, for six degrees of freedom tracking, usually one base station 214 is sufficient, but for some embodiments, multiple base stations (e.g., base stations 214a, 214b) may be necessary or desirable to provide stable room-scale tracking for the HMD device and peripheral devices. In this example, the light receivers are incorporated into the HMD device 202 and / or other objects to be tracked (such as controllers 208 and 210). In at least some implementations, on each object to be tracked, the light receivers may be combined with accelerometers and gyroscope inertial measurement units (“IMUs”) to support low-latency sensor fusion.

[0039] In at least some implementations, each base station 214 includes two rotors that sweep a linear beam along a vertical axis across the entire tracking volume 201. At the start of each sweep cycle, the base station 214 may emit an omnidirectional optical pulse (referred to as a “synchronization signal”) that is visible to all sensors attached to the object to be tracked. In this way, each sensor calculates its unique angular position in the swept volume by measuring the duration between the synchronization signal and the beam signal. The distance and orientation of the sensors may be determined using multiple sensors attached to a single stationary object.

[0040] One or more sensors disposed on a tracking object (e.g., HMD device 202, controllers 208 and 210) may include optoelectronic devices capable of detecting modulated light from the rotor. In the case of visible light or near-infrared (NIR) light, silicon photodiodes and suitable amplifier / detector circuits may be used. Since environment 200 may include static signals and time-varying signals (optical noise) having a wavelength similar to that of the base station 214 signal, in at least some implementations, the light from the base station may be modulated in a manner that facilitates distinguishing from any interfering signals and / or facilitates selecting the sensor from any emission wavelength other than the wavelength of the base station signal.

[0041] Inside-out tracking is also a type of position tracking that can be used to track the position of HMD device 202 and / or other objects (e.g., controllers 208 and 210, tablet computer, smartphone). Inside-out tracking differs from outside-in tracking in that it determines the position of the HMD using the position of a camera or other sensor. In inside-out tracking, the camera or sensor is located on the HMD or the object being tracked, while in outside-out tracking, the camera or sensor is disposed at a fixed position within the environment.

[0042] HMDs that utilize inside-out tracking use one or more "looking-out" cameras to determine how their position is changing relative to the environment. As the HMD moves, the sensors re-align its position within the room, and the virtual environment responds in real time accordingly. This type of position tracking can be achieved with or without markers placed in the environment. The cameras placed on the HMD observe the features of the surrounding environment. When using markers, the markers are designed to be easily detected by the tracking system and are placed in specific areas. In "markerless" inside-out tracking, the HMD system uses prominent features (e.g., topography) that already exist in the environment to determine position and orientation. The algorithms of the HMD system identify specific images or shapes and use them to calculate the position of the device in space. Data from accelerometers and gyroscopes can also be used to enhance the accuracy of position tracking.

[0043] FIG. 3 shows information 300 for explaining a front view of an exemplary HMD device 344 when worn on the head of user 342. The HMD device 344 includes a front structure 343 that supports a front or forward camera 346 and a plurality of sensors 348a-348d (collectively 348) of one or more types. As one example, a light sensor that detects and uses light information emitted from some or all of the sensors 348, for example, one or more external devices (not shown, but for example, the base station 214 of FIG. 2), can help determine the position and orientation of the device 344 within the space. As shown, the forward camera 346 and sensors 348 are directed forward towards the actual scene or environment (not shown) in which user 342 operates the HMD device 344. The actual physical environment can include, for example, one or more objects (e.g., walls, ceilings, furniture, stairs, cars, trees, tracking markers, or any other type of object). The specific number of sensors 348 can be less than or more than the number of sensors shown. The HMD device 344 may further include one or more additional components that are not attached to the front structure (e.g., inside the HMD device), such as an IMU (inertial measurement unit) electronic device 347 that measures and reports specific forces, angular velocities, and / or magnetic fields surrounding the HMD device 344 (e.g., using a combination of an accelerometer and a gyroscope, and optionally a magnetometer). The HMD device may further include additional components not shown, including one or more display panels and optical lens systems that face the user's both eyes (not shown), and optionally, one or more built-in motors for changing the alignment or other positioning of one or both of the optical lens system and / or display panel within the HMD device. This will be described in more detail below with respect to FIG. 4.

[0044] The example of the illustrated HMD device 344 is supported on the head of user 342, based at least in part on one or more straps 345 attached to the housing of the HMD device 344 and extending wholly or partially around the user's head. Although not illustrated here, the HMD device 344 may further include one or more external motors attached to one or more of the straps 345, for example, and adjusting such a strap using such a motor to correct the alignment or other positioning of the HMD device on the user's head may be included in an auto-correction operation. The HMD device may include other support structures (e.g., nose pads, chin straps, etc.) not illustrated here, in addition to or instead of the illustrated straps, and some embodiments include one or more such other support structures to which motors are attached, and similarly, it will be understood that the shape and / or position thereof may be adjusted to correct the alignment or other positioning of the HMD device on the user's head. Other display devices not attached to the user's head may similarly be attached to one or more structures or portions thereof that affect the positioning of the display device, and such other devices may include motors or other mechanical actuators in at least some embodiments, and similarly, the shape and / or position thereof may be modified to correct the alignment or other positioning of the display device with respect to one or more pupils of one or more users of the display device.

[0045] Figure 4 illustrates a simplified plan view 400 of an HMD device 405 including a pair of near-to-eye display systems 402 and 404. The HMD device 405 may be the same as or similar to the HMD devices illustrated in FIGS. 1 - 3, or may be a different HMD device, and the HMD device described herein may further be used in the examples further described below. The near-to-eye display systems 402 and 404 in FIG. 4 each include a display panel 406 and 408 (e.g., an OLED microdisplay, an LCD display), and respective optical lens systems 410 and 412 each having one or more optical lenses. The display systems 402 and 404 may be installed in a housing (or frame) 414 or otherwise disposed therein, and the housing includes a front portion 416 (e.g., the same as or similar to the front surface 343 in FIG. 3), a left temple 418, a right temple 420, and an inner surface 421 that contacts or is proximate to the face of the user 424 who is the wearer when the user wears the HMD device. The two display systems 402 and 404 that can be worn on the head 422 of the user 424 who is the wearer may be fixed to the housing 414 in a glasses configuration, the left temple 418 and the right temple 420 may be placed over the user's ears 426 and 428 respectively, and the nose rest 492 may be placed over the user's nose 430. In the example of FIG. 4, the HMD device 405 may be supported on the user's head partially or entirely by the nose rest and / or the left and right over-ear type temples, but in some embodiments, as in the embodiments shown in FIGS. 2 and 3, a strap (not shown) or other structure may be used to fix the HMD device to the user's head. The housing 414 may be shaped and sized to respectively dispose each of the two optical lens systems 410 and 412 in front of one of the user's eyes 432 and 434, whereby the target position of each pupil 494 will be centered both vertically and horizontally in front of the respective optical lens system and / or display panel.Housing 414 is shown in a simplified manner similar to glasses for illustrative purposes, but it should be understood that in practice, a more elaborate structure (e.g., goggles, integrated headband, helmet, strap, etc.) can be used to support display systems 402 and 404 and position them on user 424's head 422.

[0046] The HMD device 405 of FIG. 4, and other HMD devices described herein, can present virtual reality displays to a user via a corresponding video presented at a display rate such as 30 or 60 or 90 frames (or images) per second. In other embodiments of similar systems, augmented reality displays may be presented to the user. The displays 406 and 408 of FIG. 4 may each generate light that is focused onto the eyes 432 and 434 of the user 424 by the respective optical lens systems 410 and 412 after being conveyed through those optical lens systems. The aperture of the pupil 494 of each eye allows light to enter the eye there through. Typically, however, the size of the pupil ranges from 2 millimeters (mm) in a very bright state to 8 mm in a dark state, and the size of the larger iris that includes the pupil can be approximately 12 mm. The pupil (and the surrounding iris) can also typically move a few millimeters horizontally and / or vertically within the visible portion of the eye when the eyelid is open, which means that when the eyeball rotates about its center (resulting in a three-dimensional volume in which the pupil can move), the pupil is also moved to different depths with respect to different horizontal and vertical positions relative to the optical lens of the display or other physical elements. The user 424 sees the light entering the user's pupil as an image and / or video. In some implementations, the distance between each of the optical lens systems 410 and 412 and the user's eyes 432 and 434 can be relatively short (e.g., less than 30 mm, less than 20 mm), which is advantageous because the weight of the optical lens system and the display system is relatively close to the user's face, so the HMD device feels lighter to the user and can also provide the user with a larger field of view. Although not illustrated here, some embodiments of such HMD devices may include various additional built-in sensors and / or external sensors.

[0047] In the illustrated embodiment, the HMD device 405 of FIG. 4 further includes hardware sensors and additional components, such as including one or more accelerometers and / or gyroscopes 490 (e.g., as part of one or more IMU units). As described in more detail elsewhere herein, values from the accelerometers and / or gyroscopes may be used to locally determine the orientation of the HMD device. Further, the HMD device 405 may include one or more front cameras, e.g., camera 485 on the outer surface of the front portion 416, and the information may be used as part of the operations of the HMD device, such as for providing an AR function or a positioning function. Further, the HMD device 405 may further include other components 475 (e.g., electronic circuits for controlling the display of images to the display panels 406 and 408, built-in storage, one or more batteries, a position tracking device for communicating with an external base station, etc.), which are described in more detail elsewhere herein. Other embodiments may not include one or more of the components 475, 485, and / or 490. Although not illustrated here, some embodiments of such HMD devices may include various additional built-in sensors and / or external sensors for tracking various other types of movements and positions of the user's body, eyes, controller, etc.

[0048] In the illustrated embodiment, the HMD device 405 of FIG. 4 further includes hardware sensors and additional components that can be used by the disclosed embodiments as part of the described techniques to determine the direction of the user's pupil or line of sight, which may be provided for use by one or more components associated with the HMD device. This is described elsewhere in this specification. The hardware sensors in this example include one or more eye-tracking assemblies 472 of an eye-tracking subsystem that are located on or near the display panels 406 and 408 and / or on the inner surface 421 near the optical lens systems 410 and 412 for use in obtaining information about the actual position of the user's pupils 494, for example, separately for each pupil in this example.

[0049] Each of the eye-tracking assemblies 472 may include one or more light sources (e.g., IR LEDs) and one or more light detectors (e.g., silicon photodiodes). Further, for clarity, although only a total of four eye-tracking assemblies 472 are shown in FIG. 4, it should be understood that in practice, a different number of eye-tracking assemblies may be provided. In some embodiments, a total of eight eye-tracking assemblies 472 are provided, i.e., four eye-tracking assemblies for each eye of the user 424. Further, in at least some implementations, each eye-tracking assembly includes a light source directed towards one of the user's eyes 432 and 434, a light detector arranged to receive light reflected by each of the user's eyes, and a polarizer arranged and configured so that light reflected by specular reflection does not reach the light detector.

[0050] As will be described in more detail elsewhere in this specification, information from the eye tracking assembly 472 may be used to determine and track the direction of the user's line of sight during use of the HMD device 405. Further, in at least some embodiments, the HMD device 405 may be used to move (e.g., in the vertical direction, the horizontal left-right direction, and / or the horizontal front-back direction) the alignment and / or other positioning of one or both of the optical lens systems 410 and 412 and / or the display panels 406 and 408 within the housing of the HMD device 405 (439) using one or more built-in motors 438 (or other movement mechanisms), which personalizes or otherwise adjusts the target pupil positions of one or both of the near-to-eye display systems 402 and 404 corresponding to the actual position of one or both of the pupils 494. Such a motor 438 may be controlled, for example, by the user operating one or more control buttons 437 on the housing 414 and / or by the user operating one or more associated separate I / O controllers (not shown). In other embodiments, the HMD device 405 may control the alignment and / or other positioning of the optical lens systems 410 and 412 and / or the display panels 406 and 408 using an adjustable positioning mechanism (e.g., a screw, a slider, a ratchet, etc.) that can be manually changed by the user using the control buttons 437 without using such a motor 438. Further, although FIG. 4 illustrates a motor 438 for only one of the near-to-eye display systems, in some embodiments, each near-to-eye display system may have its own one or more motors, and in some embodiments, one or more motors may be used to control each of the plurality of near-to-eye display systems (e.g., independently).

[0051] The described techniques may, in some embodiments, be used with a display system similar to that illustrated, but in other embodiments, other types of display systems may be used, including, for example, those having one optical lens and a display device, or those having a plurality of such optical lenses and display devices. Non-exclusive examples of other such devices include cameras, telescopes, microscopes, binoculars, spotting scopes, surveying scopes, etc. Further, the described techniques may be used with a wide variety of display panels or other display devices that emit light to form an image, which image is viewed by one or more users through one or more optical lenses, as described elsewhere herein. In other embodiments, the user may view, through one or more optical lenses, one or more images generated, in a manner other than a display panel, on a surface that partially or wholly reflects light from another light source (e.g., a laser scan beam). [Exemplary Display System]

[0052] FIG. 5 is a side cross-sectional view of a head-mounted display system 500 that includes a display system 502 and an optical system 504 supported by a support structure 506 (such as a housing, helmet, goggles, glasses, or other headgear). The support structure 506 supports the display system 502 and the optical system 504 in front of the user's eyes (e.g., eyes 510) when the user looks at the system in the Z-axis direction indicated by arrow 512 shown in FIG. 5. The control circuit 514 may optionally be coupled to one or more components of the optical system 504 or the display system 502, as described elsewhere herein. As an example, the display system 502 and the optical system 504 may be the same as or identical to the display system and the optical system described above with reference to FIG. 4. The display system 502 and the optical system 504 may be operable to display an image to the user 510. As further described below, the optical system 504 may utilize a reflective refractive optical system or a "pancake" optical system to provide an image from the display system 502 to the user's eyes 510. The spacing and size of each component of the head-mounted display system 400 may be different from those illustrated. As an example, each component shown as being spaced apart from each other may be arranged adjacent to each other in various orders and the like.

[0053] The display system 502 includes an image source such as a pixel array 516. The pixel array 516 may include a two-dimensional array of pixels that emit light. By way of non-limiting example, the pixel array 516 may include a liquid crystal display (LCD), a liquid crystal on silicon (LCoS) display, an organic light emitting diode (OLED) display, and the like.

[0054] A linear polarizer 518 may be disposed in front of the pixel array 516 to provide polarization from the pixel array 516. In at least some implementations, the pixel array 516 may be designed such that the linear polarizer 518 may be omitted, and the pixel array 516 may generate linear polarization. As an example, the linear polarizer 518 may have a transmission axis or pass axis aligned with the X axis shown in FIG. 5.

[0055] The optical system 504 may include a spatially-varying polarizer 520 disposed in front of the linear polarizer 518. As will be further described below, the spatially-varying polarizer 520 can provide a delay characteristic that varies across its optical window, whereby the spatially-varying polarizer can provide polarization compensation to the wire grid polarizer 522 of the optical system 502, so that variations in the wavelength and incident angle of the wire grid polarizer are minimized. The spatially-varying polarizer 520 may include a wavelength retarder formed of a birefringent material, such as a multi-twist retarder. Birefringence is a property of a material in which the refractive index depends on the polarization and propagation direction of light. The wavelength retarder changes the polarization state or phase of the light traveling through the wavelength retarder. The wavelength retarder may have a slow axis (or extraordinary axis) and a fast axis (ordinary axis). When polarized light travels through the wavelength retarder, the light along the fast axis travels faster than the light along the slow axis.

[0056] The spatially-varying polarizer 520 may be aligned such that its fast axis is aligned at 45° with respect to the transmission axis of the linear polarizer 518. The spatially-varying polarizer 520 may be disposed in front of the linear polarizer 518 and may optionally be attached to the linear polarizer.

[0057] The optical system 504 further includes a lens element including a lens portion 524 and a partial reflection mirror or surface 526. The lens portion 524 and the partial reflection mirror 526 may be formed as a single component or as a plurality of components. The optical system 504 further includes a quarter-wave plate 528 disposed between the lens portion 524 and the wire grid polarizer 522.

[0058] The lens unit 524, or the display system 502, or other components of the optical system 504 are shown as being planar for purposes of illustration, but may be non-planar (e.g., plano-convex, plano-concave, etc.). Further, the optical system 504 may include additional or fewer optical structures, such as refractive lenses or diffractive lenses, partial reflection films, wave plates, reflective polarizers, linear polarizers, anti-reflection films, additional spatially variant polarizers, or other optical structures that enable light rays from the display system 502 to be focused at the user's eye 510 with a desired light intensity.

[0059] A non-limiting example of how light may pass through the display system 502 and the optical system 504 of the head-mounted display system 500 is described herein with reference to light rays 530 - 544 (arrows) in FIG. 5. Image light ray 530 may exit from the pixel array 516 and pass through the linear polarizer 518, becoming linearly polarized light aligned with the transmission axis of the linear polarizer 518. In the illustrated example, the transmission axis of the linear polarizer 518 may be aligned with the X-axis shown in FIG. 5.

[0060] After passing through the linear polarizer 518, the light ray 530 passes through the spatially variant polarizer 520 and becomes circularly polarized. Further explanation of the action of the spatially variant polarizer 520 is provided elsewhere in this specification.

[0061] The circularly polarized light ray 530 from the spatially variant polarizer 520 strikes the partial reflection mirror 526, and a portion of the light passes through the partial reflection mirror as the light ray 532. The light ray 532 is refracted or diffracted (partially focused) by the shape or characteristics of the lens portion 524 of the lens element.

[0062] The light ray 532 is circularly polarized. The quarter-wave plate 528 converts the light ray 532 into a linearly polarized light ray 534 in which the linearly polarized light is aligned with the Y-axis of FIG. 5.

[0063] The wire grid polarizer 522 may be disposed proximate to or adjacent to the quarter-wave plate 528. The wire grid polarizer 522 may have orthogonal reflection and transmission (or passing) axes. Polarized light parallel to the reflection axis of the wire grid polarizer 522 is reflected by the wire grid polarizer, and polarized light parallel to the transmission axis passes through the wire grid polarizer 522. In the illustrated example, the wire grid polarizer 522 may have a reflection axis aligned with the Y-axis, such that the light ray 534 is reflected from the wire grid polarizer as the reflected light ray 536.

[0064] The reflected light ray 536 has linearly polarized light aligned with the Y-axis shown in FIG. 5. After passing through the quarter-wave plate 528, the reflected light ray 536 becomes a circularly polarized light ray 538. The circularly polarized light ray 538 passes through the lens portion 524, and a part of the light ray 538 is reflected by the partial reflection mirror 526 as the reflected light ray 540. A part of the light ray 538 that passes through the partial reflection mirror 526 as the transmitted light ray 542 is converted from circular polarization to linear polarization by the spatial-variation polarizer 520. Since the linearly polarized light has polarization aligned with the Y-axis shown in FIG. 5, the light ray 542 is absorbed by the linear polarizer 518.

[0065] As described above, the reflected light ray 540 is circularly polarized. After passing through the lens portion 524 and the quarter-wave plate 528 again, the polarization of the light ray 540 becomes linearly polarized as the light ray 544 aligned with the X-axis, which is parallel to the transmission axis of the wire grid polarizer 522. Therefore, the light ray 544 passes through the wire grid polarizer 522 and provides an image visible to the user's eye 510.

[0066] As described above, the spatially-varying polarizer 520 that provides polarization compensation for the wire grid polarizer 522 may provide a phase delay that varies according to position (e.g., a lateral position, a longitudinal position, a radial position) across the entire field of view (e.g., from on-axis to off-axis), which is sensitive to wavelength and angle of incidence. The particular manner in which the phase difference of the spatially-varying polarizer 520 varies may depend on the particular configuration and materials of the wire grid polarizer 520 or other components, e.g., the polarization state of the incident light, the angle of incidence, the materials, the shapes of the various components, etc.

[0067] FIG. 6 shows a non-limiting and exemplary plan view 600 of the spatially-varying polarizer 520, showing the phase difference pattern of this spatially-varying polarizer. In this example, the spatially-varying polarizer 520 is configured to provide a circular or quarter-wavelength (λ / 4) phase difference at the center 602 of the optical window, and this phase difference gradually (e.g., linearly, non-linearly) decreases towards the periphery 604 where the spatially-varying polarizer provides an elliptical or eighth-wavelength (λ / 8) phase difference. Generally, the spatially-varying polarizer 520 may provide a phase difference that varies according to position by any method, and the amount of the phase difference may be any value (e.g., λ / 20, λ / 10, λ / 8, λ / 4, λ, 2λ) that is operable to provide polarization compensation for the wire grid polarizer 520. Further, the amount of the phase difference may increase only in one or more directions, decrease only in one or more directions, or both increase and decrease may occur. The amount of the phase difference may vary continuously or gradually, or may vary in multiple steps (e.g., two steps, ten steps). The amount of the phase difference may vary according to any type of function, including, for example, a linear function, a polynomial function, an exponential function, a step function, other types of functions, or a combination thereof.

[0068] As described above, in at least some implementations, the spatially-varying polarizer 520 may be formed of a multi-twist retarder (MTR), which is a waveplate-like retardation film that provides an accurate and customized level of broadband retardation, narrowband retardation, or multiple-band retardation with a single thin film. More specifically, the MTR has two or more twisted liquid crystal (LC) layers with an alignment layer on a single substrate. Since the next LC layer is directly aligned by the previous layer, simple manufacturing is enabled, automatic interlayer alignment is achieved, and a monolithic film with a continuously varying optical axis is provided.

[0069] In at least some implementations, the controller 514 may be operably coupled to the spatially-varying polarizer 520 to selectively vary the spatially-dependent phase retardation of the spatially-varying polarizer for any desired configuration. In other words, the spatially-varying polarizer 520 may be selectively switchable. In such implementations, one or more thin-film transistor layers may be provided that enable the spatially-dependent phase retardation of the spatially-varying polarizer 520 to be selectively controlled by the controller 514 in any desired manner. The controller 514 can control the phase retardation at any desired frequency, for example, once only, periodically, at a frequency equal to or a fraction of the frame rate of the display system 502.

[0070] In at least some implementations, the positions of the quarter-wave plate 528 and the spatially-varying polarizer 520 may be exchanged. In at least some implementations, the quarter-wave plate 528 may be replaced with a spatially-varying polarizer similar or identical to the spatially-varying polarizer 520 such that the optical system 504 includes two (or more) spatially-varying polarizers.

[0071] By using the spatially variant polarizer described herein, there is a significant increase in the degree of freedom for optical designers to create optical systems with improved performance and efficiency, which in turn enables display systems that provide a better viewing experience, are cost-effective, smaller in size or weight, consume less power, and offer other advantages that will be apparent to those skilled in the art.

[0072] The various implementation examples described above may be combined to provide further implementation examples. Based on the above detailed description, these and other changes can be made to each implementation example. In general, in the following claims, the terms used should not be construed as limiting the claims to the specific implementation examples disclosed herein and in the claims, but rather the claims should be construed to include all possible implementation examples together with the full scope of equivalents to which such claims are entitled. Accordingly, the claims are not limited by the present disclosure.

Claims

1. A head-mounted display system comprising a display subsystem operable to generate an image including linearly polarized light, and an optical system , wherein the optical system includes a lens element that receives light from the display subsystem, the lens element including a partially reflective surface, a wire grid polarizer that receives light from the lens element, a quarter-wave plate disposed between the lens element and the wire grid polarizer, and a spatially-varying polarizer disposed between the display subsystem and the lens element, the spatially-varying polarizer having a delay characteristic that varies across the entire optical window of the spatially-varying polarizer to compensate for off-axis light incident on the wire grid polarizer ; the spatially-varying polarizer provides a quarter-wave phase difference at the center of the optical window and gradually decreases the quarter-wave phase difference radially toward the peripheral portion of the optical window. A head-mounted display system.

2. The head-mounted display system according to claim 1, wherein the spatially-varying polarizer includes a multi-twist retarder.

3. The head-mounted display system according to claim 2, wherein the spatially-varying polarizer provides an eighth-wave phase difference at the peripheral portion of the optical window.

4. The head-mounted display system according to any one of claims 1 to 3, wherein the phase difference of the spatially-varying polarizer varies linearly or non-linearly across the entire optical window.

5. The head-mounted display system according to any one of claims 1 to 4, further comprising a control circuit operably coupled to the spatially-varying polarizer, the control circuit being operable to selectively adjust the phase difference provided by the spatially-varying polarizer.

6. A head-mounted display system comprising a display subsystem operable to generate an image including linearly polarized light, and an optical system , wherein the optical system includes a quarter-wave plate that receives light from the display subsystem, a lens element that receives light from the quarter-wave plate, the lens element including a partially reflective surface, a wire grid polarizer that receives light from the lens element A spatially-varying polarizer disposed between the lens element and the wire grid polarizer, the spatially-varying polarizer having a retardation characteristic that varies across the entire optical window thereof in order to compensate for off-axis light incident on the wire grid polarizer and having a head-mounted display system, wherein the spatially-varying polarizer provides a quarter-wave phase difference at the center of the optical window and gradually decreases the quarter-wave phase difference radially toward the peripheral portion of the optical window **Claim 7** The head-mounted display system according to claim 6, wherein the spatially-varying polarizer includes a multi-twist retarder **Claim 8** The head-mounted display system according to claim 6, wherein the spatially-varying polarizer provides an eighth-wave phase difference at the peripheral portion of the optical window **Claim 9** The head-mounted display system according to any one of claims 6 to 8, wherein the phase difference of the spatially-varying polarizer varies linearly or non-linearly across the entire optical window **Claim 10** The head-mounted display system according to any one of claims 6 to 9, further comprising a control circuit operably coupled to the spatially-varying polarizer, the control circuit being operable to selectively adjust the phase difference provided by the spatially-varying polarizer **Claim 11** A head-mounted display system comprising first and second near-to-eye display systems, wherein each of the first and second near-to-eye display systems has a display subsystem operable to generate an image including linearly polarized light, and an optical subsystem and having wherein the optical subsystem includes a lens element that receives light from the display subsystem, the lens element including a partially reflective surface, a wire grid polarizer that receives light from the lens element, a quarter-wave plate disposed between the lens element and the wire grid polarizer, and a spatially-varying polarizer disposed between the display subsystem and the lens element, the spatially-varying polarizer having a retardation characteristic that varies across the entire optical window thereof in order to compensate for off-axis light incident on the wire grid polarizer and including A head-mounted display system in which the spatially-varying polarizer provides a quarter-wave retardation at the center of the optical window and gradually decreases the quarter-wave retardation radially toward the peripheral portion of the optical window.

12. The head-mounted display system according to claim 11, wherein the spatially-varying polarizer includes a multi-twist retarder.

13. The head-mounted display system according to claim 11, wherein the spatially-varying polarizer provides an eighth-wave retardation at the peripheral portion of the optical window.

14. The head-mounted display system according to any one of claims 11 to 13, wherein the retardation of the spatially-varying polarizer varies linearly or non-linearly across the entire optical window.

Images